[SPARK-47470][SQL][TESTS] Ignore IntentionallyFaultyConnectionProvider error in CliSuite

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to make CliSuite ignore IntentionallyFaultyConnectionProvider error.

Why are the changes needed?

Sometime, hive-thriftserver CliSuite works correctly but failed due to the following because it matches Error: pattern.

spark/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala

Line 89 in bc378f4

errorResponses: Seq[String] = Seq("Error:"),

Caused by: java.lang.RuntimeException: Failed with error line 'java.util.ServiceConfigurationError: org.apache.spark.sql.jdbc.JdbcConnectionProvider: Provider org.apache.spark.sql.execution.datasources.jdbc.connection.IntentionallyFaultyConnectionProvider could not be instantiated'
	at org.apache.spark.sql.hive.thriftserver.CliSuite.$anonfun$runCliWithin$8(CliSuite.scala:164)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at org.apache.spark.sql.hive.thriftserver.CliSuite.captureOutput$1(CliSuite.scala:161)
	at org.apache.spark.sql.hive.thriftserver.CliSuite.$anonfun$runCliWithin$10(CliSuite.scala:178)
	at org.apache.spark.sql.hive.thriftserver.CliSuite.$anonfun$runCliWithin$10$adapted(CliSuite.scala:178)
	at scala.sys.process.BasicIO$.$anonfun$processFully$1(BasicIO.scala:210)
	at scala.sys.process.BasicIO$.$anonfun$processFully$1$adapted(BasicIO.scala:188)
	at org.apache.spark.ProcessTestUtils$ProcessOutputCapturer.run(ProcessTestUtils.scala:30)

IntentionallyFaultyConnectionProvider is a test resource from sql module which is not used in CliSuite.

spark/sql/core/src/test/resources/META-INF/services/org.apache.spark.sql.jdbc.JdbcConnectionProvider

Line 18 in bc378f4

org.apache.spark.sql.execution.datasources.jdbc.connection.IntentionallyFaultyConnectionProvider

Does this PR introduce any user-facing change?

No, this is a test-only fix.

How was this patch tested?

Pass the CIs.

$ build/sbt "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.CliSuite" -Phive-thriftserver
...
[info] CliSuite:
[info] - load warehouse dir from hive-site.xml (4 seconds, 415 milliseconds)
[info] - load warehouse dir from --hiveconf (4 seconds, 342 milliseconds)
[info] - load warehouse dir from --conf spark(.hadoop).hive.* (8 seconds, 450 milliseconds)
[info] - load warehouse dir from spark.sql.warehouse.dir (4 seconds, 254 milliseconds)
[info] - Simple commands (5 seconds, 765 milliseconds)
[info] - Single command with -e (3 seconds, 413 milliseconds)
[info] - Single command with --database (8 seconds, 120 milliseconds)
[info] - Commands using SerDe provided in --jars (5 seconds, 914 milliseconds)
[info] - SPARK-29022: Commands using SerDe provided in --hive.aux.jars.path (5 seconds, 814 milliseconds)
[info] - SPARK-11188 Analysis error reporting (3 seconds, 774 milliseconds)
[info] - SPARK-11624 Spark SQL CLI should set sessionState only once (2 seconds, 847 milliseconds)
[info] - list jars (3 seconds, 814 milliseconds)
[info] - list jar <jarfile> (3 seconds, 807 milliseconds)
[info] - list files (3 seconds, 843 milliseconds)
[info] - list file <filepath> (3 seconds, 943 milliseconds)
[info] - apply hiveconf from cli command (3 seconds, 900 milliseconds)
[info] - Support hive.aux.jars.path (4 seconds, 814 milliseconds)
[info] - SPARK-28840 test --jars command (4 seconds, 472 milliseconds)
[info] - SPARK-28840 test --jars and hive.aux.jars.path command (4 seconds, 725 milliseconds)
[info] - SPARK-29022 Commands using SerDe provided in ADD JAR sql (5 seconds, 831 milliseconds)
[info] - SPARK-26321 Should not split semicolon within quoted string literals (4 seconds, 290 milliseconds)
[info] - Pad Decimal numbers with trailing zeros to the scale of the column (4 seconds, 177 milliseconds)
[info] - SPARK-30049 Should not complain for quotes in commented lines (4 seconds, 208 milliseconds)
[info] - SPARK-31102 spark-sql fails to parse when contains comment (4 seconds, 154 milliseconds)
[info] - SPARK-30049 Should not complain for quotes in commented with multi-lines (4 seconds, 44 milliseconds)
[info] - SPARK-31595 Should allow unescaped quote mark in quoted string (8 seconds, 286 milliseconds)
[info] - SparkException with root cause will be printStacktrace (6 seconds, 818 milliseconds)
[info] - SPARK-30808: use Java 8 time API in Thrift SQL CLI by default (4 seconds, 206 milliseconds)
[info] - SPARK-33100: Ignore a semicolon inside a bracketed comment in spark-sql (4 seconds, 349 milliseconds)
[info] - SPARK-33100: test sql statements with hint in bracketed comment (4 seconds)
[info] - SPARK-35086: --verbose should be passed to Spark SQL CLI (4 seconds, 146 milliseconds)
[info] - SPARK-35102: Make spark.sql.hive.version meaningful and not deprecated (6 seconds, 992 milliseconds)
[info] - SPARK-37471: spark-sql support nested bracketed comment  (3 seconds, 542 milliseconds)
[info] - SPARK-37555: spark-sql should pass last unclosed comment to backend (4 seconds, 123 milliseconds)
[info] - SPARK-37694: delete [jar|file|archive] shall use spark sql processor (3 seconds, 436 milliseconds)
20:22:09.922 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20:22:10.463 WARN org.apache.spark.SparkContext: Using an existing SparkContext; some configuration may not take effect.
20:22:11.882 WARN org.apache.hadoop.hive.metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
20:22:11.882 WARN org.apache.hadoop.hive.metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore dongjoon@127.0.0.1
[info] - SPARK-37906: Spark SQL CLI should not pass final comment (2 seconds, 132 milliseconds)
[info] - SPARK-39068: support in-memory catalog and running concurrently (3 seconds, 570 milliseconds)
[info] - formats of error messages (19 seconds, 445 milliseconds)
[info] - SPARK-35242: Support change catalog default database for spark (7 seconds, 752 milliseconds)
[info] - SPARK-42448: Print correct database in prompt (7 seconds, 788 milliseconds)
[info] - SPARK-42823: multipart identifier support for specify database by --database option (6 seconds, 776 milliseconds)
20:22:57.343 WARN org.apache.spark.sql.hive.thriftserver.CliSuite:

===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.hive.thriftserver.CliSuite, threads: derby.rawStoreDaemon (daemon=true), BoneCP-pool-watch-thread (daemon=true), com.google.common.base.internal.Finalizer (daemon=true), executor-state-sync-failure-handler (daemon=true), Timer-38 (daemon=true), BoneCP-keep-alive-scheduler (daemon=true) =====
[info] Run completed in 3 minutes, 35 seconds.
[info] Total number of tests run: 41
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 41, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 266 s (04:26), completed Mar 19, 2024, 8:22:57 PM

Was this patch authored or co-authored using generative AI tooling?

No.

[yaooqinn]

+1, LGTM

[dongjoon-hyun]

Thank you, @yaooqinn !

[dongjoon-hyun]

I tested locally and pasted the result to the PR description. Let me merge this.

Merged to master for Apache Spark 4.0.0.