NoSuchMethodError: PartitionedFileUtil$.splitFiles when running with Spark 3.5.5

[andygrove]

Describe the bug

The signature of PartitionedFileUtil$.splitFiles changed in Spark 3.5.5 so Comet is not compatible.

We need to add a shim to fix this.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

[andygrove]

I see these errors when trying to compile against Spark 3.5.5:

[ERROR] /home/andy/git/apache/datafusion-comet/spark/src/main/spark-3.5/org/apache/spark/sql/comet/shims/ShimCometScanExec.scala:62: not enough arguments for method getPartitionedFile: (file: org.apache.spark.sql.execution.datasources.FileStatusWithMetadata, filePath: org.apache.hadoop.fs.Path, partitionValues: org.apache.spark.sql.catalyst.InternalRow)org.apache.spark.sql.execution.datasources.PartitionedFile.
Unspecified value parameter partitionValues.
[ERROR] /home/andy/git/apache/datafusion-comet/spark/src/main/spark-3.5/org/apache/spark/sql/comet/shims/ShimCometScanExec.scala:70: not enough arguments for method splitFiles: (sparkSession: org.apache.spark.sql.SparkSession, file: org.apache.spark.sql.execution.datasources.FileStatusWithMetadata, filePath: org.apache.hadoop.fs.Path, isSplitable: Boolean, maxSplitBytes: Long, partitionValues: org.apache.spark.sql.catalyst.InternalRow)Seq[org.apache.spark.sql.execution.datasources.PartitionedFile].
Unspecified value parameter partitionValues.