Skip to content

native_datafusion: CometNativeException not converted to SparkRuntimeException for case-sensitivity error #3719

@andygrove

Description

@andygrove

Description

When running Spark SQL tests with spark.comet.scan.impl=native_datafusion, the FileBasedDataSourceSuite test "Spark native readers should respect spark.sql.caseSensitive" fails with:

java.lang.ClassCastException: class org.apache.comet.CometNativeException cannot be cast to class org.apache.spark.SparkRuntimeException

The test expects a SparkRuntimeException when case-sensitive mode is enabled and the query references a column with different casing. DataFusion produces a CometNativeException instead.

Affected Test

  • FileBasedDataSourceSuite: "Spark native readers should respect spark.sql.caseSensitive - parquet"

Expected Behavior

The native error should be converted to a SparkRuntimeException so Spark error handling works correctly.

Parent Issue

Split from #3311.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions