What is the problem the feature request solves?
We can't do this right away because the Iceberg integration depends on native_comet for now, but we should start working towards removing the native_comet scan implementation.
Motivation:
The first step will be to review all of the tests where we currently explicitly set the scan implementation to native_comet. For each test we need to consider:
- Can we update the test such that it passes when using the default scan implementation?
- Is the test specific to
native_comet and can therefore just be removed (when we stop supporting native_comet)?
- Does the test fail with the default scan because there is a bug or missing feature? If so, can we just fall back to Spark?
We need to implement the following before removing native_comet:
Describe the potential solution
No response
Additional context
No response
What is the problem the feature request solves?
We can't do this right away because the Iceberg integration depends on
native_cometfor now, but we should start working towards removing thenative_cometscan implementation.Motivation:
The first step will be to review all of the tests where we currently explicitly set the scan implementation to
native_comet. For each test we need to consider:native_cometand can therefore just be removed (when we stop supportingnative_comet)?We need to implement the following before removing
native_comet:autoscan mode for Spark 4.0.0 #1967native_cometscan (remove fromautoscan mode) #2186native_iceberg_compatscan #2189native_iceberg_compatfor reading encrypted Parquet files #2202Describe the potential solution
No response
Additional context
No response