Based on the current validation framework, I propose to make the changes:
- make
doValidate return a structure which contains the fallback reason
- put the reason into the vanilla spark node tag
- add a rule to collect fallback reason for the whole plan, the text view whose style like the
explain formatted. It shows the fallback reason with the corresponding vanilla spark node
- post event to spark listener bus. note that, in AQE, each stage will post a new event. The behavior is similar with
SparkListenerSQLAdaptiveExecutionUpdate
- add a ui tab show the event, the style should follow the sql execution tab
An example of the fallback text view:
== Physical Plan ==
* Project (4)
+- * ColumnarToRow (3)
+- LoadArrowData (2)
+- Scan parquet default.t1 (1)
(1) Scan parquet default.t1
Fallback reason:
Velox backend parquet scan does not support map type ...
(2) LoadArrowData
(3) ColumnarToRow [codegen id : 1]
(4) Project [codegen id : 1]
Based on the current validation framework, I propose to make the changes:
doValidatereturn a structure which contains the fallback reasonexplain formatted. It shows the fallback reason with the corresponding vanilla spark nodeSparkListenerSQLAdaptiveExecutionUpdateAn example of the fallback text view: