Skip to content

[VL] Make fallback details visualize and analyzable #2139

@ulysses-you

Description

@ulysses-you

Based on the current validation framework, I propose to make the changes:

  1. make doValidate return a structure which contains the fallback reason
  2. put the reason into the vanilla spark node tag
  3. add a rule to collect fallback reason for the whole plan, the text view whose style like the explain formatted. It shows the fallback reason with the corresponding vanilla spark node
  4. post event to spark listener bus. note that, in AQE, each stage will post a new event. The behavior is similar with SparkListenerSQLAdaptiveExecutionUpdate
  5. add a ui tab show the event, the style should follow the sql execution tab

An example of the fallback text view:

== Physical Plan ==
* Project (4)
+- * ColumnarToRow (3)
   +- LoadArrowData (2)
      +-  Scan parquet default.t1 (1)


(1) Scan parquet default.t1
Fallback reason:
Velox backend parquet scan does not support map type ...

(2) LoadArrowData

(3) ColumnarToRow [codegen id : 1]

(4) Project [codegen id : 1]

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions