-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-35378][SQL][FOLLOWUP] Restore the command execution name for DataFrameWriterV2 #32919
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
cc @beliefer |
|
Test build #139817 has finished for PR 32919 at commit
|
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
|
Test build #139838 has finished for PR 32919 at commit
|
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
|
|
||
| assert(executedPlan.isInstanceOf[CommandResultExec]) | ||
| executedPlan.asInstanceOf[CommandResultExec].commandPhysicalPlan match { | ||
| executedPlan match { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why we need this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh. I got it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So I reverted these changes from your previous commit.
|
Test build #139859 has finished for PR 32919 at commit
|
|
thanks for review, merging to master! |
What changes were proposed in this pull request?
This is a followup of #32513
It's hard to keep the command execution name for
DataFrameWriter, as the command logical plan is a bit messy (DS v1, file source and hive and different command logical plans) and sometimes it's hard to distinguish "insert" and "save".However,
DataFrameWriterV2only produce v2 commands which are pretty clean. It's easy to keep the command execution name for them.Why are the changes needed?
less breaking changes.
Does this PR introduce any user-facing change?
no
How was this patch tested?
N/A