Skip to content

Conversation

@cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

This is a followup of #32513

It's hard to keep the command execution name for DataFrameWriter, as the command logical plan is a bit messy (DS v1, file source and hive and different command logical plans) and sometimes it's hard to distinguish "insert" and "save".

However, DataFrameWriterV2 only produce v2 commands which are pretty clean. It's easy to keep the command execution name for them.

Why are the changes needed?

less breaking changes.

Does this PR introduce any user-facing change?

no

How was this patch tested?

N/A

@cloud-fan
Copy link
Contributor Author

cc @beliefer

@SparkQA
Copy link

SparkQA commented Jun 15, 2021

Test build #139817 has finished for PR 32919 at commit 85a9109.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 15, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44345/

@SparkQA
Copy link

SparkQA commented Jun 16, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44366/

@SparkQA
Copy link

SparkQA commented Jun 16, 2021

Test build #139838 has finished for PR 32919 at commit 297c43d.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 16, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44389/


assert(executedPlan.isInstanceOf[CommandResultExec])
executedPlan.asInstanceOf[CommandResultExec].commandPhysicalPlan match {
executedPlan match {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need this change?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. I got it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SparkQA
Copy link

SparkQA commented Jun 16, 2021

Test build #139859 has finished for PR 32919 at commit adc141d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

thanks for review, merging to master!

@cloud-fan cloud-fan closed this in 0c5a01a Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants