Skip to content

[fix](outfile) duplicate SUCCESS file request#14550

Merged
yangzhg merged 1 commit intoapache:masterfrom
cambyzju:fix-1124
Nov 28, 2022
Merged

[fix](outfile) duplicate SUCCESS file request#14550
yangzhg merged 1 commit intoapache:masterfrom
cambyzju:fix-1124

Conversation

@cambyzju
Copy link
Contributor

@cambyzju cambyzju commented Nov 24, 2022

Proposed changes

Issue Number: close #xxx

Problem summary

Problem desc:
While we call SELECT xxx INTO OUTFILE, if we output data file success, we also write an empty SUCCESS file.
But now the problem is:

  1. if we write data file failed, we also write out an empty SUCCESS file;
  2. if we write data file success, we write out the empty SUCCESS file twice;

Why the problem:
After #5489, we refractor _close_file_writer, but not change the caller.
So we will create SUCCESS file once inside close function, and the other call while we deconstruct FileResultWriter.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@hello-stephen
Copy link
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.96 seconds
load time: 448 seconds
storage size: 17172276489 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221124111631_clickbench_pr_52159.html

Copy link
Member

@yangzhg yangzhg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yangzhg yangzhg merged commit 73a600f into apache:master Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants