[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed #41946

sollhui · 2024-10-16T07:32:05Z

Set NumberLoadedRows to zero when stream load failed.

before:

stream load result: {
    "TxnId": 8589,
    "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 3,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 2,
    "NumberUnselectedRows": 0,
    "LoadBytes": 77,
    "LoadTimeMs": 78,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 4,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 72,
    "ReceiveDataTimeMs": 7,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "XXX"
}

now:

stream load result: {
    "TxnId": 8589,
    "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 3,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 2,
    "NumberUnselectedRows": 0,
    "LoadBytes": 77,
    "LoadTimeMs": 78,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 4,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 72,
    "ReceiveDataTimeMs": 7,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "XXX"
}

doris-robot · 2024-10-16T07:32:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

sollhui · 2024-10-16T07:32:13Z

run buildall

github-actions · 2024-10-16T07:37:20Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-16T08:17:05Z

TeamCity be ut coverage result:
Function Coverage: 37.46% (9707/25914)
Line Coverage: 28.72% (80586/280606)
Region Coverage: 28.17% (41705/148071)
Branch Coverage: 24.75% (21206/85698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e27fb14686155532e8120083b1356c073ef8ceba_e27fb14686155532e8120083b1356c073ef8ceba/report/index.html

github-actions · 2024-10-17T02:29:04Z

clang-tidy review says "All clean, LGTM! 👍"

sollhui · 2024-10-17T14:53:06Z

run buildall

github-actions · 2024-10-17T14:57:04Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-17T15:29:23Z

TeamCity be ut coverage result:
Function Coverage: 37.47% (9711/25919)
Line Coverage: 28.73% (80609/280614)
Region Coverage: 28.17% (41715/148085)
Branch Coverage: 24.75% (21210/85698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c2b89b5c25c297d5785139c1be2db583b699c502_c2b89b5c25c297d5785139c1be2db583b699c502/report/index.html

github-actions · 2024-10-18T03:14:15Z

clang-tidy review says "All clean, LGTM! 👍"

sollhui · 2024-10-18T04:26:48Z

run buildall

doris-robot · 2024-10-18T05:54:06Z

TeamCity be ut coverage result:
Function Coverage: 37.46% (9709/25916)
Line Coverage: 28.74% (80636/280575)
Region Coverage: 28.17% (41718/148075)
Branch Coverage: 24.75% (21209/85688)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5df736d724319a5fca21e6c91608e866dbe07b8d_5df736d724319a5fca21e6c91608e866dbe07b8d/report/index.html

sollhui · 2024-10-18T06:40:12Z

run buildall

github-actions · 2024-10-18T06:43:44Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-18T07:18:26Z

TeamCity be ut coverage result:
Function Coverage: 37.47% (9710/25916)
Line Coverage: 28.73% (80611/280575)
Region Coverage: 28.17% (41717/148075)
Branch Coverage: 24.75% (21212/85688)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cf38e83ba3ba29312f731c358f8a21b81af202ac_cf38e83ba3ba29312f731c358f8a21b81af202ac/report/index.html

liaoxin01

LGTM

github-actions · 2024-10-18T09:29:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-10-18T09:29:27Z

PR approved by anyone and no changes requested.

sollhui · 2024-10-18T13:11:43Z

run buildall

github-actions · 2024-10-18T13:16:04Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-10-18T13:49:03Z

TeamCity be ut coverage result:
Function Coverage: 37.46% (9709/25916)
Line Coverage: 28.74% (80626/280546)
Region Coverage: 28.17% (41706/148048)
Branch Coverage: 24.75% (21202/85660)
Coverage Report: http://coverage.selectdb-in.cc/coverage/72da74402f9759702aa44fee0bb59d0b2817eeed_72da74402f9759702aa44fee0bb59d0b2817eeed/report/index.html

github-actions · 2024-10-21T02:53:49Z

PR approved by at least one committer and no changes requested.

dataroaring

LGTM

…transaction failed (#41946) (#42291) pick (#41946)

…transaction failed (#41946) (#42265) pick (#41946) Set NumberLoadedRows to zero when stream load failed. before: ``` stream load result: { "TxnId": 8589, "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003", "Comment": "", "TwoPhaseCommit": "false", "Status": "Fail", "Message": "[DATA_QUALITY_ERROR]too many filtered rows", "NumberTotalRows": 3, "NumberLoadedRows": 1, "NumberFilteredRows": 2, "NumberUnselectedRows": 0, "LoadBytes": 77, "LoadTimeMs": 78, "BeginTxnTimeMs": 0, "StreamLoadPutTimeMs": 4, "ReadDataTimeMs": 0, "WriteDataTimeMs": 72, "ReceiveDataTimeMs": 7, "CommitAndPublishTimeMs": 0, "ErrorURL": "XXX" } ``` now: ``` stream load result: { "TxnId": 8589, "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003", "Comment": "", "TwoPhaseCommit": "false", "Status": "Fail", "Message": "[DATA_QUALITY_ERROR]too many filtered rows", "NumberTotalRows": 3, "NumberLoadedRows": 0, "NumberFilteredRows": 2, "NumberUnselectedRows": 0, "LoadBytes": 77, "LoadTimeMs": 78, "BeginTxnTimeMs": 0, "StreamLoadPutTimeMs": 4, "ReadDataTimeMs": 0, "WriteDataTimeMs": 72, "ReceiveDataTimeMs": 7, "CommitAndPublishTimeMs": 0, "ErrorURL": "XXX" } ```

…41946)

…pache#41946)

set NumberLoadedRows to zero when stream load failed

e27fb14

sollhui changed the title ~~[chore](stream load) set NumberLoadedRows to zero when stream load failed~~ [improve](stream load) set NumberLoadedRows to zero when stream load failed Oct 16, 2024

sollhui marked this pull request as draft October 16, 2024 08:32

sollhui closed this Oct 16, 2024

sollhui reopened this Oct 17, 2024

sollhui changed the title ~~[improve](stream load) set NumberLoadedRows to zero when stream load failed~~ [improve](stream load) set NumberLoadedRows to zero when stream load transaction failed Oct 17, 2024

Merge branch 'master' into stream_load_failed

7ae5c85

sollhui marked this pull request as ready for review October 17, 2024 14:53

liaoxin01 previously approved these changes Oct 18, 2024

View reviewed changes

liaoxin01 added dev/2.1.x dev/3.0.x labels Oct 18, 2024

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 18, 2024

github-actions bot added the reviewed label Oct 18, 2024

fix test

72da744

sollhui dismissed liaoxin01’s stale review via 72da744 October 18, 2024 13:11

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Oct 18, 2024

liaoxin01 approved these changes Oct 21, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 21, 2024

dataroaring approved these changes Oct 21, 2024

View reviewed changes

dataroaring merged commit d9f6fe3 into apache:master Oct 21, 2024

This was referenced Oct 22, 2024

[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed (#41946) #42265

Merged

[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed (#41946) #42291

Merged

liaoxin01 pushed a commit that referenced this pull request Oct 23, 2024

[improve](stream load) set NumberLoadedRows to zero when stream load …

7630384

…transaction failed (#41946) (#42291) pick (#41946)

liaoxin01 added dev/3.0.3-merged and removed dev/3.0.x labels Oct 23, 2024

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Oct 24, 2024

dataroaring added the kind/behavior-changed label Oct 28, 2024

gavinchou mentioned this pull request Nov 26, 2024

Release Note 3.0.3 #44522

Closed

BiteTheDDDDt pushed a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Feb 7, 2025

(selectdb-cloud) Fix stream load smoke case due to change of (apache#…

acfbbf1

…41946)

BiteTheDDDDt pushed a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Feb 7, 2025

(selectdb-cloud) Fix again stream load smoke case due to change of (a…

8e619c0

…pache#41946)

[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed #41946

[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed #41946

Uh oh!

Conversation

sollhui commented Oct 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Oct 16, 2024

Uh oh!

sollhui commented Oct 16, 2024

Uh oh!

github-actions bot commented Oct 16, 2024

Uh oh!

doris-robot commented Oct 16, 2024

Uh oh!

github-actions bot commented Oct 17, 2024

Uh oh!

sollhui commented Oct 17, 2024

Uh oh!

github-actions bot commented Oct 17, 2024

Uh oh!

doris-robot commented Oct 17, 2024

Uh oh!

github-actions bot commented Oct 18, 2024

Uh oh!

sollhui commented Oct 18, 2024

Uh oh!

doris-robot commented Oct 18, 2024

Uh oh!

sollhui commented Oct 18, 2024

Uh oh!

github-actions bot commented Oct 18, 2024

Uh oh!

doris-robot commented Oct 18, 2024

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 18, 2024

Uh oh!

github-actions bot commented Oct 18, 2024

Uh oh!

sollhui commented Oct 18, 2024

Uh oh!

github-actions bot commented Oct 18, 2024

Uh oh!

doris-robot commented Oct 18, 2024

Uh oh!

github-actions bot commented Oct 21, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sollhui commented Oct 16, 2024 •

edited

Loading