Skip to content

Conversation

@xy720
Copy link
Member

@xy720 xy720 commented Aug 17, 2023

Proposed changes

For some certain bugs, jobs is stuck in FE by the table state. For example, There is a bug which causes table remains ROLLUP state after adding rollup job, then other alter jobs later will not succeed because the table state is always ROLLUP but not NORMAL.

This commit adds a statement which is used to set the state of the specified table.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@xy720
Copy link
Member Author

xy720 commented Aug 18, 2023

run buildall

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.55 seconds
stream load tsv: 537 seconds loaded 74807831229 Bytes, about 132 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162333755 Bytes

@xy720
Copy link
Member Author

xy720 commented Aug 18, 2023

run p0

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add regression test case.
You can set a table's state to , eg, rollup and set it back to normal


checkProperties();
// only support changing table state to normal
// if (tableState != OlapTableState.NORMAL) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why comment out this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useless code. It has been removed

@SerializedName(value = "table")
private String table;
@Getter
@SerializedName(value = "tabletId")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@SerializedName(value = "tabletId")
@SerializedName(value = "state")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@xy720
Copy link
Member Author

xy720 commented Aug 22, 2023

run buildall

@xy720
Copy link
Member Author

xy720 commented Aug 22, 2023

regression test has been added, please cc @morningman

@xy720
Copy link
Member Author

xy720 commented Aug 22, 2023

run compile

@xy720
Copy link
Member Author

xy720 commented Aug 23, 2023

run buildall

@xy720 xy720 force-pushed the admin-set-table-status branch from 2498360 to 68c240d Compare August 23, 2023 06:25
@xy720
Copy link
Member Author

xy720 commented Aug 23, 2023

run buildall

@xy720
Copy link
Member Author

xy720 commented Aug 23, 2023

run buildall

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.95 seconds
stream load tsv: 543 seconds loaded 74807831229 Bytes, about 131 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17161867418 Bytes

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 26, 2023
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@chenlinzhong chenlinzhong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xy720 xy720 merged commit eadffed into apache:master Aug 28, 2023
xiaokang pushed a commit that referenced this pull request Aug 30, 2023
For some certain bugs, jobs is stuck in FE by the table state. For example, There is a bug which causes table remains ROLLUP state after adding rollup job, then other alter jobs later will not succeed because the table state is always ROLLUP but not NORMAL.

This commit adds a statement which is used to set the state of the specified table.
@xiaokang
Copy link
Contributor

revert this PR in branch-2.0 since it modify fe metadata OperationType and cause rollback from 2.0.2 to 2.0.1/2.0.0

xiaokang added a commit that referenced this pull request Sep 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. merge_conflict reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants