Skip to content

[fix](plan) only scan node with limit and no predicate can reduce to 1 instance #31342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morningman merged 1 commit into from
Feb 23, 2024
Merged

[fix](plan) only scan node with limit and no predicate can reduce to 1 instance #31342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1409,6 +1409,8 @@ public boolean getShouldColoScan() {
// If scan is key search, should not enable the shared scan opt to prevent the performance problem
// 1. where contain the eq or in expr of key column slot
// 2. key column slot is distribution column and first column
// FIXME: this is not a good check, we can not guarantee that the predicate we check can truly
// help to prune the data, so we should check the predicate's effect on the data.
protected boolean isKeySearch() {
List<SlotRef> whereSlot = Lists.newArrayList();
for (Expr conjunct : conjuncts) {
Expand Down
4 changes: 2 additions & 2 deletions fe/fe-core/src/main/java/org/apache/doris/planner/ScanNode.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -735,7 +735,7 @@ public int getScanRangeNum() {
return Integer.MAX_VALUE;
}

public boolean haveLimitAndConjunts() {
return hasLimit() && !conjuncts.isEmpty();
public boolean shouldUseOneInstance() {
return hasLimit() && conjuncts.isEmpty();
}
}
8 changes: 4 additions & 4 deletions fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2070,9 +2070,9 @@ private void computeFragmentHosts() throws Exception {
if (node.isPresent() && (!node.get().shouldDisableSharedScan(context)
|| ignoreStorageDataDistribution)) {
expectedInstanceNum = Math.max(expectedInstanceNum, 1);
// if have limit and conjunts, only need 1 instance to save cpu and
// if have limit and no conjuncts, only need 1 instance to save cpu and
// mem resource
if (node.get().haveLimitAndConjunts()) {
if (node.get().shouldUseOneInstance()) {
expectedInstanceNum = 1;
}

Expand All @@ -2083,9 +2083,9 @@ private void computeFragmentHosts() throws Exception {
//the scan instance num should not larger than the tablets num
expectedInstanceNum = Math.min(perNodeScanRanges.size(), parallelExecInstanceNum);
}
// if have limit and conjunts, only need 1 instance to save cpu and
// if have limit and no conjuncts, only need 1 instance to save cpu and
// mem resource
if (node.get().haveLimitAndConjunts()) {
if (node.get().shouldUseOneInstance()) {
expectedInstanceNum = 1;
}

Expand Down