-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[enhancement](stats) Param tune which would get a better balance between execution time and cost #25767
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ee399a9 to
dd73dfb
Compare
|
run buildall |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
|
||
| public static final int LOAD_RETRY_TIMES = 3; | ||
|
|
||
| // union more relation than 512 may cause StackOverFlowException in the future. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there overflow? UNION_ALL_LIMIT is more than 512
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, in fact we could union even ten thousand query according to my test on legacy optimizer
| */ | ||
| @ConfField | ||
| public static int statistics_simultaneously_running_task_num = 5; | ||
| public static int statistics_simultaneously_running_task_num = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should move all statistics paramters from config into global variables to avoid restart fe.
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
pick from #26858 |
Proposed changes
count(1) - count(col1)rather than sum(case when) which could leverage some optimize like read count from tablet metadata directlyFurther comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...