-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[enchancement](statistics) support periodic collection of statistics #19247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ff4ea12 to
e81faad
Compare
|
run buildall |
|
TeamCity pipeline, clickbench performance test result: |
e81faad to
17b5f8e
Compare
|
run buildall |
fe/fe-core/src/main/java/org/apache/doris/catalog/InternalSchemaInitializer.java
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisTaskInfoBuilder.java
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisTaskInfoBuilder.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsAutoAnalyzer.java
Show resolved
Hide resolved
|
PR approved by anyone and no changes requested. |
17b5f8e to
fa86443
Compare
|
run buildall |
fa86443 to
5eaa23f
Compare
5eaa23f to
1603282
Compare
|
run buildall |
yiguolei
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…pache#19247) This PR enables periodic collection of statistics and is a precursor to automatic statistics collection. It mainly includes the following contents: support periodic collection of statistics. Change the type of Date in statistics p0 to DateV2(see [Enhancement](data-type) add FE config to prohibit create date and decimalv2 type apache#19077) for test locally. complement cases(remove Chinese characters, optimize code, etc) , improve stability. Supports setting whether to keep records of statistics synchronization job info, convenient for use in p0 testing. The statistics job table was modified, and some auxiliary judgments were added to avoid the user perceiving the modification. This function was removed when the table schema is stable.
Proposed changes
This PR enables periodic collection of statistics and is a precursor to automatic statistics collection. It mainly includes the following contents:
...
Related syntax:
Fe configuration:
enable_auto_collect_statistics: if true, will allow the system to collect statistics automatically.auto_check_statistics_in_sec: the system automatically checks the time interval for statistics.enable_save_statistics_sync_job: session variable. if true, when synchronously collecting statistics, the information of the statistics job will be saved, currently mainly used for p0 test.Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...