-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[optimize](parquet-reader) Optimize performace by parquet bloom filter. #57959
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[optimize](parquet-reader) Optimize performace by parquet bloom filter. #57959
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 34564 ms |
TPC-DS: Total hot run time: 187453 ms |
ClickBench: Total hot run time: 28.33 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
e21ad9d to
bbf4733
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
fb8a59b to
1f6f613
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage `` 🎉 |
1f6f613 to
71d0014
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
71d0014 to
7a01180
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage `` 🎉 |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
7a01180 to
6d463b1
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 35177 ms |
TPC-DS: Total hot run time: 187552 ms |
ClickBench: Total hot run time: 28.42 s |
681879a to
d930888
Compare
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
TPC-H: Total hot run time: 34361 ms |
TPC-DS: Total hot run time: 184847 ms |
ClickBench: Total hot run time: 28.42 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by anyone and no changes requested. |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by at least one committer and no changes requested. |
…r. (apache#57959) ### What problem does this PR solve? Problem Summary: ### Release note Optimize performance by reading parquet bloom filter. parquet bloom filter: https://parquet.apache.org/docs/file-format/bloomfilter/ ### Query Performance Test Results SQL Query | Optimized Version (time(s)) | Original Version (time(s)) -- | -- | -- SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string = 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d4'; | 0.02 | 0.23 SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string IN ('cfcd2084-cfcd-cfcd-cfcd-cfcd208495d6', 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d7'); | 0.04 | 0.24
…r. (apache#57959) ### What problem does this PR solve? Problem Summary: ### Release note Optimize performance by reading parquet bloom filter. parquet bloom filter: https://parquet.apache.org/docs/file-format/bloomfilter/ ### Query Performance Test Results SQL Query | Optimized Version (time(s)) | Original Version (time(s)) -- | -- | -- SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string = 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d4'; | 0.02 | 0.23 SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string IN ('cfcd2084-cfcd-cfcd-cfcd-cfcd208495d6', 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d7'); | 0.04 | 0.24
…r. (apache#57959) ### What problem does this PR solve? Problem Summary: ### Release note Optimize performance by reading parquet bloom filter. parquet bloom filter: https://parquet.apache.org/docs/file-format/bloomfilter/ ### Query Performance Test Results SQL Query | Optimized Version (time(s)) | Original Version (time(s)) -- | -- | -- SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string = 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d4'; | 0.02 | 0.23 SELECT * FROM cqtest.bloom_filter_perf_parquet_duckdb WHERE uuid_string IN ('cfcd2084-cfcd-cfcd-cfcd-cfcd208495d6', 'cfcd2084-cfcd-cfcd-cfcd-cfcd208495d7'); | 0.04 | 0.24
What problem does this PR solve?
Problem Summary:
Release note
Optimize performance by reading parquet bloom filter.
parquet bloom filter: https://parquet.apache.org/docs/file-format/bloomfilter/
Query Performance Test Results
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)