Skip to content

[GLUTEN-9196][CH] Use wide-table aggregation to eliminate multi-table joins#9155

Merged
lgbo-ustc merged 10 commits intoapache:mainfrom
bigo-sg:agg_union
Apr 7, 2025
Merged

[GLUTEN-9196][CH] Use wide-table aggregation to eliminate multi-table joins#9155
lgbo-ustc merged 10 commits intoapache:mainfrom
bigo-sg:agg_union

Conversation

@lgbo-ustc
Copy link
Copy Markdown
Contributor

@lgbo-ustc lgbo-ustc commented Mar 27, 2025

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

Fixes: #9196

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@lgbo-ustc lgbo-ustc changed the title [CH][WIP] Use wide-table aggregation to eliminate multi-table joins [GLUTEN-9196][CH][WIP] Use wide-table aggregation to eliminate multi-table joins Apr 1, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 1, 2025

#9196

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 1, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 1, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

@lgbo-ustc
Copy link
Copy Markdown
Contributor Author

When querying only two tables, the optimization does not significantly improve execution speed. However, when querying three tables, the optimized query speed shows a noticeable improvement. This is partly because as the number of tables involved in the query increases, the join method requires building more hash tables. Additionally, the more join operations performed within a single node, the more likely it is to trigger memory spill operations, which further slows down execution.
three_tables_aggregate_union.pdf
three_tables_join_aggregate.pdf
two_tables_by_aggregate_union.pdf
two_tables_by_join_aggregate.pdf

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

@lgbo-ustc lgbo-ustc changed the title [GLUTEN-9196][CH][WIP] Use wide-table aggregation to eliminate multi-table joins [GLUTEN-9196][CH] Use wide-table aggregation to eliminate multi-table joins Apr 2, 2025
@lgbo-ustc lgbo-ustc marked this pull request as ready for review April 2, 2025 06:44
Copy link
Copy Markdown
Contributor

@zzcclp zzcclp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2025

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 3, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 3, 2025

Run Gluten Clickhouse CI on x86

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 7, 2025

Run Gluten Clickhouse CI on x86

@lgbo-ustc lgbo-ustc merged commit 90e2d36 into apache:main Apr 7, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CH] Use wide-table aggregation to eliminate multi-table joins

2 participants