Offload entry root slice to QE by Ray-Eldath · Pull Request #385 · apache/cloudberry

Ray-Eldath · 2024-02-28T11:45:21Z

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

avamingli

LGTM.

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

Ray-Eldath · 2024-03-18T10:43:42Z

@my-ship-it Maybe merge this as well? :)

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from eb7f7fc to e5e074f Compare February 29, 2024 07:01

avamingli reviewed Mar 1, 2024

View reviewed changes

Comment thread src/backend/utils/misc/guc_gp.c Outdated

Comment thread src/test/regress/sql/offload_entry_to_qe.sql Outdated

avamingli reviewed Mar 1, 2024

View reviewed changes

Comment thread src/test/regress/expected/offload_entry_to_qe.out Outdated

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from e9ae87e to ec724c3 Compare March 1, 2024 08:44

avamingli reviewed Mar 4, 2024

View reviewed changes

Comment thread src/test/regress/expected/offload_entry_to_qe.out Outdated

Comment thread src/backend/optimizer/plan/createplan.c

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from 6229fcb to 58a5e51 Compare March 6, 2024 03:42

avamingli reviewed Mar 8, 2024

View reviewed changes