Skip to content

Conversation

@LiBinfeng-01
Copy link
Contributor

@LiBinfeng-01 LiBinfeng-01 commented May 14, 2024

Proposed changes

fix leading with cte and same subqueryalias name
Example:
with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2 from tbl1 join t2) as tbl2 join t3;
Reason:
in this case, before getting analyzed preprocess would change subquery tbl2 to cte plan, and this cte plan should be in upper level cte plan, but not in logical result sink plan

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@LiBinfeng-01
Copy link
Contributor Author

run buildall

}
return visitChildren(this, logicalCTE, context);
Plan cte = visitChildren(this, logicalCTE, context);
if (!aliasQueries.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@LiBinfeng-01
Copy link
Contributor Author

run buildall

1 similar comment
@LiBinfeng-01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39551 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1cb05b62b622cdbd095f8ae3ad7c68d97993280, data reload: false

------ Round 1 ----------------------------------
q1	17940	4358	4242	4242
q2	2022	190	186	186
q3	10474	1237	1089	1089
q4	10163	816	665	665
q5	7450	2706	2610	2610
q6	216	129	132	129
q7	998	580	566	566
q8	9223	2105	2053	2053
q9	9025	6515	6511	6511
q10	8992	3739	3680	3680
q11	457	238	231	231
q12	420	214	209	209
q13	17749	2965	2969	2965
q14	251	226	219	219
q15	523	480	461	461
q16	494	385	377	377
q17	961	743	670	670
q18	8252	7642	7367	7367
q19	5836	1490	1505	1490
q20	638	323	315	315
q21	5061	3242	3343	3242
q22	346	277	274	274
Total cold run time: 117491 ms
Total hot run time: 39551 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4344	4205	4194	4194
q2	356	277	275	275
q3	3001	2776	2723	2723
q4	1867	1606	1589	1589
q5	5278	5298	5255	5255
q6	211	124	124	124
q7	2247	1877	1882	1877
q8	3197	3323	3353	3323
q9	8329	8375	8394	8375
q10	3856	3704	3679	3679
q11	601	491	488	488
q12	769	567	614	567
q13	16526	2979	2956	2956
q14	288	282	265	265
q15	522	474	460	460
q16	465	416	411	411
q17	1754	1485	1470	1470
q18	7775	7524	7416	7416
q19	1686	1604	1589	1589
q20	1985	1762	1760	1760
q21	5043	4771	4910	4771
q22	570	486	486	486
Total cold run time: 70670 ms
Total hot run time: 54053 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187931 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1cb05b62b622cdbd095f8ae3ad7c68d97993280, data reload: false

query1	917	381	368	368
query2	6466	2457	2395	2395
query3	6659	213	214	213
query4	25608	21229	21312	21229
query5	4257	409	417	409
query6	261	171	172	171
query7	4585	297	292	292
query8	241	189	187	187
query9	8555	2410	2384	2384
query10	437	244	261	244
query11	15038	14278	14178	14178
query12	129	93	89	89
query13	1636	389	377	377
query14	10645	8562	8429	8429
query15	260	167	173	167
query16	8177	261	272	261
query17	1863	570	593	570
query18	2098	282	267	267
query19	256	146	146	146
query20	95	87	86	86
query21	199	127	134	127
query22	5041	4851	4903	4851
query23	34226	33703	33481	33481
query24	11615	2913	2958	2913
query25	656	364	375	364
query26	1733	157	153	153
query27	2861	317	326	317
query28	7565	2048	2050	2048
query29	978	611	615	611
query30	309	172	175	172
query31	969	767	751	751
query32	85	53	51	51
query33	739	247	241	241
query34	1043	480	482	480
query35	823	678	683	678
query36	1104	932	913	913
query37	167	69	69	69
query38	2894	2759	2753	2753
query39	1621	1571	1620	1571
query40	275	124	121	121
query41	46	44	45	44
query42	106	95	95	95
query43	572	544	566	544
query44	1214	725	726	725
query45	266	255	255	255
query46	1079	720	732	720
query47	1961	1899	1877	1877
query48	383	292	293	292
query49	1166	392	394	392
query50	776	381	388	381
query51	6828	6758	6850	6758
query52	128	86	91	86
query53	350	285	280	280
query54	884	420	431	420
query55	75	71	75	71
query56	242	218	225	218
query57	1261	1175	1146	1146
query58	228	198	199	198
query59	3283	3156	2996	2996
query60	259	237	235	235
query61	89	88	86	86
query62	669	470	474	470
query63	308	282	282	282
query64	9748	7368	7354	7354
query65	3155	3121	3061	3061
query66	1370	345	345	345
query67	15272	14884	15093	14884
query68	6068	533	541	533
query69	542	301	297	297
query70	1213	1123	1168	1123
query71	468	264	267	264
query72	7099	2510	2369	2369
query73	726	320	321	320
query74	6653	6117	6128	6117
query75	3958	2655	2597	2597
query76	3758	924	965	924
query77	629	261	258	258
query78	10717	10184	10168	10168
query79	2211	511	513	511
query80	833	437	424	424
query81	515	238	241	238
query82	1309	102	94	94
query83	184	160	166	160
query84	241	89	81	81
query85	1540	263	257	257
query86	481	300	314	300
query87	3279	3131	3093	3093
query88	3998	2402	2411	2402
query89	461	393	405	393
query90	1939	187	199	187
query91	125	107	106	106
query92	65	56	49	49
query93	1990	513	494	494
query94	1292	179	186	179
query95	393	296	303	296
query96	598	279	263	263
query97	3177	3024	2959	2959
query98	227	220	216	216
query99	1313	917	918	917
Total cold run time: 292806 ms
Total hot run time: 187931 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d1cb05b62b622cdbd095f8ae3ad7c68d97993280, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.05	0.05
query3	0.23	0.05	0.05
query4	1.68	0.06	0.07
query5	0.51	0.49	0.50
query6	1.12	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.55	0.54	0.54
query11	0.17	0.11	0.10
query12	0.15	0.12	0.12
query13	0.59	0.58	0.60
query14	0.77	0.79	0.78
query15	0.84	0.82	0.81
query16	0.38	0.35	0.37
query17	0.95	0.97	0.94
query18	0.22	0.23	0.24
query19	1.78	1.70	1.64
query20	0.02	0.01	0.01
query21	15.70	0.65	0.64
query22	4.03	8.18	2.03
query23	18.26	1.37	1.22
query24	1.46	0.41	0.19
query25	0.15	0.09	0.07
query26	0.26	0.16	0.17
query27	0.08	0.08	0.08
query28	13.33	1.02	1.01
query29	13.17	3.31	3.33
query30	0.24	0.06	0.05
query31	2.88	0.38	0.37
query32	3.28	0.47	0.47
query33	2.84	2.85	2.82
query34	17.08	4.43	4.42
query35	4.50	4.49	4.52
query36	0.64	0.48	0.46
query37	0.17	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.04	0.04
query40	0.16	0.15	0.14
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.32 s
Total hot run time: 30.43 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 15, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@englefly
Copy link
Contributor

run performance

@englefly englefly merged commit f7ffac0 into apache:master May 16, 2024
dataroaring pushed a commit that referenced this pull request May 16, 2024
fix leading with cte and same subqueryalias name
Example:
with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2 from tbl1 join t2) as tbl2 join t3;
Reason:
in this case, before getting analyzed preprocess would change subquery tbl2 to cte plan, and this cte plan should be in upper level cte plan, but not in logical result sink plan
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request May 20, 2024
…he#34838)

fix leading with cte and same subqueryalias name
Example:
with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2 from tbl1 join t2) as tbl2 join t3;
Reason:
in this case, before getting analyzed preprocess would change subquery tbl2 to cte plan, and this cte plan should be in upper level cte plan, but not in logical result sink plan
yiguolei pushed a commit that referenced this pull request May 20, 2024
…) (#35047)

fix leading with cte and same subqueryalias name
Example:
with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2 from tbl1 join t2) as tbl2 join t3;
Reason:
in this case, before getting analyzed preprocess would change subquery tbl2 to cte plan, and this cte plan should be in upper level cte plan, but not in logical result sink plan
LiBinfeng-01 added a commit to LiBinfeng-01/doris that referenced this pull request May 29, 2024
…he#34838)

fix leading with cte and same subqueryalias name
Example:
with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2 from tbl1 join t2) as tbl2 join t3;
Reason:
in this case, before getting analyzed preprocess would change subquery tbl2 to cte plan, and this cte plan should be in upper level cte plan, but not in logical result sink plan
morrySnow pushed a commit that referenced this pull request May 29, 2024
cherry-pick: #34838 

fix leading with cte and same subqueryalias name. 

Example:

with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2
from tbl1 join t2) as tbl2 join t3;

Reason:
in this case, before getting analyzed preprocess would change subquery
tbl2 to cte plan, and this cte plan should be in upper level cte plan,
but not in logical result sink plan
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…he#35600)

cherry-pick: apache#34838

fix leading with cte and same subqueryalias name.

Example:

with tbl1 as select t1.c1 from t1
select tbl2.c2 from (select / * + leading(t2 tbl1) * / tbl1.c1, t2.c2
from tbl1 join t2) as tbl2 join t3;

Reason:
in this case, before getting analyzed preprocess would change subquery
tbl2 to cte plan, and this cte plan should be in upper level cte plan,
but not in logical result sink plan
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants