Skip to content

Conversation

@morrySnow
Copy link
Contributor

this is a behaviour change PR.

set operation INTERSECT should evaluated before others. In Doris history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39298 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fbe92c31d874cf7d0f649a768701ccaec92c374b, data reload: false

------ Round 1 ----------------------------------
q1	17596	4343	4218	4218
q2	2021	177	174	174
q3	10495	1192	1118	1118
q4	10152	696	698	696
q5	7494	2537	2493	2493
q6	223	138	136	136
q7	964	590	593	590
q8	9225	1910	1885	1885
q9	8640	6568	6617	6568
q10	7079	2197	2170	2170
q11	487	258	246	246
q12	527	223	216	216
q13	17764	3031	3028	3028
q14	278	237	226	226
q15	518	486	503	486
q16	505	393	386	386
q17	962	679	665	665
q18	8006	7455	7469	7455
q19	5289	1054	1029	1029
q20	667	327	339	327
q21	5741	4222	4569	4222
q22	1097	1000	964	964
Total cold run time: 115730 ms
Total hot run time: 39298 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4594	4262	4279	4262
q2	371	284	270	270
q3	2816	2635	2656	2635
q4	2013	1702	1689	1689
q5	5574	5573	5492	5492
q6	227	134	134	134
q7	2105	1754	1773	1754
q8	3288	3461	3421	3421
q9	8801	8725	8946	8725
q10	3496	3251	3353	3251
q11	602	505	483	483
q12	757	579	613	579
q13	17056	3179	3210	3179
q14	307	310	282	282
q15	553	496	479	479
q16	512	449	433	433
q17	1846	1515	1520	1515
q18	8022	7930	7849	7849
q19	1765	1624	1607	1607
q20	2407	1906	1955	1906
q21	10008	5249	5277	5249
q22	1075	999	992	992
Total cold run time: 78195 ms
Total hot run time: 56186 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202636 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fbe92c31d874cf7d0f649a768701ccaec92c374b, data reload: false

query1	962	397	393	393
query2	6439	2042	1910	1910
query3	6628	206	214	206
query4	34176	23208	23263	23208
query5	3651	485	494	485
query6	263	177	174	174
query7	4574	297	288	288
query8	242	197	201	197
query9	8464	2373	2333	2333
query10	939	892	874	874
query11	17775	14931	14852	14852
query12	132	99	94	94
query13	1645	393	384	384
query14	10226	7919	7501	7501
query15	490	365	344	344
query16	7518	475	484	475
query17	1335	597	577	577
query18	2020	385	403	385
query19	292	189	221	189
query20	123	111	110	110
query21	210	107	123	107
query22	4480	4577	4316	4316
query23	34188	34047	33716	33716
query24	11124	2942	2919	2919
query25	596	386	396	386
query26	696	157	154	154
query27	2169	284	281	281
query28	5671	2018	2001	2001
query29	788	422	423	422
query30	256	153	146	146
query31	998	707	746	707
query32	90	56	55	55
query33	638	293	286	286
query34	862	481	473	473
query35	959	837	908	837
query36	1077	908	901	901
query37	135	82	77	77
query38	4208	4128	4067	4067
query39	1453	1518	1379	1379
query40	200	112	109	109
query41	44	44	41	41
query42	111	93	94	93
query43	508	471	476	471
query44	1109	725	740	725
query45	393	324	360	324
query46	1103	738	758	738
query47	1844	1797	1799	1797
query48	371	300	306	300
query49	848	408	426	408
query50	797	402	403	402
query51	6800	6725	6602	6602
query52	105	84	84	84
query53	246	181	173	173
query54	867	437	446	437
query55	78	73	74	73
query56	263	241	243	241
query57	1121	1054	1084	1054
query58	240	227	232	227
query59	3080	2951	2799	2799
query60	301	258	261	258
query61	98	93	106	93
query62	759	662	651	651
query63	204	178	180	178
query64	9236	2468	1919	1919
query65	3203	3131	3126	3126
query66	710	321	329	321
query67	15639	14743	14825	14743
query68	6297	581	541	541
query69	464	415	390	390
query70	1215	1060	1053	1053
query71	513	271	267	267
query72	19882	16372	16596	16372
query73	815	324	322	322
query74	9189	8699	8813	8699
query75	4729	2692	2674	2674
query76	4472	1025	953	953
query77	826	310	304	304
query78	9952	8862	8876	8862
query79	8357	517	526	517
query80	1172	483	484	483
query81	594	236	236	236
query82	793	136	141	136
query83	329	163	152	152
query84	268	76	81	76
query85	1387	307	298	298
query86	433	296	296	296
query87	4628	4511	4467	4467
query88	5406	2483	2514	2483
query89	523	289	285	285
query90	2083	191	190	190
query91	150	121	122	121
query92	60	48	51	48
query93	6401	544	536	536
query94	1076	302	271	271
query95	351	259	254	254
query96	637	270	269	269
query97	3235	3100	3043	3043
query98	217	200	196	196
query99	1645	1246	1276	1246
Total cold run time: 326869 ms
Total hot run time: 202636 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.61 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fbe92c31d874cf7d0f649a768701ccaec92c374b, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.08	0.09
query5	0.48	0.48	0.48
query6	1.14	0.73	0.74
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.56	0.48	0.48
query10	0.55	0.56	0.55
query11	0.15	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.61	0.59
query14	0.75	0.80	0.78
query15	0.85	0.80	0.80
query16	0.34	0.38	0.37
query17	1.04	1.00	0.99
query18	0.23	0.22	0.21
query19	1.81	1.78	1.74
query20	0.03	0.02	0.01
query21	15.41	0.75	0.65
query22	4.44	7.14	1.91
query23	18.29	1.27	1.26
query24	2.06	0.23	0.21
query25	0.14	0.09	0.08
query26	0.30	0.21	0.21
query27	0.46	0.22	0.22
query28	13.34	1.01	1.00
query29	12.60	3.28	3.28
query30	0.25	0.06	0.05
query31	2.91	0.40	0.38
query32	3.26	0.46	0.46
query33	2.89	2.92	2.91
query34	17.10	4.36	4.40
query35	4.46	4.44	4.39
query36	0.66	0.48	0.45
query37	0.19	0.16	0.16
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.15	0.13	0.12
query41	0.08	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.09 s
Total hot run time: 30.61 s

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39863 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5281989dc5636e390760740f07593b9e390ead4c, data reload: false

------ Round 1 ----------------------------------
q1	17651	4396	4314	4314
q2	2025	174	172	172
q3	10498	1217	1202	1202
q4	10144	772	753	753
q5	7633	2576	2549	2549
q6	233	149	152	149
q7	1000	631	618	618
q8	9605	1928	1889	1889
q9	8817	6547	6567	6547
q10	7057	2181	2214	2181
q11	465	252	255	252
q12	394	222	217	217
q13	18648	2976	2979	2976
q14	278	242	245	242
q15	520	482	491	482
q16	515	381	382	381
q17	963	643	678	643
q18	7920	7500	7451	7451
q19	4608	1105	1016	1016
q20	664	322	345	322
q21	5257	4511	4560	4511
q22	1130	1001	996	996
Total cold run time: 116025 ms
Total hot run time: 39863 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4459	4271	4332	4271
q2	383	263	259	259
q3	2848	2614	2617	2614
q4	1893	1613	1560	1560
q5	5243	5221	5296	5221
q6	218	132	135	132
q7	2053	1678	1663	1663
q8	3148	3358	3345	3345
q9	8389	8353	8364	8353
q10	3406	3165	3161	3161
q11	580	490	515	490
q12	762	586	612	586
q13	17487	2995	2962	2962
q14	307	276	268	268
q15	530	480	474	474
q16	466	426	409	409
q17	1786	1492	1467	1467
q18	7660	7524	7277	7277
q19	1667	1549	1592	1549
q20	1990	1770	1810	1770
q21	5298	5090	5061	5061
q22	1075	1004	1008	1004
Total cold run time: 71648 ms
Total hot run time: 53896 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202545 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5281989dc5636e390760740f07593b9e390ead4c, data reload: false

query1	944	403	391	391
query2	6453	1896	1768	1768
query3	6662	228	226	226
query4	34009	23230	23002	23002
query5	4192	535	501	501
query6	294	170	172	170
query7	4594	292	292	292
query8	245	205	199	199
query9	8356	2404	2368	2368
query10	916	886	898	886
query11	15708	14915	15029	14915
query12	146	104	97	97
query13	1635	386	398	386
query14	9545	8240	7326	7326
query15	388	344	320	320
query16	7877	468	500	468
query17	1763	585	549	549
query18	2132	388	388	388
query19	273	224	219	219
query20	113	104	109	104
query21	206	105	104	104
query22	4302	4091	3979	3979
query23	33739	33175	33182	33175
query24	11988	3020	3005	3005
query25	664	396	386	386
query26	1797	155	154	154
query27	2878	279	283	279
query28	7708	1987	1996	1987
query29	1115	432	429	429
query30	302	149	150	149
query31	964	749	728	728
query32	97	55	57	55
query33	745	313	293	293
query34	922	475	497	475
query35	1031	826	821	821
query36	1082	904	933	904
query37	242	85	86	85
query38	4190	4155	4141	4141
query39	1467	1345	1385	1345
query40	272	122	118	118
query41	54	47	47	47
query42	121	97	99	97
query43	510	460	464	460
query44	1270	749	761	749
query45	409	366	360	360
query46	1119	780	805	780
query47	1881	1772	1790	1772
query48	381	302	312	302
query49	1220	448	445	445
query50	814	418	425	418
query51	6842	6832	6733	6733
query52	105	92	89	89
query53	261	188	185	185
query54	1021	455	462	455
query55	79	75	77	75
query56	289	258	273	258
query57	1163	1096	1102	1096
query58	245	233	236	233
query59	3041	2670	2802	2670
query60	300	278	285	278
query61	122	218	99	99
query62	822	629	643	629
query63	218	186	182	182
query64	10553	2449	1938	1938
query65	3203	3196	3211	3196
query66	1375	358	333	333
query67	15453	14785	14790	14785
query68	8928	572	586	572
query69	478	427	427	427
query70	1314	1111	1095	1095
query71	591	275	267	267
query72	20286	16576	16930	16576
query73	2091	333	335	333
query74	9133	8721	8759	8721
query75	5436	2673	2691	2673
query76	5411	1023	989	989
query77	827	338	324	324
query78	10012	9077	8990	8990
query79	10063	537	540	537
query80	1169	502	501	501
query81	596	233	229	229
query82	546	131	132	131
query83	333	142	151	142
query84	272	78	81	78
query85	1032	341	301	301
query86	337	293	301	293
query87	4820	4474	4560	4474
query88	4823	2499	2485	2485
query89	519	288	291	288
query90	2113	196	198	196
query91	152	120	131	120
query92	65	48	50	48
query93	6912	545	532	532
query94	1062	301	305	301
query95	361	265	265	265
query96	630	279	278	278
query97	3219	3062	3093	3062
query98	213	203	200	200
query99	1743	1270	1266	1266
Total cold run time: 340314 ms
Total hot run time: 202545 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5281989dc5636e390760740f07593b9e390ead4c, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.08	0.09
query5	0.50	0.48	0.51
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.49	0.50
query10	0.54	0.55	0.56
query11	0.16	0.11	0.10
query12	0.15	0.12	0.12
query13	0.62	0.60	0.58
query14	0.77	0.79	0.76
query15	0.87	0.82	0.79
query16	0.37	0.37	0.37
query17	0.99	0.95	1.04
query18	0.22	0.22	0.21
query19	1.81	1.73	1.68
query20	0.01	0.01	0.01
query21	15.40	0.75	0.66
query22	4.47	6.24	2.56
query23	18.36	1.34	1.21
query24	2.05	0.22	0.21
query25	0.16	0.08	0.08
query26	0.29	0.22	0.20
query27	0.47	0.21	0.22
query28	13.44	1.00	1.00
query29	12.58	3.40	3.35
query30	0.24	0.05	0.05
query31	2.89	0.38	0.39
query32	3.28	0.49	0.47
query33	2.84	2.98	2.94
query34	16.83	4.38	4.38
query35	4.39	4.40	4.41
query36	0.66	0.49	0.46
query37	0.18	0.15	0.16
query38	0.15	0.15	0.16
query39	0.05	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.85 s
Total hot run time: 31.24 s

this is a behaviour change PR.

set operation INTERSECT should evaluated before others.
In Doris history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39598 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7edf62307fc50a46734360db77a602ee973a477d, data reload: false

------ Round 1 ----------------------------------
q1	17626	4684	4386	4386
q2	2025	206	210	206
q3	11158	998	1078	998
q4	10563	707	721	707
q5	7499	2605	2507	2507
q6	223	139	138	138
q7	979	596	603	596
q8	9223	1923	1976	1923
q9	8794	6536	6551	6536
q10	7021	2129	2214	2129
q11	452	244	251	244
q12	399	222	219	219
q13	17746	3012	2997	2997
q14	283	240	235	235
q15	531	493	484	484
q16	494	396	387	387
q17	962	626	680	626
q18	8266	7481	7551	7481
q19	7010	1132	992	992
q20	769	340	341	340
q21	5446	4429	4562	4429
q22	1113	1046	1038	1038
Total cold run time: 118582 ms
Total hot run time: 39598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4549	4270	4241	4241
q2	372	272	267	267
q3	2839	2791	2763	2763
q4	1959	1658	1750	1658
q5	5539	5473	5450	5450
q6	228	135	131	131
q7	2173	1805	1765	1765
q8	3261	3484	3411	3411
q9	8761	8845	8808	8808
q10	3557	3176	3218	3176
q11	602	519	502	502
q12	817	659	671	659
q13	16981	3142	3192	3142
q14	343	292	315	292
q15	535	495	509	495
q16	497	452	442	442
q17	1834	1539	1509	1509
q18	8121	8006	7843	7843
q19	2227	1649	1748	1649
q20	2126	1897	1879	1879
q21	5349	5394	5144	5144
q22	1167	1044	1078	1044
Total cold run time: 73837 ms
Total hot run time: 56270 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203470 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7edf62307fc50a46734360db77a602ee973a477d, data reload: false

query1	1671	1476	1139	1139
query2	6438	2034	1867	1867
query3	13963	8396	313	313
query4	27574	23133	23123	23123
query5	3830	484	496	484
query6	268	168	178	168
query7	3939	299	292	292
query8	246	211	202	202
query9	8592	2504	2458	2458
query10	528	471	460	460
query11	16712	15030	14850	14850
query12	154	100	92	92
query13	1539	372	355	355
query14	9136	7698	7560	7560
query15	270	224	226	224
query16	7685	519	522	519
query17	1628	569	563	563
query18	1871	297	307	297
query19	188	145	142	142
query20	117	111	120	111
query21	204	106	100	100
query22	4693	4628	4363	4363
query23	34790	33674	33707	33674
query24	11706	2608	2557	2557
query25	584	382	377	377
query26	1391	159	177	159
query27	2635	276	285	276
query28	7358	2082	2055	2055
query29	714	396	407	396
query30	250	148	149	148
query31	967	775	737	737
query32	90	51	55	51
query33	720	274	285	274
query34	857	452	473	452
query35	953	806	762	762
query36	1076	925	928	925
query37	143	78	79	78
query38	4332	4120	4174	4120
query39	1446	1403	1408	1403
query40	253	113	117	113
query41	44	43	43	43
query42	115	93	93	93
query43	508	494	465	465
query44	1198	720	717	717
query45	248	203	193	193
query46	1085	742	777	742
query47	1846	1763	1756	1756
query48	398	297	297	297
query49	921	426	438	426
query50	812	413	409	409
query51	6780	6714	6689	6689
query52	102	89	95	89
query53	253	186	183	183
query54	902	453	459	453
query55	80	81	79	79
query56	271	253	254	253
query57	1164	1068	1088	1068
query58	232	237	243	237
query59	2978	2828	2737	2737
query60	308	278	292	278
query61	120	117	115	115
query62	843	631	620	620
query63	215	184	193	184
query64	10395	2346	1843	1843
query65	3222	3189	3209	3189
query66	1175	340	344	340
query67	15388	14753	14924	14753
query68	4553	553	549	549
query69	401	371	376	371
query70	1190	1115	1156	1115
query71	438	278	283	278
query72	20645	17189	18158	17189
query73	757	328	326	326
query74	9157	8806	8756	8756
query75	3643	2640	2698	2640
query76	2951	990	963	963
query77	614	310	329	310
query78	9777	9071	8923	8923
query79	7225	520	527	520
query80	1589	498	488	488
query81	612	236	225	225
query82	1054	139	135	135
query83	330	146	150	146
query84	267	79	77	77
query85	2077	288	275	275
query86	460	301	292	292
query87	4654	4608	4437	4437
query88	5622	2479	2487	2479
query89	606	288	283	283
query90	2076	198	200	198
query91	122	94	96	94
query92	70	52	50	50
query93	6757	542	530	530
query94	1016	293	294	293
query95	358	266	265	265
query96	619	282	272	272
query97	3204	3052	3046	3046
query98	225	204	198	198
query99	1760	1295	1257	1257
Total cold run time: 327943 ms
Total hot run time: 203470 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7edf62307fc50a46734360db77a602ee973a477d, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.08	0.09
query5	0.52	0.49	0.51
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.56	0.48	0.50
query10	0.54	0.54	0.54
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.60	0.59
query14	0.77	0.78	0.78
query15	0.89	0.84	0.83
query16	0.36	0.37	0.36
query17	1.01	1.03	1.03
query18	0.24	0.21	0.21
query19	1.87	1.74	1.83
query20	0.01	0.02	0.03
query21	15.43	0.74	0.64
query22	4.40	7.12	1.61
query23	18.26	1.30	1.22
query24	2.17	0.23	0.22
query25	0.15	0.09	0.08
query26	0.31	0.21	0.21
query27	0.46	0.22	0.23
query28	13.22	1.03	0.99
query29	12.60	3.30	3.25
query30	0.24	0.06	0.05
query31	2.85	0.39	0.39
query32	3.27	0.49	0.47
query33	2.89	2.94	2.94
query34	17.06	4.30	4.33
query35	4.45	4.40	4.42
query36	0.66	0.48	0.48
query37	0.21	0.16	0.16
query38	0.15	0.16	0.15
query39	0.06	0.03	0.04
query40	0.16	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.12 s
Total hot run time: 30.37 s

@morrySnow
Copy link
Contributor Author

run feut

1 similar comment
@morrySnow
Copy link
Contributor Author

run feut

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 13, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow removed the doing label Aug 13, 2024
@morrySnow morrySnow merged commit 9d91208 into apache:master Aug 13, 2024
@morrySnow morrySnow deleted the intersect_real branch August 13, 2024 11:33
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
…#39095)

this is a behaviour change PR.

set operation INTERSECT should evaluated before others. In Doris
history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.
morrySnow added a commit that referenced this pull request Aug 15, 2024
pick from master #39095

this is a behaviour change PR.

set operation INTERSECT should evaluated before others. In Doris
history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.
morrySnow added a commit that referenced this pull request Aug 16, 2024
#39437)

pick from master #39095

this is a behaviour change PR.

set operation INTERSECT should evaluated before others. In Doris
history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
this is a behaviour change PR.

set operation INTERSECT should evaluated before others. In Doris
history, all set operators have same priority.

This PR change Nereids, let it be same with MySQL.
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants