Skip to content

Conversation

@starocean999
Copy link
Contributor

Proposed changes

convert
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY GROUPING SETS ((k1, k2));
to
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY (k1, k2);

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40064 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4454381ff0a19ac736860de915c5a5f375a609d6, data reload: false

------ Round 1 ----------------------------------
q1	17615	4326	4293	4293
q2	2028	198	200	198
q3	10729	1176	1191	1176
q4	10590	822	764	764
q5	7673	2693	2668	2668
q6	223	141	140	140
q7	973	629	608	608
q8	9322	2097	2090	2090
q9	8978	6650	6592	6592
q10	9584	3661	3715	3661
q11	455	251	237	237
q12	467	227	227	227
q13	17785	3005	2988	2988
q14	269	215	221	215
q15	498	473	466	466
q16	517	389	379	379
q17	958	673	639	639
q18	8161	7476	7418	7418
q19	3153	1486	1406	1406
q20	653	318	308	308
q21	4935	3248	4003	3248
q22	404	356	343	343
Total cold run time: 115970 ms
Total hot run time: 40064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4402	4266	4276	4266
q2	377	260	271	260
q3	2957	2787	2752	2752
q4	1858	1617	1598	1598
q5	5239	5274	5264	5264
q6	225	126	127	126
q7	2103	1739	1729	1729
q8	3237	3326	3298	3298
q9	8310	8281	8305	8281
q10	3870	3629	3708	3629
q11	574	484	475	475
q12	815	605	595	595
q13	16420	2957	3005	2957
q14	296	271	265	265
q15	517	473	474	473
q16	467	406	403	403
q17	1763	1460	1461	1460
q18	7529	7575	7328	7328
q19	1715	1655	1560	1560
q20	1991	1774	1796	1774
q21	4938	4796	4776	4776
q22	602	566	546	546
Total cold run time: 70205 ms
Total hot run time: 53815 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170615 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4454381ff0a19ac736860de915c5a5f375a609d6, data reload: false

query1	925	380	363	363
query2	6461	2420	2287	2287
query3	6647	203	205	203
query4	19798	17235	17423	17235
query5	4175	464	449	449
query6	256	153	151	151
query7	4596	310	303	303
query8	312	297	286	286
query9	8584	2391	2358	2358
query10	445	311	269	269
query11	10647	10051	10155	10051
query12	129	92	86	86
query13	1643	367	358	358
query14	9414	7549	7670	7549
query15	232	188	188	188
query16	7795	267	268	267
query17	1644	552	523	523
query18	1914	272	282	272
query19	214	156	153	153
query20	94	84	92	84
query21	203	132	139	132
query22	4426	4413	4085	4085
query23	33565	33201	33043	33043
query24	12063	2805	2923	2805
query25	638	376	366	366
query26	1629	157	152	152
query27	2987	319	321	319
query28	7414	2039	2040	2039
query29	1025	604	610	604
query30	284	150	150	150
query31	962	763	741	741
query32	99	53	54	53
query33	768	307	306	306
query34	920	472	480	472
query35	750	668	644	644
query36	1083	951	922	922
query37	160	68	70	68
query38	2892	2753	2709	2709
query39	866	773	798	773
query40	291	122	122	122
query41	53	53	50	50
query42	118	101	99	99
query43	578	535	540	535
query44	1195	735	732	732
query45	199	180	169	169
query46	1084	744	716	716
query47	1890	1788	1769	1769
query48	376	304	290	290
query49	1132	411	419	411
query50	779	384	397	384
query51	6956	6787	6754	6754
query52	112	86	92	86
query53	359	298	287	287
query54	1025	452	441	441
query55	71	74	71	71
query56	284	250	263	250
query57	1121	1064	1064	1064
query58	264	248	248	248
query59	3382	3199	3200	3199
query60	293	275	270	270
query61	93	90	95	90
query62	646	440	458	440
query63	322	290	286	286
query64	9989	2182	1708	1708
query65	3168	3074	3111	3074
query66	1360	332	327	327
query67	15336	14914	14900	14900
query68	5293	539	532	532
query69	579	414	398	398
query70	1136	1034	1116	1034
query71	457	301	290	290
query72	7375	5290	2799	2799
query73	765	323	316	316
query74	5954	5551	5503	5503
query75	3741	2660	2679	2660
query76	3307	887	942	887
query77	634	309	298	298
query78	10242	9875	9670	9670
query79	1715	507	517	507
query80	1982	477	481	477
query81	580	233	220	220
query82	822	109	105	105
query83	314	178	180	178
query84	270	86	87	86
query85	1557	281	294	281
query86	486	295	308	295
query87	3262	3082	3082	3082
query88	3958	2431	2347	2347
query89	477	392	392	392
query90	1774	190	195	190
query91	130	100	112	100
query92	64	50	48	48
query93	2125	514	498	498
query94	1208	193	188	188
query95	407	324	318	318
query96	589	267	269	267
query97	3155	3041	2994	2994
query98	254	233	220	220
query99	1126	845	847	845
Total cold run time: 276873 ms
Total hot run time: 170615 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4454381ff0a19ac736860de915c5a5f375a609d6, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.22	0.05	0.05
query4	1.67	0.06	0.07
query5	0.53	0.50	0.51
query6	1.12	0.73	0.72
query7	0.03	0.01	0.01
query8	0.05	0.04	0.04
query9	0.53	0.47	0.51
query10	0.54	0.55	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.59
query14	0.77	0.76	0.80
query15	0.83	0.81	0.80
query16	0.35	0.36	0.36
query17	1.00	0.98	0.96
query18	0.19	0.27	0.23
query19	1.80	1.66	1.72
query20	0.02	0.01	0.01
query21	15.64	0.67	0.66
query22	4.58	6.41	2.28
query23	18.38	1.42	1.23
query24	1.74	0.32	0.23
query25	0.15	0.08	0.08
query26	0.28	0.18	0.17
query27	0.07	0.09	0.08
query28	13.22	1.02	0.99
query29	13.21	3.28	3.29
query30	0.25	0.05	0.05
query31	2.86	0.39	0.39
query32	3.26	0.47	0.46
query33	2.86	2.87	2.90
query34	17.18	4.38	4.37
query35	4.46	4.48	4.50
query36	0.64	0.46	0.47
query37	0.19	0.16	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.2 s
Total hot run time: 30.74 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40120 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fe8f62e5cf0e877095dfe1419765b7ba759cc4f4, data reload: false

------ Round 1 ----------------------------------
q1	18611	4502	4410	4410
q2	2648	203	201	201
q3	11705	1122	1232	1122
q4	10713	822	790	790
q5	7573	2693	2726	2693
q6	221	139	139	139
q7	943	590	595	590
q8	9242	2062	2085	2062
q9	9158	6481	6449	6449
q10	8998	3732	3707	3707
q11	447	245	239	239
q12	442	225	220	220
q13	17774	2967	2983	2967
q14	260	227	207	207
q15	508	475	469	469
q16	528	372	378	372
q17	956	641	722	641
q18	8072	7501	7472	7472
q19	3512	1475	1517	1475
q20	648	302	295	295
q21	4951	3956	3262	3262
q22	403	339	338	338
Total cold run time: 118313 ms
Total hot run time: 40120 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4369	4232	4234	4232
q2	358	266	260	260
q3	2938	2785	2711	2711
q4	1887	1581	1577	1577
q5	5294	5254	5283	5254
q6	218	127	125	125
q7	2103	1737	1715	1715
q8	3185	3264	3284	3264
q9	8312	8289	8360	8289
q10	3869	3707	3670	3670
q11	598	496	483	483
q12	777	614	607	607
q13	17381	2971	2984	2971
q14	301	261	256	256
q15	516	479	475	475
q16	466	416	411	411
q17	1760	1469	1443	1443
q18	7632	7444	7429	7429
q19	3058	1509	1669	1509
q20	2009	1788	1758	1758
q21	10297	4581	4706	4581
q22	627	539	532	532
Total cold run time: 77955 ms
Total hot run time: 53552 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170243 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fe8f62e5cf0e877095dfe1419765b7ba759cc4f4, data reload: false

query1	927	377	379	377
query2	6434	2358	2388	2358
query3	6650	203	203	203
query4	19356	17468	17166	17166
query5	4110	450	456	450
query6	246	160	158	158
query7	4586	293	303	293
query8	326	300	283	283
query9	8510	2376	2370	2370
query10	447	298	264	264
query11	10439	9973	9995	9973
query12	132	87	85	85
query13	1629	369	363	363
query14	9533	7646	7518	7518
query15	245	182	188	182
query16	7808	261	281	261
query17	1832	531	505	505
query18	1913	273	265	265
query19	195	151	163	151
query20	91	83	80	80
query21	206	132	128	128
query22	4333	3998	4008	3998
query23	33683	33059	32925	32925
query24	12009	2796	2793	2793
query25	672	355	357	355
query26	1841	158	155	155
query27	3029	317	316	316
query28	7468	2052	2056	2052
query29	1204	606	584	584
query30	294	146	155	146
query31	958	752	727	727
query32	91	56	52	52
query33	763	281	274	274
query34	980	487	462	462
query35	728	598	622	598
query36	1092	906	933	906
query37	211	73	72	72
query38	2878	2740	2774	2740
query39	856	787	792	787
query40	279	125	127	125
query41	53	53	52	52
query42	119	97	97	97
query43	550	554	559	554
query44	1253	749	769	749
query45	194	167	166	166
query46	1100	709	708	708
query47	1852	1761	1772	1761
query48	387	304	294	294
query49	1187	416	411	411
query50	790	379	385	379
query51	6852	6846	6768	6768
query52	107	89	91	89
query53	358	293	290	290
query54	1014	454	430	430
query55	73	71	70	70
query56	273	259	262	259
query57	1154	1016	1062	1016
query58	258	238	256	238
query59	3575	3310	3225	3225
query60	291	264	263	263
query61	91	88	89	88
query62	632	457	452	452
query63	326	281	292	281
query64	9840	2206	1801	1801
query65	3387	3101	3117	3101
query66	1375	338	383	338
query67	15257	15148	15014	15014
query68	4583	544	525	525
query69	452	313	297	297
query70	1133	1111	1028	1028
query71	418	281	269	269
query72	7078	5322	2715	2715
query73	733	321	319	319
query74	5988	5542	5482	5482
query75	3349	2652	2643	2643
query76	2428	979	918	918
query77	441	316	294	294
query78	10132	9808	9702	9702
query79	2662	514	509	509
query80	2147	466	457	457
query81	567	220	222	220
query82	898	159	103	103
query83	283	172	173	172
query84	269	84	89	84
query85	2071	282	262	262
query86	507	325	298	298
query87	3280	3104	3076	3076
query88	4518	2474	2443	2443
query89	475	382	380	380
query90	1822	191	190	190
query91	126	95	95	95
query92	60	48	47	47
query93	2809	520	493	493
query94	1260	193	189	189
query95	397	316	317	316
query96	603	275	269	269
query97	3181	3007	3022	3007
query98	234	217	215	215
query99	1353	832	835	832
Total cold run time: 277589 ms
Total hot run time: 170243 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fe8f62e5cf0e877095dfe1419765b7ba759cc4f4, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.68	0.07	0.07
query5	0.50	0.50	0.50
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.52	0.49	0.49
query10	0.54	0.55	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.61
query14	0.77	0.78	0.77
query15	0.82	0.80	0.80
query16	0.37	0.37	0.35
query17	0.99	0.96	0.98
query18	0.22	0.25	0.21
query19	1.77	1.71	1.65
query20	0.02	0.01	0.02
query21	15.70	0.67	0.67
query22	4.63	6.01	3.12
query23	18.27	1.42	1.31
query24	1.40	0.38	0.22
query25	0.15	0.08	0.08
query26	0.26	0.18	0.17
query27	0.08	0.08	0.08
query28	13.38	1.02	1.00
query29	13.19	3.32	3.27
query30	0.24	0.05	0.06
query31	2.86	0.39	0.38
query32	3.28	0.47	0.48
query33	2.85	2.92	2.91
query34	17.24	4.43	4.42
query35	4.58	4.49	4.54
query36	0.66	0.46	0.47
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.15 s
Total hot run time: 31.76 s

@starocean999 starocean999 marked this pull request as ready for review June 4, 2024 01:50
@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41477 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6387e4a5545e05885009b0b0b2949a66d3e464f, data reload: false

------ Round 1 ----------------------------------
q1	17629	4919	4213	4213
q2	2045	199	199	199
q3	10514	1152	1181	1152
q4	10280	757	880	757
q5	7565	2683	2649	2649
q6	223	139	141	139
q7	973	656	622	622
q8	9232	2150	2086	2086
q9	9340	6665	6645	6645
q10	9431	3914	3910	3910
q11	448	246	245	245
q12	471	242	238	238
q13	17532	3206	3335	3206
q14	258	211	210	210
q15	501	486	471	471
q16	510	418	383	383
q17	982	613	674	613
q18	8485	7729	7783	7729
q19	3739	1347	1559	1347
q20	647	320	316	316
q21	5184	4003	4059	4003
q22	408	344	358	344
Total cold run time: 116397 ms
Total hot run time: 41477 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4594	4411	4411	4411
q2	372	282	266	266
q3	3132	3004	2824	2824
q4	1878	1616	1644	1616
q5	5505	5486	5465	5465
q6	223	130	132	130
q7	2205	1830	1780	1780
q8	3222	3396	3355	3355
q9	8675	8661	8807	8661
q10	4057	3845	3755	3755
q11	591	490	485	485
q12	782	598	588	588
q13	17256	3173	3170	3170
q14	326	262	264	262
q15	531	486	485	485
q16	484	419	453	419
q17	1810	1491	1494	1491
q18	8020	7505	7275	7275
q19	1752	1608	1661	1608
q20	3042	1762	1799	1762
q21	4900	4708	4752	4708
q22	859	515	534	515
Total cold run time: 74216 ms
Total hot run time: 55031 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173974 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6387e4a5545e05885009b0b0b2949a66d3e464f, data reload: false

query1	919	385	384	384
query2	6451	2602	2351	2351
query3	6639	217	211	211
query4	19499	17310	17275	17275
query5	4155	445	443	443
query6	245	163	161	161
query7	4579	308	289	289
query8	323	286	283	283
query9	8359	2454	2439	2439
query10	454	309	284	284
query11	10591	10024	9968	9968
query12	135	92	90	90
query13	1642	380	368	368
query14	9520	7723	7563	7563
query15	242	189	188	188
query16	7701	269	275	269
query17	1394	553	507	507
query18	1834	267	274	267
query19	198	150	157	150
query20	99	83	88	83
query21	210	134	126	126
query22	4430	4142	4163	4142
query23	33584	33014	33051	33014
query24	11153	2802	2797	2797
query25	610	356	363	356
query26	934	153	154	153
query27	2314	322	324	322
query28	6819	2056	2068	2056
query29	871	612	608	608
query30	276	151	149	149
query31	941	792	723	723
query32	96	53	53	53
query33	767	287	278	278
query34	938	485	486	485
query35	755	617	607	607
query36	1086	940	929	929
query37	138	69	68	68
query38	2863	2742	2700	2700
query39	886	789	798	789
query40	208	126	129	126
query41	55	58	54	54
query42	126	97	96	96
query43	596	550	587	550
query44	1222	723	740	723
query45	195	166	169	166
query46	1081	708	702	702
query47	1826	1745	1776	1745
query48	370	317	305	305
query49	955	386	436	386
query50	776	395	410	395
query51	6838	6808	6830	6808
query52	112	92	93	92
query53	356	289	294	289
query54	871	455	446	446
query55	74	74	74	74
query56	277	260	256	256
query57	1130	1056	1045	1045
query58	265	255	255	255
query59	3395	3277	3356	3277
query60	288	281	271	271
query61	95	89	88	88
query62	655	449	452	449
query63	319	299	286	286
query64	8981	2207	1722	1722
query65	3174	3071	3126	3071
query66	811	330	319	319
query67	15486	14916	14965	14916
query68	4520	555	547	547
query69	472	305	303	303
query70	1132	1054	1152	1054
query71	388	285	281	281
query72	7277	5734	5891	5734
query73	741	332	330	330
query74	5912	5469	5396	5396
query75	3383	2691	2693	2691
query76	2417	912	905	905
query77	442	293	297	293
query78	10306	9818	9722	9722
query79	2277	519	527	519
query80	2200	483	478	478
query81	561	231	226	226
query82	1198	106	105	105
query83	303	186	181	181
query84	273	93	88	88
query85	1406	331	313	313
query86	483	314	309	309
query87	3227	3092	3070	3070
query88	3937	2454	2464	2454
query89	479	404	392	392
query90	1835	193	192	192
query91	136	108	110	108
query92	62	49	54	49
query93	2550	523	514	514
query94	1165	202	198	198
query95	422	328	326	326
query96	592	273	273	273
query97	3195	3046	3009	3009
query98	248	224	225	224
query99	1232	863	850	850
Total cold run time: 269958 ms
Total hot run time: 173974 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6387e4a5545e05885009b0b0b2949a66d3e464f, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.06	0.05
query4	1.67	0.06	0.07
query5	0.49	0.49	0.49
query6	1.12	0.74	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.54	0.49	0.49
query10	0.54	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.78	0.79
query15	0.83	0.81	0.81
query16	0.37	0.37	0.37
query17	1.01	0.97	0.95
query18	0.22	0.26	0.25
query19	1.83	1.69	1.68
query20	0.01	0.01	0.01
query21	15.60	0.68	0.67
query22	4.45	6.77	2.14
query23	18.27	1.44	1.25
query24	1.32	0.38	0.26
query25	0.14	0.09	0.08
query26	0.28	0.17	0.18
query27	0.08	0.09	0.07
query28	13.34	1.02	0.99
query29	13.87	3.36	3.29
query30	0.24	0.05	0.07
query31	2.87	0.38	0.39
query32	3.27	0.47	0.46
query33	2.96	2.89	2.91
query34	17.03	4.38	4.44
query35	4.53	4.48	4.65
query36	0.66	0.45	0.46
query37	0.19	0.17	0.16
query38	0.17	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.15
query41	0.09	0.05	0.04
query42	0.04	0.04	0.05
query43	0.04	0.03	0.03
Total cold run time: 110.41 s
Total hot run time: 30.75 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit e8420ac into apache:master Jun 4, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
…as possible (apache#35813)

if grouping sets only have one set and no group scalar function in project list,
transform it to trivial aggregation

for example, transform
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY GROUPING SETS ((k1, k2));
to
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY (k1, k2);
dataroaring pushed a commit that referenced this pull request Jun 7, 2024
…as possible (#35813)

if grouping sets only have one set and no group scalar function in project list,
transform it to trivial aggregation

for example, transform
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY GROUPING SETS ((k1, k2));
to
SELECT k1, k2, SUM(k3) FROM d_table GROUP BY (k1, k2);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants