Skip to content

[fix](nereids)the preagg state for count(*) is wrong #35254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 2 commits into from
May 24, 2024
Merged

[fix](nereids)the preagg state for count(*) is wrong #35254

morrySnow merged 2 commits into from
May 24, 2024

Conversation

@starocean999
Copy link
Contributor

@starocean999 starocean999 commented May 23, 2024 •
edited
Loading

Proposed changes

fix bug introduced by #34738
set preagg to off if agg function has no children( like count(*) )

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented May 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

starocean999 commented May 23, 2024

run buildall

@starocean999
Copy link
Contributor Author

starocean999 commented May 23, 2024

run buildall

@doris-robot
Copy link

doris-robot commented May 23, 2024

TPC-H: Total hot run time: 41394 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 08176575370621c11ead6626bc016590e612dd17, data reload: false

------ Round 1 ----------------------------------
q1	17699	5242	4356	4356
q2	2018	192	194	192
q3	10547	1178	1231	1178
q4	10189	813	912	813
q5	7489	2818	2727	2727
q6	227	136	131	131
q7	997	607	606	606
q8	9254	2173	2135	2135
q9	9150	6782	6778	6778
q10	8967	3849	3921	3849
q11	475	230	243	230
q12	415	224	221	221
q13	17796	3222	3176	3176
q14	270	219	223	219
q15	529	470	496	470
q16	537	410	395	395
q17	1022	751	729	729
q18	8526	8051	7662	7662
q19	7196	1587	1545	1545
q20	693	323	326	323
q21	5139	3373	4045	3373
q22	362	293	286	286
Total cold run time: 119497 ms
Total hot run time: 41394 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4517	4411	4283	4283
q2	376	245	270	245
q3	3017	2739	2750	2739
q4	1931	1605	1588	1588
q5	5245	5310	5274	5274
q6	207	121	125	121
q7	2188	1754	1711	1711
q8	3199	3345	3330	3330
q9	8439	8356	8328	8328
q10	3939	3707	3728	3707
q11	578	476	494	476
q12	762	581	603	581
q13	17491	2996	3091	2996
q14	292	268	264	264
q15	518	479	459	459
q16	465	421	435	421
q17	1788	1506	1464	1464
q18	7696	7593	7347	7347
q19	1693	1594	1608	1594
q20	2004	1801	1779	1779
q21	7076	4675	4720	4675
q22	578	489	486	486
Total cold run time: 73999 ms
Total hot run time: 53868 ms

@doris-robot
Copy link

doris-robot commented May 23, 2024

TPC-DS: Total hot run time: 169448 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 08176575370621c11ead6626bc016590e612dd17, data reload: false

query1	939	387	376	376
query2	6452	2414	2337	2337
query3	6647	211	207	207
query4	21055	17305	17416	17305
query5	4097	425	408	408
query6	243	156	166	156
query7	4589	294	299	294
query8	234	184	184	184
query9	8492	2467	2440	2440
query10	456	277	283	277
query11	10560	10030	10103	10030
query12	140	96	89	89
query13	1651	376	377	376
query14	9676	7696	6963	6963
query15	217	167	166	166
query16	7807	260	265	260
query17	1704	536	514	514
query18	1941	297	272	272
query19	216	155	157	155
query20	100	91	85	85
query21	196	127	139	127
query22	4036	3899	3878	3878
query23	33661	33052	32754	32754
query24	11717	2847	2832	2832
query25	648	358	386	358
query26	1809	161	158	158
query27	3025	339	324	324
query28	7598	2086	2085	2085
query29	1157	623	596	596
query30	309	180	170	170
query31	951	749	753	749
query32	85	52	56	52
query33	755	269	266	266
query34	1005	477	485	477
query35	752	608	600	600
query36	1047	916	909	909
query37	268	72	71	71
query38	2880	2779	2799	2779
query39	849	796	774	774
query40	272	124	123	123
query41	46	43	46	43
query42	101	94	98	94
query43	535	555	560	555
query44	1207	721	737	721
query45	186	165	163	163
query46	1089	706	740	706
query47	1867	1771	1795	1771
query48	378	296	300	296
query49	1103	376	380	376
query50	778	392	391	391
query51	6821	6707	6702	6702
query52	103	88	91	88
query53	362	285	286	285
query54	996	436	418	418
query55	73	72	70	70
query56	256	244	253	244
query57	1150	1034	1053	1034
query58	223	213	208	208
query59	3595	3262	3394	3262
query60	278	263	249	249
query61	88	99	92	92
query62	667	461	470	461
query63	313	288	289	288
query64	9794	2227	1723	1723
query65	3182	3120	3142	3120
query66	1370	340	324	324
query67	15173	14843	14733	14733
query68	4749	535	534	534
query69	442	274	264	264
query70	1148	1041	1095	1041
query71	456	272	271	271
query72	7749	5479	2719	2719
query73	739	334	320	320
query74	6024	5618	5662	5618
query75	3364	2651	2651	2651
query76	2842	1073	984	984
query77	437	265	264	264
query78	10314	9755	9743	9743
query79	1821	523	517	517
query80	1126	442	427	427
query81	534	244	245	244
query82	972	100	92	92
query83	245	176	173	173
query84	249	84	91	84
query85	1428	396	266	266
query86	481	326	292	292
query87	3267	3105	3136	3105
query88	3580	2425	2417	2417
query89	476	398	380	380
query90	2070	188	192	188
query91	126	96	97	96
query92	58	49	45	45
query93	1745	508	508	508
query94	1362	188	185	185
query95	405	318	315	315
query96	592	276	269	269
query97	3134	2968	3043	2968
query98	234	219	217	217
query99	1107	878	858	858
Total cold run time: 275246 ms
Total hot run time: 169448 ms

@doris-robot
Copy link

doris-robot commented May 23, 2024

ClickBench: Total hot run time: 31.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 08176575370621c11ead6626bc016590e612dd17, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.66	0.09	0.08
query5	0.50	0.48	0.51
query6	1.11	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.55	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.60	0.58	0.60
query14	0.77	0.79	0.79
query15	0.84	0.84	0.82
query16	0.36	0.36	0.36
query17	1.02	1.02	1.03
query18	0.23	0.25	0.25
query19	1.80	1.72	1.74
query20	0.01	0.01	0.01
query21	15.54	0.68	0.67
query22	4.18	6.83	2.70
query23	18.33	1.44	1.26
query24	1.75	0.27	0.20
query25	0.15	0.08	0.09
query26	0.26	0.17	0.17
query27	0.08	0.07	0.08
query28	13.31	1.01	1.01
query29	12.67	3.32	3.27
query30	0.24	0.05	0.05
query31	2.89	0.40	0.40
query32	3.25	0.47	0.47
query33	2.85	2.92	2.85
query34	16.91	4.46	4.42
query35	4.48	4.45	4.59
query36	0.66	0.46	0.47
query37	0.18	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.17	0.15	0.15
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.01 s
Total hot run time: 31.39 s

@starocean999
Copy link
Contributor Author

starocean999 commented May 24, 2024

run buildall

@doris-robot
Copy link

doris-robot commented May 24, 2024

TPC-H: Total hot run time: 41052 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 927c756db7ec89c17cdbb93478a1ab828fb731bd, data reload: false

------ Round 1 ----------------------------------
q1	17613	4341	4195	4195
q2	2018	191	180	180
q3	10477	1242	1243	1242
q4	10193	797	849	797
q5	7490	2728	2699	2699
q6	225	130	132	130
q7	972	606	608	606
q8	9245	2179	2090	2090
q9	10470	6698	6750	6698
q10	9708	3950	3940	3940
q11	433	238	237	237
q12	422	216	226	216
q13	17554	3199	3187	3187
q14	250	213	213	213
q15	529	475	474	474
q16	494	403	383	383
q17	966	658	686	658
q18	8317	7689	7820	7689
q19	2028	1513	1541	1513
q20	641	321	313	313
q21	5216	3325	3330	3325
q22	360	267	270	267
Total cold run time: 115621 ms
Total hot run time: 41052 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4473	4354	4389	4354
q2	385	280	273	273
q3	3135	2907	2938	2907
q4	1966	1640	1619	1619
q5	5358	5597	5486	5486
q6	209	123	127	123
q7	2154	1856	1797	1797
q8	3232	3405	3395	3395
q9	8582	8701	8654	8654
q10	4061	3672	3772	3672
q11	578	491	514	491
q12	788	638	637	637
q13	16052	3139	3141	3139
q14	288	284	278	278
q15	530	504	506	504
q16	500	440	444	440
q17	1818	1515	1451	1451
q18	7696	7664	7559	7559
q19	1655	1546	1544	1544
q20	1973	1782	1781	1781
q21	8020	4845	4636	4636
q22	581	469	478	469
Total cold run time: 74034 ms
Total hot run time: 55209 ms

@doris-robot
Copy link

doris-robot commented May 24, 2024

TPC-DS: Total hot run time: 169404 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 927c756db7ec89c17cdbb93478a1ab828fb731bd, data reload: false

query1	917	383	372	372
query2	6436	2560	2338	2338
query3	6645	207	206	206
query4	19642	17415	17293	17293
query5	4125	432	422	422
query6	239	157	157	157
query7	4588	312	292	292
query8	240	188	185	185
query9	8629	2373	2368	2368
query10	454	296	263	263
query11	10596	10167	10167	10167
query12	136	88	93	88
query13	1656	379	367	367
query14	9870	8573	7099	7099
query15	218	169	170	169
query16	7754	263	258	258
query17	1605	552	511	511
query18	1949	271	268	268
query19	203	147	154	147
query20	89	80	85	80
query21	192	129	129	129
query22	4159	3983	3948	3948
query23	33407	32953	33030	32953
query24	5715	2902	2940	2902
query25	469	346	374	346
query26	689	155	155	155
query27	1842	334	323	323
query28	3753	2032	2023	2023
query29	836	609	596	596
query30	226	169	178	169
query31	950	762	753	753
query32	61	52	65	52
query33	506	265	263	263
query34	859	466	477	466
query35	726	598	584	584
query36	1050	919	927	919
query37	108	74	75	74
query38	2864	2813	2738	2738
query39	849	787	814	787
query40	197	122	126	122
query41	47	43	45	43
query42	104	97	104	97
query43	581	571	541	541
query44	1046	723	730	723
query45	181	169	165	165
query46	1057	708	695	695
query47	1857	1770	1766	1766
query48	366	288	285	285
query49	768	386	383	383
query50	773	380	370	370
query51	7046	6682	6773	6682
query52	102	92	91	91
query53	357	319	284	284
query54	529	427	443	427
query55	76	74	75	74
query56	266	242	242	242
query57	1125	1036	1078	1036
query58	229	202	221	202
query59	3485	3046	3081	3046
query60	272	257	253	253
query61	88	90	88	88
query62	586	458	448	448
query63	307	283	286	283
query64	2531	1733	1696	1696
query65	3189	3097	3083	3083
query66	801	324	331	324
query67	15166	14748	14857	14748
query68	4579	537	536	536
query69	438	264	312	264
query70	1081	1168	1142	1142
query71	382	279	263	263
query72	7665	5342	2729	2729
query73	706	320	321	320
query74	5969	5648	5630	5630
query75	3246	2679	2610	2610
query76	2356	1005	1018	1005
query77	369	271	271	271
query78	10196	9785	9655	9655
query79	2447	523	520	520
query80	1082	450	440	440
query81	526	243	244	243
query82	941	99	94	94
query83	244	173	174	173
query84	254	86	91	86
query85	979	320	370	320
query86	466	305	279	279
query87	3315	3116	3103	3103
query88	3984	2319	2317	2317
query89	475	393	383	383
query90	1998	186	188	186
query91	125	95	95	95
query92	65	54	48	48
query93	2044	506	492	492
query94	1155	189	186	186
query95	408	309	311	309
query96	585	263	262	262
query97	3193	2974	2987	2974
query98	251	214	214	214
query99	1219	848	828	828
Total cold run time: 251127 ms
Total hot run time: 169404 ms

@doris-robot
Copy link

doris-robot commented May 24, 2024

ClickBench: Total hot run time: 30.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 927c756db7ec89c17cdbb93478a1ab828fb731bd, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.06	0.05
query4	1.66	0.08	0.07
query5	0.51	0.49	0.51
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.52	0.48	0.48
query10	0.53	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.59	0.58	0.60
query14	0.79	0.76	0.79
query15	0.82	0.81	0.80
query16	0.37	0.37	0.37
query17	0.98	0.96	0.95
query18	0.22	0.25	0.25
query19	1.77	1.71	1.69
query20	0.02	0.01	0.01
query21	15.43	0.75	0.69
query22	4.99	6.39	1.68
query23	18.24	1.37	1.31
query24	2.02	0.22	0.20
query25	0.16	0.08	0.08
query26	0.27	0.17	0.16
query27	0.08	0.08	0.08
query28	13.25	1.00	1.00
query29	13.31	3.27	3.21
query30	0.24	0.05	0.05
query31	2.88	0.38	0.38
query32	3.35	0.46	0.46
query33	2.88	2.95	2.91
query34	17.16	4.45	4.50
query35	4.48	4.51	4.60
query36	0.65	0.48	0.46
query37	0.17	0.16	0.16
query38	0.14	0.14	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.15
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.05	0.05
Total cold run time: 110.71 s
Total hot run time: 30.32 s

@starocean999 starocean999 mentioned this pull request May 24, 2024
Merged
@starocean999 starocean999 marked this pull request as ready for review May 24, 2024 06:27
@morrySnow morrySnow added the p0_b label May 24, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 24, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 24, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented May 24, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 31529db into apache:master May 24, 2024
dataroaring pushed a commit that referenced this pull request May 26, 2024
@starocean999 @dataroaring
[fix](nereids) the preagg state for count(*) is wrong (#35254)
833c5a7 
fix bug introduced by #34738
set preagg to off if agg function has no children( like count(*) )
seawinde pushed a commit to seawinde/doris that referenced this pull request May 27, 2024
@starocean999 @seawinde
[fix](nereids) the preagg state for count(*) is wrong (apache#35254)
7245ae9 
fix bug introduced by apache#34738
set preagg to off if agg function has no children( like count(*) )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@starocean999 @doris-robot @englefly @morrySnow @yiguolei @dataroaring