Skip to content

[fix](nereids)modify agg function nullability in PhysicalHashAggregate #41943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
starocean999 merged 2 commits into from
Oct 17, 2024
Merged

[fix](nereids)modify agg function nullability in PhysicalHashAggregate #41943

starocean999 merged 2 commits into from
Oct 17, 2024

Conversation

@starocean999
Copy link
Contributor

@starocean999 starocean999 commented Oct 16, 2024

Proposed changes

this is a follow up pr of #41576 , the previous pr may miss some agg function in AggregateExpression, this pr fix it.

@doris-robot
Copy link

doris-robot commented Oct 16, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

starocean999 commented Oct 16, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 16, 2024

TPC-H: Total hot run time: 43013 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9282bafb87df28dd40e2dce256768e6aeebd32b1, data reload: false

------ Round 1 ----------------------------------
q1	17596	7800	7642	7642
q2	2026	295	278	278
q3	12012	1130	1177	1130
q4	10578	884	855	855
q5	7804	3276	3249	3249
q6	260	159	155	155
q7	1101	644	632	632
q8	9367	2013	2073	2013
q9	6808	6710	6684	6684
q10	7150	2500	2474	2474
q11	452	249	249	249
q12	447	225	226	225
q13	17786	3005	3019	3005
q14	259	210	221	210
q15	600	553	540	540
q16	659	587	597	587
q17	1008	556	560	556
q18	7352	6763	6740	6740
q19	1359	1068	1071	1068
q20	518	187	182	182
q21	4246	3525	3605	3525
q22	1104	1031	1014	1014
Total cold run time: 110492 ms
Total hot run time: 43013 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7683	7695	7897	7695
q2	367	238	240	238
q3	3165	3057	3132	3057
q4	2143	1853	1884	1853
q5	5914	5860	5959	5860
q6	268	151	154	151
q7	2286	1800	1871	1800
q8	3474	3792	3803	3792
q9	9081	9073	9043	9043
q10	3676	3706	3740	3706
q11	601	493	495	493
q12	830	629	615	615
q13	9061	3209	3249	3209
q14	324	299	275	275
q15	625	568	562	562
q16	691	648	655	648
q17	1991	1742	1702	1702
q18	8442	7826	7418	7418
q19	1880	1821	1641	1641
q20	2190	1867	1869	1867
q21	5659	5524	5513	5513
q22	1231	1073	1079	1073
Total cold run time: 71582 ms
Total hot run time: 62211 ms

@doris-robot
Copy link

doris-robot commented Oct 16, 2024

TPC-DS: Total hot run time: 192170 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9282bafb87df28dd40e2dce256768e6aeebd32b1, data reload: false

query1	878	410	401	401
query2	6281	2048	1999	1999
query3	8681	195	198	195
query4	34248	23648	23802	23648
query5	3536	476	466	466
query6	290	168	170	168
query7	4194	296	297	296
query8	305	232	231	231
query9	9621	2737	2738	2737
query10	476	266	279	266
query11	19171	15423	15430	15423
query12	144	102	99	99
query13	1569	442	422	422
query14	9994	7607	7165	7165
query15	261	175	173	173
query16	7979	490	496	490
query17	1657	594	577	577
query18	2235	307	316	307
query19	368	160	153	153
query20	139	116	114	114
query21	218	104	103	103
query22	4698	4607	4642	4607
query23	34736	34169	34126	34126
query24	10931	2676	2702	2676
query25	649	408	388	388
query26	1162	156	164	156
query27	2248	286	290	286
query28	7365	2446	2444	2444
query29	821	432	420	420
query30	257	150	152	150
query31	1039	814	770	770
query32	95	57	56	56
query33	775	288	290	288
query34	912	510	507	507
query35	935	750	760	750
query36	1094	932	962	932
query37	154	91	84	84
query38	4126	4024	3938	3938
query39	1507	1448	1395	1395
query40	214	97	98	97
query41	48	43	44	43
query42	125	96	101	96
query43	537	494	493	493
query44	1294	802	824	802
query45	198	168	163	163
query46	1136	688	676	676
query47	1940	1865	1840	1840
query48	423	316	338	316
query49	914	426	440	426
query50	817	380	420	380
query51	7072	6980	6929	6929
query52	96	90	83	83
query53	251	176	180	176
query54	1176	429	408	408
query55	77	77	73	73
query56	253	256	253	253
query57	1303	1161	1146	1146
query58	242	240	249	240
query59	3337	3187	2965	2965
query60	287	259	267	259
query61	102	97	99	97
query62	862	675	680	675
query63	219	184	180	180
query64	3931	650	648	648
query65	3285	3225	3246	3225
query66	841	295	299	295
query67	16028	15817	15693	15693
query68	4669	548	558	548
query69	526	283	287	283
query70	1206	1130	1155	1130
query71	347	268	277	268
query72	7408	3942	3932	3932
query73	773	353	365	353
query74	10215	8961	9002	8961
query75	3462	2640	2654	2640
query76	3266	919	896	896
query77	627	291	286	286
query78	10687	9668	9570	9570
query79	1878	590	593	590
query80	1501	439	450	439
query81	592	240	244	240
query82	686	138	137	137
query83	293	138	130	130
query84	282	72	68	68
query85	1667	290	283	283
query86	437	306	299	299
query87	4394	4301	4368	4301
query88	3092	2197	2167	2167
query89	401	290	287	287
query90	2122	185	185	185
query91	139	99	99	99
query92	59	47	50	47
query93	1531	531	534	531
query94	1192	279	306	279
query95	351	240	249	240
query96	612	279	283	279
query97	3297	3149	3139	3139
query98	216	203	191	191
query99	1553	1289	1296	1289
Total cold run time: 302238 ms
Total hot run time: 192170 ms

@doris-robot
Copy link

doris-robot commented Oct 16, 2024

ClickBench: Total hot run time: 32.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9282bafb87df28dd40e2dce256768e6aeebd32b1, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.07
query4	1.65	0.10	0.10
query5	0.51	0.52	0.49
query6	1.12	0.72	0.73
query7	0.02	0.01	0.01
query8	0.04	0.05	0.03
query9	0.56	0.49	0.55
query10	0.56	0.56	0.54
query11	0.14	0.11	0.11
query12	0.14	0.11	0.10
query13	0.60	0.60	0.59
query14	2.72	2.72	2.71
query15	0.90	0.82	0.84
query16	0.38	0.36	0.38
query17	1.05	1.06	1.01
query18	0.20	0.20	0.20
query19	1.92	1.82	1.93
query20	0.02	0.01	0.00
query21	15.36	0.57	0.57
query22	2.34	2.41	1.68
query23	16.88	1.03	0.69
query24	2.97	1.80	0.66
query25	0.30	0.20	0.09
query26	0.34	0.13	0.12
query27	0.04	0.04	0.04
query28	10.46	1.10	1.07
query29	12.54	3.23	3.24
query30	0.24	0.06	0.06
query31	2.87	0.38	0.38
query32	3.29	0.46	0.45
query33	3.02	3.05	3.05
query34	16.89	4.48	4.42
query35	4.51	4.52	4.53
query36	0.68	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.06 s
Total hot run time: 32.03 s

@starocean999 starocean999 marked this pull request as ready for review October 17, 2024 02:03
@starocean999
Copy link
Contributor Author

starocean999 commented Oct 17, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 17, 2024

TPC-H: Total hot run time: 40997 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7f947913d848e2c99c9a24229b95405b6abb96e1, data reload: false

------ Round 1 ----------------------------------
q1	17573	7439	7275	7275
q2	2029	271	274	271
q3	12203	1071	1183	1071
q4	10569	855	892	855
q5	7754	3077	3066	3066
q6	238	151	151	151
q7	1032	605	595	595
q8	9359	1961	1907	1907
q9	6570	6476	6361	6361
q10	7024	2382	2417	2382
q11	446	242	233	233
q12	405	222	216	216
q13	17761	2959	2995	2959
q14	251	221	208	208
q15	576	519	512	512
q16	662	581	586	581
q17	968	576	496	496
q18	7290	6738	6716	6716
q19	1346	1011	955	955
q20	473	183	188	183
q21	3971	3078	3017	3017
q22	1127	1003	987	987
Total cold run time: 109627 ms
Total hot run time: 40997 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7290	7210	7240	7210
q2	330	238	230	230
q3	3058	2877	2918	2877
q4	2091	1849	1816	1816
q5	5777	5736	5779	5736
q6	231	146	143	143
q7	2225	1854	1799	1799
q8	3341	3534	3469	3469
q9	8924	8948	8926	8926
q10	3574	3561	3595	3561
q11	592	487	479	479
q12	864	658	620	620
q13	10819	3205	3207	3205
q14	303	267	293	267
q15	562	533	536	533
q16	674	646	630	630
q17	1856	1627	1612	1612
q18	8326	7773	7592	7592
q19	1713	1535	1493	1493
q20	2124	1877	1899	1877
q21	5364	5382	5211	5211
q22	1156	1041	1033	1033
Total cold run time: 71194 ms
Total hot run time: 60319 ms

@doris-robot
Copy link

doris-robot commented Oct 17, 2024

TPC-DS: Total hot run time: 192136 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7f947913d848e2c99c9a24229b95405b6abb96e1, data reload: false

query1	942	421	402	402
query2	6253	2133	2074	2074
query3	8685	193	199	193
query4	34527	23554	23518	23518
query5	3437	475	465	465
query6	283	173	180	173
query7	4201	296	301	296
query8	301	230	231	230
query9	9190	2726	2718	2718
query10	462	289	267	267
query11	18243	15311	15328	15311
query12	152	102	103	102
query13	1560	441	415	415
query14	10076	7088	7404	7088
query15	264	169	178	169
query16	7987	490	496	490
query17	1720	603	566	566
query18	2134	306	310	306
query19	373	152	150	150
query20	122	113	117	113
query21	212	108	111	108
query22	5073	4755	4590	4590
query23	35583	34015	34587	34015
query24	10934	2814	2770	2770
query25	554	408	400	400
query26	722	162	158	158
query27	2024	291	292	291
query28	6832	2413	2416	2413
query29	684	418	422	418
query30	263	150	146	146
query31	1044	800	831	800
query32	97	53	55	53
query33	773	287	301	287
query34	924	531	532	531
query35	889	738	758	738
query36	1115	957	977	957
query37	155	95	88	88
query38	4108	3922	3933	3922
query39	1450	1405	1445	1405
query40	205	97	100	97
query41	48	45	46	45
query42	126	105	98	98
query43	528	502	491	491
query44	1261	812	803	803
query45	202	165	167	165
query46	1155	717	682	682
query47	1935	1856	1891	1856
query48	418	319	322	319
query49	921	436	448	436
query50	829	383	378	378
query51	7110	6936	6962	6936
query52	100	88	86	86
query53	257	176	178	176
query54	1159	414	406	406
query55	76	71	75	71
query56	272	249	265	249
query57	1315	1137	1143	1137
query58	238	241	232	232
query59	3127	3039	3055	3039
query60	289	263	263	263
query61	105	101	104	101
query62	855	686	690	686
query63	223	184	185	184
query64	3923	637	628	628
query65	3310	3229	3210	3210
query66	853	310	296	296
query67	15998	15674	15633	15633
query68	4412	564	562	562
query69	478	290	285	285
query70	1193	1175	1156	1156
query71	343	274	270	270
query72	7004	3970	3969	3969
query73	775	356	371	356
query74	10280	8988	9131	8988
query75	3457	2676	2651	2651
query76	2465	883	844	844
query77	622	313	336	313
query78	10598	9660	9618	9618
query79	2495	613	607	607
query80	2624	456	447	447
query81	597	238	239	238
query82	699	136	141	136
query83	307	132	136	132
query84	288	66	77	66
query85	1569	284	282	282
query86	432	282	286	282
query87	4581	4358	4327	4327
query88	3193	2232	2165	2165
query89	401	291	285	285
query90	2004	185	178	178
query91	151	98	94	94
query92	60	45	48	45
query93	1909	535	546	535
query94	1079	292	293	292
query95	358	249	250	249
query96	638	282	277	277
query97	3233	3135	3141	3135
query98	217	211	196	196
query99	1570	1317	1328	1317
Total cold run time: 301218 ms
Total hot run time: 192136 ms

@doris-robot
Copy link

doris-robot commented Oct 17, 2024

ClickBench: Total hot run time: 32.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7f947913d848e2c99c9a24229b95405b6abb96e1, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.52	0.49	0.52
query6	1.15	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.50	0.49
query10	0.56	0.53	0.56
query11	0.14	0.10	0.10
query12	0.16	0.11	0.11
query13	0.61	0.59	0.59
query14	2.84	2.83	2.86
query15	0.89	0.83	0.83
query16	0.38	0.37	0.40
query17	1.05	0.96	1.07
query18	0.19	0.19	0.19
query19	1.99	1.87	2.02
query20	0.02	0.01	0.01
query21	15.35	0.57	0.56
query22	2.98	2.16	1.79
query23	16.93	0.96	0.71
query24	3.34	1.12	0.86
query25	0.28	0.12	0.09
query26	0.44	0.14	0.13
query27	0.05	0.05	0.04
query28	10.48	1.09	1.07
query29	12.53	3.23	3.21
query30	0.24	0.06	0.06
query31	2.87	0.40	0.39
query32	3.25	0.45	0.46
query33	2.99	3.05	3.02
query34	16.98	4.43	4.44
query35	4.49	4.54	4.48
query36	0.68	0.48	0.48
query37	0.08	0.06	0.05
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 107.44 s
Total hot run time: 32.39 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2024

PR approved by anyone and no changes requested.

zhiqiang-hhhh
zhiqiang-hhhh approved these changes Oct 17, 2024
View reviewed changes
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@starocean999 starocean999 mentioned this pull request Oct 17, 2024
Merged
@starocean999 starocean999 merged commit 64fd929 into apache:master Oct 17, 2024
This was referenced Oct 17, 2024
Merged
Merged
starocean999 added a commit that referenced this pull request Oct 22, 2024
@starocean999
[fix](nereids)modify agg function nullability in PhysicalHashAggregate (
d002261 
#42101)

## Proposed changes

pick from master #41943 and
#42093

<!--Describe your changes.-->
starocean999 added a commit that referenced this pull request Oct 22, 2024
@starocean999
[fix](nereids)modify agg function nullability in PhysicalHashAggregate (
a21bb80 
#42054)

## Proposed changes
pick from master #41943

<!--Describe your changes.-->
starocean999 added a commit that referenced this pull request Oct 22, 2024
@starocean999
[fix](nereids)modify agg function nullability in PhysicalHashAggregate (
803c052 
#42018)

## Proposed changes

pick from master #41943

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@zhiqiang-hhhh zhiqiang-hhhh zhiqiang-hhhh approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.16-merged dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@starocean999 @doris-robot @zhiqiang-hhhh @morrySnow