Skip to content

[fix](mtmv) Fix aggregate function roll up fail when function is always nullable #52960 #54785

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
seawinde wants to merge 2 commits into from
Closed

[fix](mtmv) Fix aggregate function roll up fail when function is always nullable #52960 #54785

seawinde wants to merge 2 commits into from

Conversation

@seawinde
Copy link
Contributor

@seawinde seawinde commented Aug 14, 2025

What problem does this PR solve?

pr: #52960
commitId: 0a5cd95

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@seawinde
[fix](mtmv) Fix aggregate function roll up fail when function is alwa…
e92f440 
…ys nullable (apache#52960)

### What problem does this PR solve?

Related PR: apache#36318

Problem Summary:

materaialized view def is as fllowing:

create materialized view as
select k1, k3, sum(k2), count(k4) from ${tblName} group by k1, k3;

`sum(k2) ` nullable is true
if query is as following, would rewritten fail with err info 'query
aggregate function roll up fail', the pr fix this

select sum(distinct k1) from agg_use_key_direct
@seawinde
fix code usage
b5e57ab
@seawinde seawinde requested a review from dataroaring as a code owner August 14, 2025 10:10
@seawinde
Copy link
Contributor Author

seawinde commented Aug 14, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Aug 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

doris-robot commented Aug 14, 2025

TPC-H: Total hot run time: 39964 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b5e57abbf582faa54eb7b79809ed5557f1b4ec6e, data reload: false

------ Round 1 ----------------------------------
q1	17644	6876	6680	6680
q2	2036	199	183	183
q3	10612	1124	1132	1124
q4	10422	748	744	744
q5	7735	2824	2802	2802
q6	212	138	136	136
q7	992	649	640	640
q8	9379	1904	1987	1904
q9	6552	6379	6363	6363
q10	7028	2262	2324	2262
q11	476	267	271	267
q12	421	225	220	220
q13	17916	2990	2984	2984
q14	232	205	212	205
q15	506	479	460	460
q16	487	378	387	378
q17	994	650	534	534
q18	7268	6677	6604	6604
q19	1381	1052	1043	1043
q20	506	206	209	206
q21	3893	3210	3316	3210
q22	1115	1015	1021	1015
Total cold run time: 107807 ms
Total hot run time: 39964 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6653	6620	7038	6620
q2	328	232	234	232
q3	2898	2975	2972	2972
q4	2048	1876	1791	1791
q5	5698	5714	5731	5714
q6	212	132	129	129
q7	2291	1808	1803	1803
q8	3399	3604	3547	3547
q9	8850	8956	8894	8894
q10	3617	3561	3576	3561
q11	593	495	525	495
q12	798	599	614	599
q13	8110	3146	3158	3146
q14	314	264	287	264
q15	510	449	460	449
q16	502	437	463	437
q17	1869	1664	1613	1613
q18	8565	7943	8077	7943
q19	1667	1587	1521	1521
q20	2143	1866	1920	1866
q21	5312	4991	5130	4991
q22	1176	1060	1065	1060
Total cold run time: 67553 ms
Total hot run time: 59647 ms

@doris-robot
Copy link

doris-robot commented Aug 14, 2025

TPC-DS: Total hot run time: 193648 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b5e57abbf582faa54eb7b79809ed5557f1b4ec6e, data reload: false

query1	918	408	402	402
query2	6231	2020	1904	1904
query3	8690	198	194	194
query4	33794	23525	23546	23525
query5	3475	457	435	435
query6	284	196	180	180
query7	4202	319	326	319
query8	298	231	224	224
query9	9594	2608	2595	2595
query10	491	270	267	267
query11	18095	15267	15210	15210
query12	156	108	103	103
query13	1544	440	427	427
query14	9611	7627	7467	7467
query15	248	188	194	188
query16	8132	496	463	463
query17	1692	599	579	579
query18	2148	315	325	315
query19	377	169	160	160
query20	122	115	124	115
query21	205	107	111	107
query22	4755	4403	4403	4403
query23	34929	34659	34120	34120
query24	11860	2978	2948	2948
query25	704	427	436	427
query26	1780	179	176	176
query27	2698	356	365	356
query28	7559	2176	2146	2146
query29	1062	460	465	460
query30	268	162	169	162
query31	1056	863	842	842
query32	93	59	53	53
query33	766	308	305	305
query34	988	531	531	531
query35	872	735	751	735
query36	1129	940	937	937
query37	212	65	70	65
query38	4080	4044	3984	3984
query39	1540	1438	1473	1438
query40	251	97	103	97
query41	50	50	52	50
query42	117	103	103	103
query43	522	485	489	485
query44	1317	823	822	822
query45	187	167	173	167
query46	1147	751	741	741
query47	1988	1900	1919	1900
query48	475	409	371	371
query49	1130	396	406	396
query50	852	439	459	439
query51	7523	7369	7552	7369
query52	99	88	88	88
query53	255	183	182	182
query54	1249	471	483	471
query55	77	80	79	79
query56	256	248	247	247
query57	1358	1198	1206	1198
query58	220	202	230	202
query59	3391	3157	3176	3157
query60	293	259	251	251
query61	115	113	109	109
query62	866	692	719	692
query63	230	196	197	196
query64	5208	667	661	661
query65	3379	3337	3339	3337
query66	1558	306	321	306
query67	16244	15677	15565	15565
query68	4952	572	579	572
query69	436	262	261	261
query70	1152	1116	1098	1098
query71	344	254	257	254
query72	6145	4070	4081	4070
query73	757	349	362	349
query74	10727	9194	9067	9067
query75	3373	2650	2637	2637
query76	2709	1072	991	991
query77	399	277	277	277
query78	10585	9659	9639	9639
query79	2518	615	611	611
query80	1029	428	437	428
query81	542	216	222	216
query82	605	89	85	85
query83	227	141	144	141
query84	228	78	91	78
query85	1793	298	295	295
query86	470	294	297	294
query87	4445	4256	4236	4236
query88	4041	2389	2369	2369
query89	413	295	295	295
query90	1995	180	184	180
query91	178	150	148	148
query92	59	49	49	49
query93	2373	569	560	560
query94	890	308	306	306
query95	364	255	259	255
query96	608	282	280	280
query97	3310	3196	3163	3163
query98	225	201	199	199
query99	1508	1298	1297	1297
Total cold run time: 306415 ms
Total hot run time: 193648 ms

@doris-robot
Copy link

doris-robot commented Aug 14, 2025

ClickBench: Total hot run time: 30.55 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b5e57abbf582faa54eb7b79809ed5557f1b4ec6e, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.62	0.11	0.10
query5	0.53	0.52	0.53
query6	1.12	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.58	0.50	0.49
query10	0.57	0.56	0.55
query11	0.14	0.10	0.11
query12	0.14	0.11	0.10
query13	0.60	0.59	0.59
query14	0.79	0.80	0.80
query15	0.83	0.82	0.82
query16	0.41	0.38	0.37
query17	1.06	1.06	1.06
query18	0.24	0.22	0.23
query19	1.98	1.86	1.89
query20	0.02	0.01	0.02
query21	15.38	0.59	0.59
query22	2.25	2.68	1.30
query23	16.85	1.13	0.96
query24	3.35	2.11	1.57
query25	0.16	0.16	0.11
query26	0.60	0.13	0.14
query27	0.04	0.04	0.04
query28	8.54	0.52	0.46
query29	12.61	3.21	3.20
query30	0.25	0.06	0.06
query31	2.87	0.38	0.38
query32	3.28	0.45	0.46
query33	3.00	3.00	3.04
query34	17.23	4.50	4.51
query35	4.51	4.54	4.56
query36	0.67	0.48	0.47
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.09 s
Total hot run time: 30.55 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 14, 2025

FE UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉
Increment coverage report
Complete coverage report

@github-actions
Copy link
Contributor

github-actions bot commented Sep 10, 2025

PR approved by anyone and no changes requested.

@morrySnow morrySnow closed this Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring Awaiting requested review from dataroaring

1 more reviewer

@zddr zddr zddr approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@seawinde @Thearas @doris-robot @hello-stephen @zddr @morrySnow