Skip to content

[fix](mtmv) Fix query rewrite by mv fail when both use cte and group sets #57741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
starocean999 merged 4 commits into from
Dec 12, 2025
Merged

[fix](mtmv) Fix query rewrite by mv fail when both use cte and group sets #57741

starocean999 merged 4 commits into from
Dec 12, 2025

Conversation

@seawinde
Copy link
Contributor

@seawinde seawinde commented Nov 5, 2025

What problem does this PR solve?

Fix query rewrite by mv fail when both use cte and group sets

Such as mv def is as following:

            select l_shipdate, o_orderdate, l_partkey, l_suppkey,
            sum(o_totalprice) as sum_total,
            max(o_totalprice) as max_total,
            min(o_totalprice) as min_total,
            count(*) as count_all,
            bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) as bitmap_union_basic
            from lineitem
            left join orders on lineitem.l_orderkey = orders.o_orderkey and l_shipdate = o_orderdate
            group by
            l_shipdate,
            o_orderdate,
            l_partkey,
            l_suppkey;

query is as fllowing, use both group sets and cte, would rewrite fail, the pr fix this

            with t as (
                select t1.l_partkey, t1.l_suppkey, o_orderdate,
                grouping(t1.l_suppkey),
                grouping(o_orderdate),
                grouping_id(t1.l_partkey, t1.l_suppkey),
                grouping_id(t1.l_partkey, t1.l_suppkey, o_orderdate),
                sum(o_totalprice),
                max(o_totalprice),
                min(o_totalprice),
                count(*),
                count(distinct case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)
                from (select * from lineitem where l_shipdate = '2023-12-11') t1
                left join orders on t1.l_orderkey = orders.o_orderkey and t1.l_shipdate = o_orderdate
                group by
                GROUPING SETS ((l_shipdate, o_orderdate, l_partkey), (l_partkey, l_suppkey), (l_suppkey), ())
            )
            select t1.l_suppkey, t2.o_orderdate
            from
            t t1
            inner join
            t t2 on t1.l_partkey = t2.l_partkey;

Issue Number: close #xxx

Related PR: #36056

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde
Copy link
Contributor Author

seawinde commented Nov 5, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 5, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Nov 5, 2025

TPC-DS: Total hot run time: 189866 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9afd1a1acb18ac51a93c506c37d29531cc958dd, data reload: false

query1	1053	412	397	397
query2	6577	1711	1687	1687
query3	6749	222	223	222
query4	26651	23597	23363	23363
query5	4936	668	492	492
query6	342	250	257	250
query7	4655	506	301	301
query8	321	282	262	262
query9	8739	2602	2590	2590
query10	511	350	300	300
query11	15622	15178	14897	14897
query12	193	122	117	117
query13	1694	577	442	442
query14	12569	9402	9466	9402
query15	208	194	186	186
query16	7685	715	491	491
query17	1614	809	666	666
query18	2129	458	368	368
query19	229	230	195	195
query20	149	147	144	144
query21	212	160	131	131
query22	4691	4678	4565	4565
query23	35034	33669	34373	33669
query24	8290	2601	2589	2589
query25	593	564	463	463
query26	1324	287	177	177
query27	2970	522	346	346
query28	4392	2224	2213	2213
query29	862	662	543	543
query30	297	229	202	202
query31	960	852	798	798
query32	87	78	78	78
query33	613	419	355	355
query34	832	891	552	552
query35	854	912	787	787
query36	1008	1060	962	962
query37	144	115	92	92
query38	3604	3572	3486	3486
query39	1481	1418	1431	1418
query40	221	122	117	117
query41	59	58	55	55
query42	116	110	110	110
query43	491	504	465	465
query44	1243	758	747	747
query45	184	175	171	171
query46	879	1005	659	659
query47	1727	1779	1727	1727
query48	393	428	318	318
query49	763	527	395	395
query50	641	690	406	406
query51	3922	3913	3876	3876
query52	109	110	100	100
query53	237	275	197	197
query54	305	285	269	269
query55	91	84	79	79
query56	324	304	318	304
query57	1172	1188	1119	1119
query58	280	277	272	272
query59	2528	2643	2519	2519
query60	355	348	317	317
query61	161	163	160	160
query62	781	728	665	665
query63	230	198	200	198
query64	4470	1172	862	862
query65	4042	3963	3939	3939
query66	1093	455	333	333
query67	15434	15217	15013	15013
query68	8511	957	597	597
query69	490	326	299	299
query70	1357	1295	1357	1295
query71	508	351	313	313
query72	5885	4918	4365	4365
query73	698	622	367	367
query74	8834	9188	8981	8981
query75	4065	3388	2885	2885
query76	3821	1149	784	784
query77	823	414	302	302
query78	9471	9938	8985	8985
query79	2381	821	589	589
query80	669	560	508	508
query81	502	262	225	225
query82	428	171	133	133
query83	286	272	251	251
query84	297	116	94	94
query85	912	474	440	440
query86	335	324	308	308
query87	3675	3710	3657	3657
query88	3508	2278	2256	2256
query89	390	322	295	295
query90	2022	223	223	223
query91	178	173	134	134
query92	81	67	71	67
query93	1640	978	641	641
query94	696	449	343	343
query95	409	325	303	303
query96	493	567	283	283
query97	2967	2973	2876	2876
query98	247	213	213	213
query99	1434	1435	1264	1264
Total cold run time: 281138 ms
Total hot run time: 189866 ms

@doris-robot
Copy link

doris-robot commented Nov 5, 2025

ClickBench: Total hot run time: 27.62 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9afd1a1acb18ac51a93c506c37d29531cc958dd, data reload: false

query1	0.05	0.05	0.06
query2	0.09	0.05	0.06
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.26	0.26	0.26
query6	1.16	0.65	0.64
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.59	0.54	0.53
query10	0.58	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.61	0.60	0.61
query14	1.00	1.00	1.00
query15	0.85	0.84	0.85
query16	0.39	0.41	0.42
query17	1.02	1.02	1.03
query18	0.22	0.21	0.20
query19	1.91	1.80	1.84
query20	0.02	0.01	0.01
query21	15.44	0.17	0.13
query22	5.11	0.06	0.05
query23	15.67	0.26	0.10
query24	3.06	0.67	0.40
query25	0.08	0.07	0.05
query26	0.15	0.14	0.14
query27	0.06	0.06	0.06
query28	4.81	1.16	0.93
query29	12.57	3.85	3.30
query30	0.29	0.14	0.12
query31	2.84	0.59	0.38
query32	3.23	0.55	0.47
query33	3.17	3.08	3.06
query34	15.82	5.20	4.55
query35	4.56	4.64	4.56
query36	0.68	0.51	0.50
query37	0.11	0.06	0.06
query38	0.06	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 99.1 s
Total hot run time: 27.62 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 5, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (5/5) 🎉
Increment coverage report
Complete coverage report

@seawinde
Copy link
Contributor Author

seawinde commented Nov 6, 2025

run buildall

1 similar comment
@seawinde
Copy link
Contributor Author

seawinde commented Nov 6, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 6, 2025

TPC-DS: Total hot run time: 189251 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e0a98933a05ff13d07b9e2c6d10eceec71935f34, data reload: false

query1	1050	411	413	411
query2	6581	1708	1719	1708
query3	6755	231	223	223
query4	26457	23726	23491	23491
query5	5768	640	463	463
query6	356	224	213	213
query7	4647	489	296	296
query8	301	247	243	243
query9	8704	2566	2573	2566
query10	535	338	289	289
query11	15719	15066	14838	14838
query12	182	116	110	110
query13	1677	546	425	425
query14	11426	9244	9270	9244
query15	203	195	172	172
query16	7646	645	536	536
query17	1600	760	653	653
query18	2606	442	334	334
query19	234	225	185	185
query20	137	129	133	129
query21	264	132	123	123
query22	4662	4552	4316	4316
query23	34402	33684	33833	33684
query24	8491	2506	2508	2506
query25	582	551	461	461
query26	1207	280	168	168
query27	2737	538	385	385
query28	4496	2235	2229	2229
query29	835	818	557	557
query30	314	226	199	199
query31	925	837	777	777
query32	85	77	72	72
query33	594	397	386	386
query34	839	884	564	564
query35	883	870	818	818
query36	1000	1066	947	947
query37	128	119	94	94
query38	3750	3629	3457	3457
query39	1473	1410	1407	1407
query40	217	128	123	123
query41	65	61	64	61
query42	123	115	118	115
query43	478	485	466	466
query44	1210	750	746	746
query45	190	179	178	178
query46	883	999	640	640
query47	1740	1757	1704	1704
query48	401	423	325	325
query49	749	509	429	429
query50	633	691	396	396
query51	3959	3894	3935	3894
query52	107	105	98	98
query53	246	273	200	200
query54	319	298	280	280
query55	88	97	89	89
query56	331	317	335	317
query57	1183	1161	1119	1119
query58	298	291	281	281
query59	2597	2650	2541	2541
query60	360	355	351	351
query61	191	187	188	187
query62	787	754	657	657
query63	228	197	194	194
query64	3881	1164	855	855
query65	4009	3956	3963	3956
query66	910	420	351	351
query67	15540	15284	14910	14910
query68	8634	925	580	580
query69	523	318	279	279
query70	1323	1242	1288	1242
query71	526	334	308	308
query72	6043	4956	4877	4877
query73	695	585	353	353
query74	8948	9143	8757	8757
query75	4248	3372	2821	2821
query76	3825	1145	728	728
query77	811	411	322	322
query78	9540	9717	8810	8810
query79	2376	856	607	607
query80	644	567	508	508
query81	495	258	233	233
query82	426	163	126	126
query83	301	262	249	249
query84	301	120	104	104
query85	869	468	433	433
query86	335	325	304	304
query87	3748	3729	3618	3618
query88	3394	2261	2237	2237
query89	402	322	301	301
query90	2052	229	226	226
query91	160	168	134	134
query92	83	67	67	67
query93	1559	971	643	643
query94	684	450	335	335
query95	383	325	317	317
query96	486	577	283	283
query97	2945	2962	2849	2849
query98	251	211	208	208
query99	1440	1383	1250	1250
Total cold run time: 280424 ms
Total hot run time: 189251 ms

@doris-robot
Copy link

doris-robot commented Nov 6, 2025

ClickBench: Total hot run time: 27.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e0a98933a05ff13d07b9e2c6d10eceec71935f34, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.61	0.12	0.11
query5	0.28	0.26	0.25
query6	1.16	0.65	0.64
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.60	0.53	0.51
query10	0.59	0.58	0.57
query11	0.17	0.11	0.11
query12	0.15	0.11	0.12
query13	0.61	0.60	0.60
query14	1.01	1.01	1.01
query15	0.85	0.83	0.84
query16	0.39	0.40	0.41
query17	0.99	1.03	1.03
query18	0.25	0.20	0.20
query19	1.91	1.87	1.80
query20	0.02	0.02	0.02
query21	15.43	0.18	0.13
query22	5.05	0.07	0.04
query23	15.66	0.27	0.10
query24	2.98	0.70	0.67
query25	0.07	0.07	0.06
query26	0.14	0.12	0.14
query27	0.06	0.05	0.05
query28	4.62	1.13	0.93
query29	12.60	4.00	3.26
query30	0.28	0.13	0.13
query31	2.82	0.59	0.39
query32	3.23	0.54	0.47
query33	3.10	3.01	3.04
query34	15.90	5.22	4.53
query35	4.60	4.57	4.53
query36	0.68	0.50	0.49
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.82 s
Total hot run time: 27.7 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 6, 2025

FE UT Coverage Report

Increment line coverage 28.57% (4/14) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 6, 2025

FE Regression Coverage Report

Increment line coverage 85.71% (12/14) 🎉
Increment coverage report
Complete coverage report

@seawinde seawinde force-pushed the fix_group_sets_rewrite_fail_when_cte branch from e0a9893 to adcc93e Compare November 7, 2025 02:12
@seawinde
Copy link
Contributor Author

seawinde commented Nov 7, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 7, 2025

TPC-DS: Total hot run time: 189266 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6d68309d134a901eedd2859a0b5626918c8ad3e7, data reload: false

query1	1016	403	397	397
query2	6587	1681	1715	1681
query3	6749	231	217	217
query4	26335	23447	23463	23447
query5	5226	631	465	465
query6	342	236	217	217
query7	4645	491	293	293
query8	303	280	251	251
query9	8743	2584	2566	2566
query10	540	362	290	290
query11	15985	15128	14834	14834
query12	190	118	114	114
query13	1689	571	440	440
query14	11872	9047	9270	9047
query15	199	187	182	182
query16	7675	686	538	538
query17	1618	799	630	630
query18	2050	487	378	378
query19	228	237	200	200
query20	139	132	145	132
query21	234	157	122	122
query22	4721	4815	4609	4609
query23	34912	33648	34039	33648
query24	8507	2482	2464	2464
query25	593	543	504	504
query26	1263	288	159	159
query27	2836	533	370	370
query28	4346	2220	2189	2189
query29	838	628	507	507
query30	307	236	207	207
query31	966	860	755	755
query32	81	83	68	68
query33	594	401	341	341
query34	794	879	532	532
query35	824	844	751	751
query36	1001	1028	974	974
query37	121	110	88	88
query38	3652	3709	3654	3654
query39	1538	1428	1403	1403
query40	216	128	118	118
query41	59	57	57	57
query42	128	113	110	110
query43	489	489	474	474
query44	1214	743	736	736
query45	178	175	192	175
query46	872	981	633	633
query47	1768	1797	1757	1757
query48	383	419	312	312
query49	771	505	420	420
query50	651	679	402	402
query51	4136	4006	3882	3882
query52	108	105	101	101
query53	250	279	186	186
query54	306	278	268	268
query55	84	83	84	83
query56	319	310	289	289
query57	1165	1204	1129	1129
query58	276	267	263	263
query59	2567	2752	2489	2489
query60	327	331	344	331
query61	158	156	155	155
query62	763	722	678	678
query63	236	193	193	193
query64	4460	1130	831	831
query65	4038	3918	3935	3918
query66	1052	420	331	331
query67	15442	15334	14873	14873
query68	7197	880	598	598
query69	484	311	284	284
query70	1362	1288	1253	1253
query71	470	356	297	297
query72	6143	4998	5034	4998
query73	553	620	360	360
query74	9049	9074	8878	8878
query75	3958	3315	2798	2798
query76	3301	1139	728	728
query77	791	390	305	305
query78	9487	9583	8914	8914
query79	2184	820	585	585
query80	620	571	490	490
query81	500	254	226	226
query82	471	160	132	132
query83	257	259	250	250
query84	259	116	90	90
query85	897	485	432	432
query86	376	301	274	274
query87	3707	3645	3631	3631
query88	4088	2266	2257	2257
query89	372	330	294	294
query90	1961	216	227	216
query91	166	174	138	138
query92	81	62	63	62
query93	1871	957	639	639
query94	701	441	334	334
query95	393	318	311	311
query96	480	592	280	280
query97	2897	2969	2842	2842
query98	232	208	205	205
query99	1625	1394	1307	1307
Total cold run time: 279608 ms
Total hot run time: 189266 ms

@doris-robot
Copy link

doris-robot commented Nov 7, 2025

ClickBench: Total hot run time: 27.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6d68309d134a901eedd2859a0b5626918c8ad3e7, data reload: false

query1	0.06	0.05	0.06
query2	0.09	0.04	0.04
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.27	0.26	0.25
query6	1.20	0.65	0.64
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.59	0.51	0.52
query10	0.58	0.58	0.57
query11	0.17	0.12	0.11
query12	0.15	0.11	0.12
query13	0.61	0.60	0.60
query14	0.99	1.01	1.00
query15	0.84	0.84	0.84
query16	0.39	0.38	0.41
query17	1.03	1.01	1.02
query18	0.21	0.20	0.19
query19	1.87	1.77	1.77
query20	0.01	0.02	0.02
query21	15.44	0.19	0.14
query22	5.01	0.06	0.04
query23	15.69	0.26	0.10
query24	3.36	0.55	0.42
query25	0.08	0.06	0.06
query26	0.14	0.12	0.13
query27	0.07	0.06	0.05
query28	4.58	1.12	0.93
query29	12.55	3.91	3.22
query30	0.28	0.14	0.12
query31	2.82	0.61	0.38
query32	3.24	0.55	0.47
query33	3.03	2.98	3.10
query34	15.80	5.19	4.52
query35	4.57	4.60	4.60
query36	0.66	0.49	0.50
query37	0.09	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.15	0.14
query41	0.09	0.04	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.86 s
Total hot run time: 27.36 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 7, 2025

FE Regression Coverage Report

Increment line coverage 67.31% (35/52) 🎉
Increment coverage report
Complete coverage report

@seawinde
Copy link
Contributor Author

seawinde commented Nov 7, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 7, 2025

TPC-DS: Total hot run time: 189104 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e0b271fff88a83c5575bbb0a5b9c4c50122e2ddc, data reload: false

query1	1325	403	397	397
query2	6649	1657	1646	1646
query3	6851	231	218	218
query4	25954	23403	23132	23132
query5	5028	654	469	469
query6	347	245	228	228
query7	4675	488	294	294
query8	308	265	257	257
query9	8746	2567	2647	2567
query10	519	353	303	303
query11	16534	16097	15721	15721
query12	195	123	128	123
query13	2055	571	470	470
query14	12922	9552	9429	9429
query15	241	192	175	175
query16	8693	687	567	567
query17	1622	753	640	640
query18	2096	422	325	325
query19	268	197	194	194
query20	132	126	126	126
query21	570	130	115	115
query22	4636	4836	4610	4610
query23	34587	33915	32980	32980
query24	8011	2371	2377	2371
query25	549	504	438	438
query26	1301	267	152	152
query27	2628	485	339	339
query28	4312	2177	2164	2164
query29	763	597	482	482
query30	323	222	195	195
query31	914	793	765	765
query32	73	67	66	66
query33	569	370	320	320
query34	776	841	509	509
query35	799	820	733	733
query36	943	982	900	900
query37	145	108	82	82
query38	3533	3513	3514	3513
query39	1483	1438	1415	1415
query40	219	126	118	118
query41	60	57	60	57
query42	118	108	106	106
query43	484	492	463	463
query44	1192	755	739	739
query45	185	178	170	170
query46	882	990	637	637
query47	1813	1814	1750	1750
query48	384	420	329	329
query49	781	516	432	432
query50	644	682	401	401
query51	3915	3876	3875	3875
query52	109	108	102	102
query53	233	273	199	199
query54	317	303	290	290
query55	90	84	82	82
query56	322	349	346	346
query57	1186	1207	1131	1131
query58	284	285	280	280
query59	2595	2710	2499	2499
query60	361	363	340	340
query61	193	185	183	183
query62	801	761	664	664
query63	226	194	194	194
query64	4555	1256	933	933
query65	4014	3948	3935	3935
query66	1110	434	328	328
query67	15598	15215	14854	14854
query68	8350	928	594	594
query69	473	318	283	283
query70	1332	1266	1231	1231
query71	491	332	310	310
query72	5839	5043	4800	4800
query73	532	564	357	357
query74	8819	8859	8897	8859
query75	3884	3325	2870	2870
query76	3264	1148	721	721
query77	820	398	309	309
query78	9523	9915	8884	8884
query79	2306	795	586	586
query80	632	542	490	490
query81	512	258	228	228
query82	433	158	124	124
query83	297	260	245	245
query84	296	121	90	90
query85	914	493	434	434
query86	334	323	289	289
query87	3718	3651	3633	3633
query88	3373	2258	2268	2258
query89	377	315	289	289
query90	2033	211	211	211
query91	166	163	134	134
query92	85	70	64	64
query93	1199	971	646	646
query94	702	427	328	328
query95	398	322	308	308
query96	484	583	291	291
query97	2939	2947	2850	2850
query98	244	215	207	207
query99	1436	1409	1314	1314
Total cold run time: 280951 ms
Total hot run time: 189104 ms

@doris-robot
Copy link

doris-robot commented Nov 7, 2025

ClickBench: Total hot run time: 27.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e0b271fff88a83c5575bbb0a5b9c4c50122e2ddc, data reload: false

query1	0.05	0.05	0.05
query2	0.11	0.06	0.05
query3	0.27	0.08	0.08
query4	1.79	0.12	0.11
query5	0.28	0.25	0.26
query6	1.17	0.64	0.64
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.69	0.52	0.52
query10	0.58	0.56	0.57
query11	0.18	0.12	0.12
query12	0.16	0.12	0.12
query13	0.62	0.61	0.59
query14	0.99	1.00	1.00
query15	0.86	0.82	0.82
query16	0.38	0.38	0.38
query17	1.05	1.05	1.01
query18	0.22	0.19	0.19
query19	2.02	1.73	1.84
query20	0.01	0.02	0.01
query21	15.44	0.18	0.14
query22	4.95	0.13	0.04
query23	15.61	0.25	0.10
query24	3.62	1.55	0.51
query25	0.09	0.07	0.07
query26	0.14	0.14	0.13
query27	0.06	0.06	0.05
query28	5.55	1.12	0.93
query29	12.67	3.87	3.23
query30	0.29	0.14	0.11
query31	2.82	0.57	0.38
query32	3.25	0.56	0.47
query33	2.95	3.04	3.02
query34	15.82	5.15	4.51
query35	4.58	4.53	4.53
query36	0.65	0.49	0.50
query37	0.16	0.07	0.07
query38	0.09	0.04	0.03
query39	0.05	0.03	0.03
query40	0.20	0.15	0.14
query41	0.11	0.03	0.03
query42	0.06	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 100.72 s
Total hot run time: 27.36 s

morrySnow
morrySnow previously approved these changes Nov 10, 2025
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 10, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 10, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 10, 2025

PR approved by anyone and no changes requested.

@seawinde seawinde force-pushed the fix_group_sets_rewrite_fail_when_cte branch from e0b271f to 0e86ad0 Compare November 10, 2025 09:10
@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 8, 2025

FE Regression Coverage Report

Increment line coverage 88.89% (24/27) 🎉
Increment coverage report
Complete coverage report

@seawinde seawinde force-pushed the fix_group_sets_rewrite_fail_when_cte branch from 86ffb4e to 716b025 Compare December 9, 2025 08:57
@seawinde
Copy link
Contributor Author

seawinde commented Dec 9, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 9, 2025

FE UT Coverage Report

Increment line coverage 59.26% (16/27) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 9, 2025

FE Regression Coverage Report

Increment line coverage 88.89% (24/27) 🎉
Increment coverage report
Complete coverage report

2 similar comments
@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 9, 2025

FE Regression Coverage Report

Increment line coverage 88.89% (24/27) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 9, 2025

FE Regression Coverage Report

Increment line coverage 88.89% (24/27) 🎉
Increment coverage report
Complete coverage report

@seawinde
Copy link
Contributor Author

seawinde commented Dec 9, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Dec 10, 2025

TPC-DS: Total hot run time: 182857 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 571a11da70eec16f397b9d1429a7dc1380442d8d, data reload: false

query5	6101	655	473	473
query6	329	239	224	224
query7	4229	483	285	285
query8	311	256	244	244
query9	8758	2628	2629	2628
query10	532	401	329	329
query11	15599	15097	14587	14587
query12	195	116	119	116
query13	1299	529	404	404
query14	6695	3309	3014	3014
query14_1	2951	2982	2947	2947
query15	212	196	188	188
query16	1055	469	462	462
query17	1257	726	616	616
query18	2716	450	358	358
query19	235	236	209	209
query20	129	116	115	115
query21	561	143	119	119
query22	4602	4659	4505	4505
query23	16982	16532	16236	16236
query23_1	16648	16606	16307	16307
query24	8358	1767	1252	1252
query24_1	1345	1318	1253	1253
query25	539	473	417	417
query26	1638	271	154	154
query27	2852	491	325	325
query28	5758	2204	2208	2204
query29	853	583	508	508
query30	354	267	233	233
query31	1446	788	649	649
query32	71	69	63	63
query33	513	342	301	301
query34	928	962	549	549
query35	840	844	761	761
query36	920	964	859	859
query37	171	88	80	80
query38	3924	3847	3894	3847
query39	754	740	718	718
query39_1	706	699	700	699
query40	265	132	116	116
query41	66	61	59	59
query42	101	97	98	97
query43	425	429	398	398
query44	1303	752	754	752
query45	197	188	189	188
query46	866	979	613	613
query47	1622	1674	1670	1670
query48	316	332	245	245
query49	765	428	355	355
query50	647	285	217	217
query51	3837	3893	3828	3828
query52	109	107	98	98
query53	322	352	298	298
query54	294	271	249	249
query55	79	72	70	70
query56	305	299	285	285
query57	1157	1140	1075	1075
query58	266	258	237	237
query59	2318	2414	2326	2326
query60	313	311	293	293
query61	161	173	163	163
query62	699	668	662	662
query63	323	289	290	289
query64	4882	1319	1002	1002
query65	4067	3940	3955	3940
query66	1350	436	309	309
query67	15072	14879	14873	14873
query68	8617	1032	743	743
query69	562	348	317	317
query70	1077	1055	993	993
query71	414	315	291	291
query72	5813	4889	4952	4889
query73	658	548	312	312
query74	8806	8993	8655	8655
query75	3865	3515	3165	3165
query76	4222	1162	753	753
query77	834	402	288	288
query78	9418	9778	8838	8838
query79	1724	850	622	622
query80	732	646	565	565
query81	520	263	246	246
query82	197	140	108	108
query83	256	301	244	244
query84	272	125	107	107
query85	904	503	453	453
query86	340	294	298	294
query87	4048	4182	3937	3937
query88	3209	2317	2297	2297
query89	465	414	384	384
query90	2255	155	150	150
query91	170	167	142	142
query92	89	69	60	60
query93	2048	908	557	557
query94	460	304	274	274
query95	547	375	308	308
query96	584	458	210	210
query97	2623	2698	2552	2552
query98	232	191	191	191
query99	1293	1289	1215	1215
Total cold run time: 273290 ms
Total hot run time: 182857 ms

@doris-robot
Copy link

doris-robot commented Dec 10, 2025

ClickBench: Total hot run time: 27.68 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 571a11da70eec16f397b9d1429a7dc1380442d8d, data reload: false

query1	0.06	0.05	0.04
query2	0.16	0.05	0.05
query3	0.27	0.09	0.08
query4	1.62	0.11	0.10
query5	0.27	0.26	0.26
query6	1.19	0.64	0.64
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.58	0.50	0.49
query10	0.56	0.55	0.56
query11	0.19	0.10	0.10
query12	0.14	0.12	0.12
query13	0.63	0.61	0.60
query14	1.00	0.98	0.98
query15	0.81	0.80	0.80
query16	0.42	0.40	0.41
query17	1.06	1.08	1.05
query18	0.24	0.22	0.22
query19	1.95	1.78	1.77
query20	0.02	0.02	0.01
query21	15.50	0.30	0.14
query22	4.73	0.05	0.05
query23	16.10	0.27	0.10
query24	2.09	0.78	0.52
query25	0.08	0.07	0.07
query26	0.15	0.14	0.13
query27	0.08	0.05	0.06
query28	5.37	1.22	1.04
query29	12.64	4.01	3.26
query30	0.28	0.14	0.11
query31	2.82	0.62	0.39
query32	3.24	0.57	0.45
query33	3.02	3.12	3.09
query34	16.76	5.16	4.53
query35	4.48	4.57	4.58
query36	0.66	0.50	0.50
query37	0.13	0.07	0.06
query38	0.08	0.04	0.04
query39	0.06	0.03	0.04
query40	0.17	0.14	0.13
query41	0.15	0.03	0.03
query42	0.06	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 99.95 s
Total hot run time: 27.68 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 10, 2025

FE UT Coverage Report

Increment line coverage 50.00% (10/20) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 10, 2025

FE Regression Coverage Report

Increment line coverage 85.00% (17/20) 🎉
Increment coverage report
Complete coverage report

3 similar comments
@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 10, 2025

FE Regression Coverage Report

Increment line coverage 85.00% (17/20) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 10, 2025

FE Regression Coverage Report

Increment line coverage 85.00% (17/20) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 11, 2025

FE Regression Coverage Report

Increment line coverage 85.00% (17/20) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 12, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 12, 2025

PR approved by at least one committer and no changes requested.

@starocean999 starocean999 merged commit 580ed5a into apache:master Dec 12, 2025
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 12, 2025
@seawinde
[fix](mtmv) Fix query rewrite by mv fail when both use cte and group …
f02c754 
…sets (#57741)

Fix query rewrite by mv fail when both use cte and group sets

Such as mv def is as following:
```sql
            select l_shipdate, o_orderdate, l_partkey, l_suppkey,
            sum(o_totalprice) as sum_total,
            max(o_totalprice) as max_total,
            min(o_totalprice) as min_total,
            count(*) as count_all,
            bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) as bitmap_union_basic
            from lineitem
            left join orders on lineitem.l_orderkey = orders.o_orderkey and l_shipdate = o_orderdate
            group by
            l_shipdate,
            o_orderdate,
            l_partkey,
            l_suppkey;
```

query is as fllowing, use both group sets and cte, would rewrite fail,
the pr fix this

```sql
            with t as (
                select t1.l_partkey, t1.l_suppkey, o_orderdate,
                grouping(t1.l_suppkey),
                grouping(o_orderdate),
                grouping_id(t1.l_partkey, t1.l_suppkey),
                grouping_id(t1.l_partkey, t1.l_suppkey, o_orderdate),
                sum(o_totalprice),
                max(o_totalprice),
                min(o_totalprice),
                count(*),
                count(distinct case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)
                from (select * from lineitem where l_shipdate = '2023-12-11') t1
                left join orders on t1.l_orderkey = orders.o_orderkey and t1.l_shipdate = o_orderdate
                group by
                GROUPING SETS ((l_shipdate, o_orderdate, l_partkey), (l_partkey, l_suppkey), (l_suppkey), ())
            )
            select t1.l_suppkey, t2.o_orderdate
            from
            t t1
            inner join
            t t2 on t1.l_partkey = t2.l_partkey;
```
@github-actions github-actions bot mentioned this pull request Dec 12, 2025
Merged
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@seawinde @nagisa-kunhah
[fix](mtmv) Fix query rewrite by mv fail when both use cte and group …
6d79478 
…sets (apache#57741)

Fix query rewrite by mv fail when both use cte and group sets

Such as mv def is as following:
```sql
            select l_shipdate, o_orderdate, l_partkey, l_suppkey,
            sum(o_totalprice) as sum_total,
            max(o_totalprice) as max_total,
            min(o_totalprice) as min_total,
            count(*) as count_all,
            bitmap_union(to_bitmap(case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)) as bitmap_union_basic
            from lineitem
            left join orders on lineitem.l_orderkey = orders.o_orderkey and l_shipdate = o_orderdate
            group by
            l_shipdate,
            o_orderdate,
            l_partkey,
            l_suppkey;
```

query is as fllowing, use both group sets and cte, would rewrite fail,
the pr fix this

```sql
            with t as (
                select t1.l_partkey, t1.l_suppkey, o_orderdate,
                grouping(t1.l_suppkey),
                grouping(o_orderdate),
                grouping_id(t1.l_partkey, t1.l_suppkey),
                grouping_id(t1.l_partkey, t1.l_suppkey, o_orderdate),
                sum(o_totalprice),
                max(o_totalprice),
                min(o_totalprice),
                count(*),
                count(distinct case when o_shippriority > 1 and o_orderkey IN (1, 3) then o_custkey else null end)
                from (select * from lineitem where l_shipdate = '2023-12-11') t1
                left join orders on t1.l_orderkey = orders.o_orderkey and t1.l_shipdate = o_orderdate
                group by
                GROUPING SETS ((l_shipdate, o_orderdate, l_partkey), (l_partkey, l_suppkey), (l_suppkey), ())
            )
            select t1.l_suppkey, t2.o_orderdate
            from
            t t1
            inner join
            t t2 on t1.l_partkey = t2.l_partkey;
```
yiguolei pushed a commit that referenced this pull request Dec 17, 2025
@github-actions @seawinde
branch-4.0: [fix](mtmv) Fix query rewrite by mv fail when both use ct…
10666f2 
…e and group sets #57741 (#58994)

Cherry-picked from #57741

Co-authored-by: seawinde <wusi@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@seawinde @hello-stephen @doris-robot @starocean999 @morrySnow @yiguolei