[enhance](mtmv)insert overwrite of mtmv force drop partition #48074

zddr · 2025-02-19T07:17:03Z

What problem does this PR solve?

insert overwrite of mtmv force drop partition

not need pick to 2.1,because #36361 not pick to 2.1

case pr: #48946

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
insert overwrite of mtmv force drop partition

Release note

insert overwrite of mtmv force drop partition

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-02-19T07:17:20Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zddr · 2025-02-19T07:17:32Z

run buildall

doris-robot · 2025-02-19T07:55:39Z

TPC-H: Total hot run time: 31360 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 75c3d79fa4030c18ac646e6533c41a95aaec6bdf, data reload: false

------ Round 1 ----------------------------------
q1	17617	5383	5038	5038
q2	2055	298	171	171
q3	10452	1255	721	721
q4	10292	1008	549	549
q5	8167	2454	2286	2286
q6	192	170	131	131
q7	898	772	617	617
q8	9305	1319	1181	1181
q9	4868	4625	4643	4625
q10	6834	2316	1856	1856
q11	506	280	258	258
q12	356	361	217	217
q13	17755	3705	3080	3080
q14	225	231	207	207
q15	521	473	459	459
q16	611	608	580	580
q17	576	850	328	328
q18	6956	6108	6150	6108
q19	1207	950	572	572
q20	312	318	183	183
q21	2850	2273	1898	1898
q22	358	326	295	295
Total cold run time: 102913 ms
Total hot run time: 31360 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5413	5117	5114	5114
q2	227	325	224	224
q3	2155	2663	2290	2290
q4	1475	1801	1350	1350
q5	4188	4179	4158	4158
q6	209	161	124	124
q7	1872	1812	1745	1745
q8	2607	2538	2629	2538
q9	7232	7255	7196	7196
q10	2993	3221	2804	2804
q11	566	513	507	507
q12	683	768	636	636
q13	3385	3951	3254	3254
q14	274	294	282	282
q15	499	476	472	472
q16	624	696	655	655
q17	1120	1598	1380	1380
q18	7601	7312	7384	7312
q19	788	855	955	855
q20	1939	2018	1860	1860
q21	5527	4990	4800	4800
q22	614	566	544	544
Total cold run time: 51991 ms
Total hot run time: 50100 ms

doris-robot · 2025-02-19T08:07:46Z

TPC-DS: Total hot run time: 190578 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 75c3d79fa4030c18ac646e6533c41a95aaec6bdf, data reload: false

query1	1319	967	943	943
query2	6241	1827	1825	1825
query3	10986	4476	4589	4476
query4	56893	24889	23593	23593
query5	5079	540	495	495
query6	392	215	186	186
query7	5282	526	299	299
query8	328	255	238	238
query9	6748	2535	2541	2535
query10	420	309	274	274
query11	15238	15086	14871	14871
query12	157	112	108	108
query13	1202	524	391	391
query14	10691	6777	6425	6425
query15	205	207	183	183
query16	7063	648	496	496
query17	1077	752	571	571
query18	1574	423	324	324
query19	212	231	167	167
query20	133	127	116	116
query21	215	142	113	113
query22	4361	4588	4246	4246
query23	34066	33261	33407	33261
query24	5761	2460	2449	2449
query25	454	457	406	406
query26	676	281	152	152
query27	1781	478	337	337
query28	2856	2501	2425	2425
query29	572	545	432	432
query30	220	201	159	159
query31	894	866	824	824
query32	76	67	63	63
query33	453	349	329	329
query34	752	858	553	553
query35	797	850	775	775
query36	982	1007	906	906
query37	120	94	70	70
query38	4326	4336	4248	4248
query39	1498	1464	1464	1464
query40	204	122	100	100
query41	51	49	53	49
query42	133	110	103	103
query43	518	512	490	490
query44	1307	818	808	808
query45	182	179	164	164
query46	878	1066	656	656
query47	1841	1840	1749	1749
query48	397	431	330	330
query49	703	520	458	458
query50	717	764	433	433
query51	4277	4325	4279	4279
query52	110	114	96	96
query53	233	284	189	189
query54	487	510	427	427
query55	84	90	82	82
query56	280	258	255	255
query57	1173	1174	1133	1133
query58	250	236	249	236
query59	2710	2790	2605	2605
query60	291	271	258	258
query61	127	115	116	115
query62	758	756	679	679
query63	238	206	202	202
query64	1494	993	672	672
query65	3195	3132	3156	3132
query66	683	387	294	294
query67	15874	15507	15535	15507
query68	5203	784	545	545
query69	506	297	262	262
query70	1249	1138	1111	1111
query71	451	309	258	258
query72	6283	3706	3659	3659
query73	921	742	357	357
query74	9120	9092	8871	8871
query75	3530	3135	2722	2722
query76	3761	1302	725	725
query77	516	382	298	298
query78	9921	10003	9286	9286
query79	2648	824	635	635
query80	712	518	510	510
query81	517	278	239	239
query82	594	124	99	99
query83	177	169	149	149
query84	296	93	79	79
query85	774	351	314	314
query86	425	294	293	293
query87	4382	4514	4493	4493
query88	3768	2276	2214	2214
query89	419	326	287	287
query90	1892	193	194	193
query91	128	135	108	108
query92	79	64	57	57
query93	2222	1013	590	590
query94	669	422	273	273
query95	345	277	265	265
query96	520	577	276	276
query97	2822	2858	2738	2738
query98	233	202	194	194
query99	1666	1393	1249	1249
Total cold run time: 298044 ms
Total hot run time: 190578 ms

.../src/main/java/org/apache/doris/nereids/trees/plans/commands/UpdateMvByPartitionCommand.java

doris-robot · 2025-02-19T08:13:05Z

ClickBench: Total hot run time: 30.18 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 75c3d79fa4030c18ac646e6533c41a95aaec6bdf, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.10	0.10
query5	0.43	0.42	0.39
query6	1.17	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.51
query10	0.58	0.58	0.57
query11	0.16	0.11	0.11
query12	0.14	0.12	0.11
query13	0.62	0.59	0.59
query14	2.69	2.71	2.75
query15	0.90	0.84	0.85
query16	0.38	0.38	0.38
query17	1.04	0.99	1.05
query18	0.22	0.19	0.20
query19	1.90	1.75	2.02
query20	0.01	0.01	0.01
query21	15.35	0.86	0.55
query22	0.77	1.12	0.69
query23	15.00	1.38	0.60
query24	9.59	2.73	0.51
query25	0.31	0.21	0.11
query26	0.98	0.19	0.15
query27	0.05	0.04	0.05
query28	6.39	0.81	0.44
query29	12.58	3.87	3.28
query30	0.26	0.09	0.06
query31	2.83	0.58	0.38
query32	3.24	0.58	0.47
query33	3.16	3.08	3.05
query34	15.78	5.12	4.52
query35	4.50	4.47	4.53
query36	0.67	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.19	0.15	0.13
query41	0.07	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 104.83 s
Total hot run time: 30.18 s

zddr · 2025-02-19T11:15:45Z

run buildall

github-actions · 2025-02-19T11:20:54Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-19T11:20:56Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-19T11:34:33Z

TPC-H: Total hot run time: 31367 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

------ Round 1 ----------------------------------
q1	17637	5350	5105	5105
q2	2040	291	166	166
q3	10427	1296	712	712
q4	10245	1011	578	578
q5	7541	2523	2294	2294
q6	186	167	135	135
q7	895	759	604	604
q8	9306	1297	1091	1091
q9	4863	4735	4515	4515
q10	6834	2317	1876	1876
q11	487	278	266	266
q12	351	352	215	215
q13	17758	3665	3058	3058
q14	221	219	206	206
q15	496	461	451	451
q16	624	614	576	576
q17	588	859	348	348
q18	6438	6235	6301	6235
q19	1075	937	544	544
q20	314	322	189	189
q21	2726	2093	1906	1906
q22	359	339	297	297
Total cold run time: 101411 ms
Total hot run time: 31367 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5202	5132	5175	5132
q2	235	330	234	234
q3	2181	2655	2264	2264
q4	1411	1820	1375	1375
q5	4203	4168	4136	4136
q6	206	161	123	123
q7	1856	1805	1673	1673
q8	2579	2639	2559	2559
q9	7350	7199	7167	7167
q10	3005	3205	2757	2757
q11	641	525	482	482
q12	672	740	605	605
q13	3540	3906	3256	3256
q14	270	297	263	263
q15	502	458	469	458
q16	649	700	646	646
q17	1119	1637	1289	1289
q18	7646	7277	7343	7277
q19	772	811	857	811
q20	2007	1988	1837	1837
q21	5370	4845	4808	4808
q22	589	551	533	533
Total cold run time: 52005 ms
Total hot run time: 49685 ms

doris-robot · 2025-02-19T11:45:50Z

TPC-DS: Total hot run time: 183615 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

query1	999	393	364	364
query2	6525	1831	1779	1779
query3	6797	218	217	217
query4	25966	23656	23436	23436
query5	4323	643	471	471
query6	286	195	180	180
query7	4609	499	308	308
query8	290	279	244	244
query9	8626	2542	2534	2534
query10	456	304	266	266
query11	15288	15116	15016	15016
query12	163	108	105	105
query13	1666	539	398	398
query14	9622	6514	6539	6514
query15	217	197	173	173
query16	7713	636	489	489
query17	1211	750	567	567
query18	2000	433	304	304
query19	204	185	169	169
query20	129	113	119	113
query21	213	123	106	106
query22	4217	4257	4050	4050
query23	33910	32923	32778	32778
query24	7689	2360	2375	2360
query25	536	442	379	379
query26	1225	280	156	156
query27	2084	498	339	339
query28	3871	2412	2390	2390
query29	716	547	436	436
query30	233	207	160	160
query31	943	857	787	787
query32	68	60	65	60
query33	576	350	299	299
query34	753	882	504	504
query35	806	804	725	725
query36	940	982	908	908
query37	121	96	74	74
query38	4130	4189	4137	4137
query39	1464	1403	1608	1403
query40	199	117	102	102
query41	55	50	51	50
query42	126	110	104	104
query43	482	496	471	471
query44	1267	780	780	780
query45	173	170	159	159
query46	868	1034	644	644
query47	1752	1780	1699	1699
query48	394	414	304	304
query49	766	523	418	418
query50	688	745	407	407
query51	4157	4224	4103	4103
query52	110	103	98	98
query53	233	249	186	186
query54	477	501	418	418
query55	82	102	76	76
query56	263	278	252	252
query57	1131	1139	1063	1063
query58	248	247	243	243
query59	2518	2750	2494	2494
query60	288	280	268	268
query61	120	122	124	122
query62	775	718	660	660
query63	234	190	196	190
query64	4228	1045	653	653
query65	3208	3139	3136	3136
query66	1085	410	308	308
query67	15721	15531	15312	15312
query68	5603	791	522	522
query69	467	296	261	261
query70	1221	1137	1111	1111
query71	399	290	259	259
query72	5200	3570	3770	3570
query73	729	735	356	356
query74	9210	9061	8898	8898
query75	3143	3166	2683	2683
query76	3242	1166	726	726
query77	492	357	291	291
query78	9929	10084	9306	9306
query79	1895	837	621	621
query80	641	527	471	471
query81	507	274	245	245
query82	194	126	96	96
query83	166	248	159	159
query84	242	97	75	75
query85	752	350	301	301
query86	373	298	282	282
query87	4472	4465	4432	4432
query88	2904	2227	2209	2209
query89	386	320	282	282
query90	1841	200	190	190
query91	129	139	111	111
query92	63	62	63	62
query93	1233	1005	580	580
query94	618	406	344	344
query95	348	270	257	257
query96	496	523	284	284
query97	2763	2858	2750	2750
query98	221	203	195	195
query99	1286	1415	1268	1268
Total cold run time: 264398 ms
Total hot run time: 183615 ms

doris-robot · 2025-02-19T11:51:09Z

ClickBench: Total hot run time: 29.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.65	0.11	0.11
query5	0.41	0.41	0.38
query6	1.18	0.64	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.60	0.50	0.52
query10	0.58	0.58	0.56
query11	0.15	0.10	0.10
query12	0.15	0.11	0.11
query13	0.63	0.60	0.61
query14	2.77	2.71	2.68
query15	0.92	0.84	0.84
query16	0.37	0.38	0.38
query17	1.05	1.04	1.04
query18	0.22	0.20	0.19
query19	1.88	1.79	2.00
query20	0.01	0.01	0.01
query21	15.36	0.89	0.54
query22	0.75	1.19	0.62
query23	15.01	1.40	0.65
query24	10.88	0.97	0.38
query25	0.31	0.10	0.10
query26	0.84	0.18	0.15
query27	0.05	0.05	0.04
query28	6.43	0.88	0.42
query29	12.55	3.86	3.23
query30	0.26	0.09	0.06
query31	2.83	0.57	0.38
query32	3.22	0.54	0.46
query33	2.96	3.01	3.09
query34	15.77	5.16	4.53
query35	4.59	4.53	4.57
query36	0.67	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.96 s
Total hot run time: 29.97 s

zddr · 2025-02-20T02:57:38Z

run buildall

doris-robot · 2025-02-20T03:32:25Z

TPC-H: Total hot run time: 31520 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

------ Round 1 ----------------------------------
q1	17596	5291	5092	5092
q2	2045	315	174	174
q3	10387	1241	757	757
q4	10221	1020	571	571
q5	7529	2460	2277	2277
q6	198	166	140	140
q7	919	772	602	602
q8	9316	1322	1217	1217
q9	4884	4569	4691	4569
q10	6817	2330	1870	1870
q11	481	290	256	256
q12	354	357	227	227
q13	17772	3656	3060	3060
q14	231	234	219	219
q15	497	462	457	457
q16	622	605	582	582
q17	593	855	358	358
q18	6478	6210	6065	6065
q19	1212	971	538	538
q20	307	341	198	198
q21	3029	2173	1990	1990
q22	370	329	301	301
Total cold run time: 101858 ms
Total hot run time: 31520 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5208	5155	5184	5155
q2	237	335	234	234
q3	2167	2674	2298	2298
q4	1444	1851	1357	1357
q5	4230	4191	4206	4191
q6	212	162	124	124
q7	1892	1837	1711	1711
q8	2654	2616	2608	2608
q9	7282	7317	7233	7233
q10	3124	3249	2798	2798
q11	613	533	494	494
q12	738	789	632	632
q13	3684	4046	3366	3366
q14	289	304	265	265
q15	521	447	480	447
q16	650	690	664	664
q17	1151	1551	1410	1410
q18	7564	7482	7305	7305
q19	870	868	1000	868
q20	1974	2050	1881	1881
q21	5656	5150	4962	4962
q22	642	615	568	568
Total cold run time: 52802 ms
Total hot run time: 50571 ms

doris-robot · 2025-02-20T03:44:17Z

TPC-DS: Total hot run time: 190521 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

query1	3185	1000	986	986
query2	7925	1838	1868	1838
query3	12383	4629	4804	4629
query4	25631	23946	23323	23323
query5	5118	650	488	488
query6	306	195	202	195
query7	4006	521	320	320
query8	289	247	261	247
query9	8532	2553	2549	2549
query10	476	324	258	258
query11	15433	15231	14895	14895
query12	178	109	109	109
query13	1628	549	391	391
query14	9872	6406	6406	6406
query15	202	196	172	172
query16	7685	677	536	536
query17	1280	770	588	588
query18	2067	428	344	344
query19	216	210	189	189
query20	140	125	121	121
query21	212	131	126	126
query22	4498	4400	4444	4400
query23	34130	33748	33268	33268
query24	7555	2487	2468	2468
query25	517	480	400	400
query26	1218	279	161	161
query27	2597	508	356	356
query28	4800	2467	2471	2467
query29	694	587	463	463
query30	220	191	151	151
query31	940	878	826	826
query32	76	72	60	60
query33	549	370	320	320
query34	800	875	519	519
query35	810	884	779	779
query36	991	1008	920	920
query37	116	99	70	70
query38	4285	4311	4353	4311
query39	1581	1430	1446	1430
query40	214	129	116	116
query41	53	54	50	50
query42	135	104	108	104
query43	488	512	496	496
query44	1392	828	841	828
query45	182	172	163	163
query46	896	1067	690	690
query47	1794	1852	1787	1787
query48	393	464	321	321
query49	772	515	422	422
query50	713	733	452	452
query51	4272	4374	4218	4218
query52	111	105	97	97
query53	238	260	197	197
query54	503	524	435	435
query55	80	89	83	83
query56	282	300	270	270
query57	1156	1151	1099	1099
query58	278	258	246	246
query59	2766	2923	2609	2609
query60	304	302	283	283
query61	122	116	119	116
query62	777	736	683	683
query63	239	205	201	201
query64	4147	1062	758	758
query65	3270	3128	3193	3128
query66	911	390	299	299
query67	15745	15450	15115	15115
query68	8046	806	521	521
query69	476	301	277	277
query70	1180	1057	1115	1057
query71	422	315	281	281
query72	5716	3585	3761	3585
query73	737	748	363	363
query74	9147	9140	8893	8893
query75	3182	3168	2679	2679
query76	3159	1181	808	808
query77	471	408	283	283
query78	10062	10077	9308	9308
query79	2100	832	621	621
query80	1135	528	454	454
query81	536	284	238	238
query82	441	133	95	95
query83	185	169	156	156
query84	246	93	73	73
query85	748	370	312	312
query86	410	301	293	293
query87	4521	4537	4350	4350
query88	2981	2241	2274	2241
query89	402	327	284	284
query90	1830	191	193	191
query91	150	142	110	110
query92	70	61	57	57
query93	1764	1000	586	586
query94	706	422	303	303
query95	352	347	260	260
query96	497	534	281	281
query97	2768	2872	2731	2731
query98	231	208	207	207
query99	1314	1393	1278	1278
Total cold run time: 281359 ms
Total hot run time: 190521 ms

doris-robot · 2025-02-20T03:49:36Z

ClickBench: Total hot run time: 31.5 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4bfb2c44633016506f9db30506922a48629ab956, data reload: false

query1	0.05	0.03	0.03
query2	0.07	0.04	0.03
query3	0.24	0.07	0.06
query4	1.61	0.10	0.10
query5	0.42	0.41	0.41
query6	1.18	0.66	0.67
query7	0.02	0.01	0.02
query8	0.03	0.03	0.03
query9	0.58	0.52	0.51
query10	0.58	0.57	0.57
query11	0.16	0.11	0.10
query12	0.15	0.11	0.11
query13	0.62	0.59	0.60
query14	2.69	2.74	2.72
query15	0.91	0.87	0.85
query16	0.38	0.37	0.37
query17	1.00	1.03	1.04
query18	0.21	0.20	0.19
query19	1.86	1.86	1.96
query20	0.01	0.01	0.02
query21	15.35	0.87	0.55
query22	0.75	1.19	0.63
query23	15.02	1.36	0.60
query24	8.79	1.81	2.16
query25	0.41	0.22	0.08
query26	0.79	0.17	0.13
query27	0.06	0.05	0.06
query28	7.33	0.77	0.44
query29	12.62	3.93	3.24
query30	0.25	0.09	0.06
query31	2.81	0.59	0.39
query32	3.24	0.52	0.47
query33	2.99	2.96	2.99
query34	15.89	5.09	4.53
query35	4.52	4.51	4.53
query36	0.68	0.51	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.04	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 104.76 s
Total hot run time: 31.5 s

zddr · 2025-02-20T04:00:09Z

run cloud_p0

zddr · 2025-02-20T06:14:08Z

run cloud_p0

zddr · 2025-02-20T06:14:12Z

run p0

zddr · 2025-02-20T10:13:20Z

run cloud_p0

zddr · 2025-02-20T10:13:26Z

run p0

zddr · 2025-02-26T03:02:51Z

run buildall

doris-robot · 2025-02-26T03:37:25Z

TPC-H: Total hot run time: 32244 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da6d0b7fe89cfb39cc6b44a810106640b274fdd4, data reload: false

------ Round 1 ----------------------------------
q1	18801	5242	5101	5101
q2	2067	308	166	166
q3	11455	1245	781	781
q4	10215	1034	557	557
q5	7588	2415	2403	2403
q6	189	169	138	138
q7	936	783	610	610
q8	9401	1309	1160	1160
q9	5228	5002	4897	4897
q10	7908	2308	1924	1924
q11	469	281	266	266
q12	349	352	225	225
q13	20237	3764	3165	3165
q14	226	230	216	216
q15	531	480	466	466
q16	648	633	598	598
q17	577	863	350	350
q18	6451	6333	6127	6127
q19	2169	949	571	571
q20	311	315	185	185
q21	2917	2188	2038	2038
q22	370	334	300	300
Total cold run time: 109043 ms
Total hot run time: 32244 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5245	5104	5117	5104
q2	243	327	232	232
q3	2137	2711	2242	2242
q4	1433	1829	1366	1366
q5	4219	4112	4296	4112
q6	219	166	126	126
q7	1997	1890	1791	1791
q8	2600	2768	2608	2608
q9	7207	7098	7234	7098
q10	3066	3237	2729	2729
q11	576	513	488	488
q12	656	732	591	591
q13	3622	3979	3267	3267
q14	279	296	285	285
q15	500	462	461	461
q16	642	661	653	653
q17	1175	1622	1313	1313
q18	7520	7463	7376	7376
q19	842	833	895	833
q20	1974	2011	1870	1870
q21	5623	4907	4790	4790
q22	631	606	543	543
Total cold run time: 52406 ms
Total hot run time: 49878 ms

doris-robot · 2025-02-26T03:49:40Z

TPC-DS: Total hot run time: 191253 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da6d0b7fe89cfb39cc6b44a810106640b274fdd4, data reload: false

query1	1296	966	955	955
query2	6129	1872	1858	1858
query3	11006	4597	4495	4495
query4	57200	25837	23302	23302
query5	5103	535	503	503
query6	370	193	185	185
query7	5016	501	299	299
query8	340	245	250	245
query9	7508	2692	2685	2685
query10	441	346	282	282
query11	15580	15196	14810	14810
query12	165	107	108	107
query13	1159	543	417	417
query14	10389	7009	6692	6692
query15	214	208	191	191
query16	7121	638	469	469
query17	1067	719	586	586
query18	1521	406	308	308
query19	194	198	183	183
query20	131	123	131	123
query21	211	135	115	115
query22	4405	4343	4273	4273
query23	34042	33362	33154	33154
query24	6240	2471	2551	2471
query25	509	499	428	428
query26	699	278	165	165
query27	1772	505	345	345
query28	2848	2475	2492	2475
query29	623	598	457	457
query30	224	200	164	164
query31	881	925	847	847
query32	76	67	65	65
query33	460	378	369	369
query34	775	869	512	512
query35	806	851	771	771
query36	968	1001	909	909
query37	126	100	76	76
query38	4081	4503	4298	4298
query39	1539	1448	1568	1448
query40	207	119	105	105
query41	52	50	53	50
query42	126	104	109	104
query43	499	555	485	485
query44	1373	843	817	817
query45	182	177	168	168
query46	895	1075	682	682
query47	1813	1842	1749	1749
query48	394	419	320	320
query49	702	520	428	428
query50	754	784	460	460
query51	4224	4394	4229	4229
query52	107	110	102	102
query53	247	270	199	199
query54	486	494	428	428
query55	89	86	83	83
query56	272	273	275	273
query57	1160	1158	1110	1110
query58	249	245	244	244
query59	2707	2814	2615	2615
query60	289	278	266	266
query61	122	129	127	127
query62	744	761	678	678
query63	239	200	230	200
query64	1533	1039	688	688
query65	3327	3253	3257	3253
query66	724	387	306	306
query67	15804	15357	15228	15228
query68	7964	882	507	507
query69	559	295	263	263
query70	1183	1135	1129	1129
query71	487	300	281	281
query72	5862	3707	3894	3707
query73	1393	768	353	353
query74	9046	8985	8974	8974
query75	3708	3285	2697	2697
query76	4151	1187	771	771
query77	613	382	294	294
query78	10058	10250	9205	9205
query79	2513	822	595	595
query80	686	525	466	466
query81	516	292	244	244
query82	560	127	95	95
query83	178	167	155	155
query84	288	95	79	79
query85	778	384	304	304
query86	420	298	261	261
query87	4486	4504	4469	4469
query88	3519	2221	2193	2193
query89	410	308	281	281
query90	1784	193	198	193
query91	135	136	113	113
query92	77	61	57	57
query93	2012	1047	582	582
query94	643	397	286	286
query95	346	264	253	253
query96	478	573	268	268
query97	3340	3345	3300	3300
query98	215	205	200	200
query99	1414	1386	1244	1244
Total cold run time: 302377 ms
Total hot run time: 191253 ms

doris-robot · 2025-02-26T03:54:59Z

ClickBench: Total hot run time: 30.96 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da6d0b7fe89cfb39cc6b44a810106640b274fdd4, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.04	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.56	0.54	0.56
query6	1.17	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.60	0.53	0.52
query10	0.58	0.58	0.58
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.66	2.73	2.81
query15	0.93	0.85	0.85
query16	0.38	0.39	0.37
query17	1.06	1.00	1.02
query18	0.22	0.20	0.20
query19	1.92	1.90	1.97
query20	0.01	0.01	0.02
query21	15.37	0.92	0.54
query22	0.76	1.05	0.62
query23	15.14	1.36	0.66
query24	7.19	0.96	1.39
query25	0.55	0.18	0.12
query26	0.56	0.16	0.14
query27	0.06	0.05	0.04
query28	10.82	0.85	0.43
query29	12.53	3.95	3.25
query30	0.25	0.08	0.07
query31	2.81	0.59	0.38
query32	3.22	0.56	0.47
query33	3.00	3.02	3.06
query34	15.70	5.09	4.47
query35	4.55	4.50	4.47
query36	0.66	0.50	0.50
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.66 s
Total hot run time: 30.96 s

zddr · 2025-02-26T07:53:35Z

run external

insert overwrite of mtmv force drop partition, because recyclebin process partition lead to performance issue.

…ion #48074 (#48627) Cherry-picked from #48074 Co-authored-by: zhangdong <zhangdong@selectdb.com>

#48946) ### What problem does this PR solve? Related PR: #48074 Problem Summary: add case for insert overwrite of mtmv force drop partition

…#48074) insert overwrite of mtmv force drop partition, because recyclebin process partition lead to performance issue.

apache#48946) ### What problem does this PR solve? Related PR: apache#48074 Problem Summary: add case for insert overwrite of mtmv force drop partition

1

75c3d79

morrySnow reviewed Feb 19, 2025

View reviewed changes

.../src/main/java/org/apache/doris/nereids/trees/plans/commands/UpdateMvByPartitionCommand.java Show resolved Hide resolved

1

4bfb2c4

morrySnow approved these changes Feb 19, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 19, 2025

github-actions bot added the reviewed label Feb 19, 2025

morrySnow added p0_b dev/2.1.x dev/3.0.x dev/3.1.x labels Feb 24, 2025

Merge branch 'master' into mv_not_recycle

da6d0b7

morrySnow closed this Mar 4, 2025

morrySnow reopened this Mar 4, 2025

morrySnow approved these changes Mar 4, 2025

View reviewed changes

morrySnow merged commit ccbc328 into apache:master Mar 4, 2025
37 checks passed

github-actions bot pushed a commit that referenced this pull request Mar 4, 2025

[enhance](mtmv) insert overwrite of mtmv force drop partition (#48074)

7431232

insert overwrite of mtmv force drop partition, because recyclebin process partition lead to performance issue.

github-actions bot mentioned this pull request Mar 4, 2025

branch-3.0: [enhance](mtmv)insert overwrite of mtmv force drop partition #48074 #48627

Merged

github-actions bot added the dev/2.1.x-conflict label Mar 4, 2025

morrySnow removed dev/2.1.x dev/2.1.x-conflict labels Mar 4, 2025

dataroaring added the kind/need-regression-test Need to add regression test for this case label Mar 10, 2025

dataroaring pushed a commit that referenced this pull request Mar 10, 2025

branch-3.0: [enhance](mtmv)insert overwrite of mtmv force drop partit…

72445f7

…ion #48074 (#48627) Cherry-picked from #48074 Co-authored-by: zhangdong <zhangdong@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Mar 10, 2025

zddr mentioned this pull request Mar 12, 2025

[case](mtmv)add case for insert overwrite of mtmv force drop partition #48946

Merged

16 tasks

morrySnow added dev/2.1.x and removed dev/2.1.x labels Mar 13, 2025

github-actions bot added the dev/2.1.x-conflict label Mar 13, 2025

morrySnow pushed a commit that referenced this pull request Apr 3, 2025

[case](mtmv)add case for insert overwrite of mtmv force drop partition (

b9d0cf1

#48946) ### What problem does this PR solve? Related PR: #48074 Problem Summary: add case for insert overwrite of mtmv force drop partition

gavinchou mentioned this pull request Apr 23, 2025

Release Notes 3.0.5 #50333

Closed

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[enhance](mtmv) insert overwrite of mtmv force drop partition (apache…

6d4fa9c

…#48074) insert overwrite of mtmv force drop partition, because recyclebin process partition lead to performance issue.

morrySnow removed the dev/3.1.x label Jun 19, 2025

[enhance](mtmv)insert overwrite of mtmv force drop partition #48074

[enhance](mtmv)insert overwrite of mtmv force drop partition #48074

Uh oh!

Conversation

zddr commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Feb 19, 2025

Uh oh!

zddr commented Feb 19, 2025

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

zddr commented Feb 19, 2025

Uh oh!

github-actions bot commented Feb 19, 2025

Uh oh!

github-actions bot commented Feb 19, 2025

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

doris-robot commented Feb 19, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

zddr commented Feb 20, 2025

Uh oh!

zddr commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

doris-robot commented Feb 26, 2025

Uh oh!

zddr commented Feb 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

zddr commented Feb 19, 2025 •

edited

Loading