[enhance](mtmv)Refuse to execute insert overwrite on the same table #40558

zddr · 2024-09-09T09:57:11Z

Refuse to execute insert overwrite on the same table:

Sometimes it is not possible to cancel a running insert overwrite
When executing insert overwrite in parallel, it causes temporary partitions to affect each other

doc: apache/doris-website#1139

doris-robot · 2024-09-09T09:57:15Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zddr · 2024-09-09T10:06:32Z

run buildall

zddr · 2024-09-09T10:28:19Z

run buildall

doris-robot · 2024-09-09T10:48:12Z

TPC-H: Total hot run time: 38536 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

------ Round 1 ----------------------------------
q1	17738	4616	4390	4390
q2	2041	197	184	184
q3	11693	966	1081	966
q4	10486	808	771	771
q5	7767	2848	2804	2804
q6	229	142	145	142
q7	973	619	599	599
q8	9340	2105	2124	2105
q9	7229	6632	6656	6632
q10	7057	2310	2249	2249
q11	454	248	246	246
q12	394	221	221	221
q13	18905	3066	3045	3045
q14	280	234	239	234
q15	563	512	498	498
q16	568	422	415	415
q17	1006	719	752	719
q18	7663	6841	6870	6841
q19	1399	1055	1140	1055
q20	684	339	334	334
q21	4032	3084	3105	3084
q22	1125	1037	1002	1002
Total cold run time: 111626 ms
Total hot run time: 38536 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4621	4345	4353	4345
q2	394	267	280	267
q3	2977	2688	2712	2688
q4	1981	1738	1777	1738
q5	5769	5722	5819	5722
q6	242	141	134	134
q7	2295	1833	1859	1833
q8	3309	3478	3558	3478
q9	8952	8908	8884	8884
q10	3784	3485	3345	3345
q11	623	514	525	514
q12	837	664	690	664
q13	14529	3282	3303	3282
q14	326	292	286	286
q15	553	507	520	507
q16	542	517	511	511
q17	1907	1543	1523	1523
q18	8370	7998	8049	7998
q19	1737	1648	1564	1564
q20	2242	1885	1955	1885
q21	5959	5481	5593	5481
q22	1142	1032	1061	1032
Total cold run time: 73091 ms
Total hot run time: 57681 ms

doris-robot · 2024-09-09T11:00:25Z

TPC-DS: Total hot run time: 191771 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

query1	1274	887	876	876
query2	6355	1897	1929	1897
query3	10765	4272	4047	4047
query4	59834	27280	23136	23136
query5	5423	522	503	503
query6	397	166	172	166
query7	5769	301	294	294
query8	335	231	233	231
query9	8967	2537	2487	2487
query10	503	291	277	277
query11	18221	15049	15580	15049
query12	147	102	107	102
query13	1594	418	400	400
query14	10962	6557	7364	6557
query15	217	180	170	170
query16	7531	480	497	480
query17	1130	569	564	564
query18	2077	296	308	296
query19	273	149	150	149
query20	120	113	110	110
query21	204	103	101	101
query22	4782	4402	4493	4402
query23	34183	33378	33448	33378
query24	5964	2885	2830	2830
query25	521	377	375	375
query26	679	158	157	157
query27	1754	276	291	276
query28	3722	2033	2015	2015
query29	689	418	431	418
query30	239	158	158	158
query31	949	750	793	750
query32	90	53	56	53
query33	486	302	303	302
query34	892	487	466	466
query35	857	718	724	718
query36	1077	968	952	952
query37	145	88	91	88
query38	4049	3888	3863	3863
query39	1453	1382	1381	1381
query40	192	118	113	113
query41	46	43	43	43
query42	120	95	98	95
query43	527	478	468	468
query44	1113	739	732	732
query45	201	166	164	164
query46	1089	770	732	732
query47	1913	1798	1813	1798
query48	380	287	285	285
query49	769	439	446	439
query50	832	422	428	422
query51	7067	6817	6829	6817
query52	101	86	88	86
query53	256	183	183	183
query54	591	489	449	449
query55	76	79	76	76
query56	273	255	270	255
query57	1223	1066	1058	1058
query58	231	243	226	226
query59	3000	2979	2753	2753
query60	295	262	271	262
query61	101	102	106	102
query62	753	624	631	624
query63	216	189	179	179
query64	2649	717	662	662
query65	3206	3119	3181	3119
query66	675	338	333	333
query67	15368	15289	15172	15172
query68	3037	586	601	586
query69	403	286	309	286
query70	1211	1110	1194	1110
query71	365	293	281	281
query72	5407	4123	4033	4033
query73	767	316	321	316
query74	9142	8850	8776	8776
query75	3348	2675	2699	2675
query76	1374	997	997	997
query77	551	340	327	327
query78	9601	9006	9218	9006
query79	1046	549	531	531
query80	703	508	571	508
query81	465	229	226	226
query82	240	142	141	141
query83	176	148	148	148
query84	252	80	76	76
query85	690	294	288	288
query86	299	283	285	283
query87	4260	4259	4247	4247
query88	3062	2296	2301	2296
query89	378	289	300	289
query90	1928	193	193	193
query91	127	103	103	103
query92	62	50	51	50
query93	1069	535	542	535
query94	684	309	307	307
query95	313	250	254	250
query96	591	265	262	262
query97	3166	3056	3063	3056
query98	223	207	198	198
query99	1519	1292	1270	1270
Total cold run time: 304351 ms
Total hot run time: 191771 ms

doris-robot · 2024-09-09T11:05:55Z

ClickBench: Total hot run time: 31.09 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.05	0.04
query3	0.22	0.04	0.05
query4	1.68	0.07	0.07
query5	0.50	0.48	0.49
query6	1.12	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.49
query10	0.54	0.54	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.60	0.60
query14	1.39	1.44	1.44
query15	0.84	0.81	0.82
query16	0.37	0.37	0.37
query17	1.05	1.04	1.00
query18	0.21	0.20	0.20
query19	1.97	1.86	1.76
query20	0.01	0.00	0.01
query21	15.39	0.68	0.67
query22	4.67	7.22	1.53
query23	18.27	1.38	1.33
query24	2.11	0.24	0.22
query25	0.16	0.09	0.07
query26	0.27	0.18	0.18
query27	0.08	0.07	0.08
query28	13.22	1.01	1.00
query29	12.66	3.35	3.30
query30	0.24	0.06	0.06
query31	2.88	0.40	0.39
query32	3.24	0.48	0.47
query33	3.00	3.00	3.00
query34	17.16	4.43	4.47
query35	4.48	4.46	4.42
query36	0.65	0.49	0.47
query37	0.19	0.15	0.16
query38	0.14	0.14	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.78 s
Total hot run time: 31.09 s

zddr · 2024-09-10T02:07:52Z

run buildall

doris-robot · 2024-09-10T02:46:07Z

TPC-H: Total hot run time: 38111 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

------ Round 1 ----------------------------------
q1	17602	4388	4373	4373
q2	2013	193	187	187
q3	11772	966	1100	966
q4	10533	732	722	722
q5	7749	2835	2804	2804
q6	224	139	139	139
q7	961	633	614	614
q8	9335	2077	2042	2042
q9	7036	6516	6567	6516
q10	6978	2244	2256	2244
q11	459	254	250	250
q12	394	224	222	222
q13	17749	3043	3051	3043
q14	272	233	229	229
q15	543	496	480	480
q16	507	448	445	445
q17	966	724	744	724
q18	7311	6840	6853	6840
q19	1395	1155	1005	1005
q20	712	346	342	342
q21	3960	2957	2915	2915
q22	1099	1056	1009	1009
Total cold run time: 109570 ms
Total hot run time: 38111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4288	4331	4288
q2	377	281	269	269
q3	2876	2685	2603	2603
q4	1992	1658	1618	1618
q5	5544	5717	5775	5717
q6	237	137	134	134
q7	2231	1841	1794	1794
q8	3279	3433	3491	3433
q9	8809	8786	8729	8729
q10	3552	3383	3355	3355
q11	618	512	509	509
q12	855	675	638	638
q13	13374	3181	3307	3181
q14	318	290	284	284
q15	541	489	488	488
q16	550	509	518	509
q17	1851	1566	1548	1548
q18	8078	7714	7875	7714
q19	1725	1570	1737	1570
q20	2184	1883	1934	1883
q21	5816	5462	5412	5412
q22	1133	1043	1060	1043
Total cold run time: 70325 ms
Total hot run time: 56719 ms

doris-robot · 2024-09-10T02:58:19Z

TPC-DS: Total hot run time: 193003 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

query1	1275	895	877	877
query2	6426	1954	1969	1954
query3	10604	3914	3886	3886
query4	59467	26709	23283	23283
query5	5410	494	522	494
query6	408	174	158	158
query7	5795	302	292	292
query8	319	213	214	213
query9	8887	2503	2503	2503
query10	501	283	266	266
query11	17921	15078	15438	15078
query12	165	109	101	101
query13	1606	417	400	400
query14	10933	7357	7406	7357
query15	269	169	172	169
query16	7455	472	500	472
query17	1184	585	562	562
query18	2004	324	289	289
query19	302	151	152	151
query20	129	112	110	110
query21	215	102	106	102
query22	4479	4537	4651	4537
query23	34483	33829	33568	33568
query24	5945	2855	2867	2855
query25	531	380	380	380
query26	681	153	159	153
query27	1781	271	270	270
query28	3774	2032	2018	2018
query29	683	399	413	399
query30	248	159	155	155
query31	916	725	767	725
query32	83	54	56	54
query33	457	320	286	286
query34	869	472	462	462
query35	828	702	714	702
query36	1015	933	962	933
query37	143	81	90	81
query38	4056	3877	3868	3868
query39	1444	1406	1377	1377
query40	203	112	113	112
query41	46	48	45	45
query42	122	98	99	98
query43	500	482	482	482
query44	1076	754	736	736
query45	193	165	166	165
query46	1087	724	728	724
query47	1901	1796	1799	1796
query48	375	289	299	289
query49	768	452	452	452
query50	855	417	416	416
query51	6908	6819	6935	6819
query52	97	86	86	86
query53	247	179	184	179
query54	554	443	438	438
query55	74	72	73	72
query56	281	262	261	261
query57	1197	1066	1085	1066
query58	227	230	227	227
query59	3116	2784	2781	2781
query60	297	265	264	264
query61	103	101	101	101
query62	766	637	670	637
query63	217	184	183	183
query64	2820	726	670	670
query65	3219	3171	3210	3171
query66	612	337	344	337
query67	15562	15206	15184	15184
query68	2962	568	569	568
query69	408	283	267	267
query70	1133	1095	1117	1095
query71	333	280	272	272
query72	5433	4007	3948	3948
query73	751	332	327	327
query74	9083	8894	8857	8857
query75	3340	2672	2714	2672
query76	1442	988	993	988
query77	595	325	322	322
query78	9610	9036	9039	9036
query79	1025	542	545	542
query80	690	507	505	505
query81	476	230	235	230
query82	239	139	139	139
query83	169	148	148	148
query84	262	77	80	77
query85	674	288	275	275
query86	314	295	289	289
query87	4449	4390	4318	4318
query88	2834	2289	2274	2274
query89	388	289	285	285
query90	2002	189	191	189
query91	129	101	101	101
query92	60	115	50	50
query93	1078	529	542	529
query94	682	293	300	293
query95	326	257	247	247
query96	585	260	265	260
query97	3186	3084	3087	3084
query98	230	205	208	205
query99	1507	1291	1280	1280
Total cold run time: 303509 ms
Total hot run time: 193003 ms

doris-robot · 2024-09-10T03:03:55Z

ClickBench: Total hot run time: 31.64 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7b04297337f88bd06869152fe2889d204eabbe1b, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.05	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.52	0.48	0.50
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.54	0.49	0.48
query10	0.54	0.56	0.55
query11	0.17	0.11	0.11
query12	0.15	0.12	0.11
query13	0.60	0.59	0.60
query14	1.41	1.43	1.44
query15	0.84	0.81	0.83
query16	0.36	0.37	0.38
query17	1.03	1.08	1.01
query18	0.20	0.19	0.19
query19	1.94	1.73	1.83
query20	0.01	0.02	0.01
query21	15.68	0.68	0.67
query22	4.55	6.53	2.26
query23	18.26	1.37	1.29
query24	2.12	0.23	0.21
query25	0.16	0.08	0.08
query26	0.27	0.19	0.18
query27	0.08	0.07	0.08
query28	13.29	1.01	0.99
query29	12.62	3.31	3.33
query30	0.25	0.06	0.05
query31	2.87	0.40	0.40
query32	3.24	0.48	0.48
query33	2.96	2.99	3.01
query34	17.02	4.33	4.40
query35	4.43	4.44	4.42
query36	0.66	0.48	0.51
query37	0.19	0.16	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.05	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.75 s
Total hot run time: 31.64 s

...n/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertOverwriteTableCommand.java

fe/fe-core/src/main/java/org/apache/doris/job/extensions/mtmv/MTMVTask.java

zddr · 2024-09-11T02:04:31Z

run buildall

doris-robot · 2024-09-11T02:39:57Z

TPC-H: Total hot run time: 38059 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fabd24bf5eef2a9ed621a500a45ac88d6c4613a, data reload: false

------ Round 1 ----------------------------------
q1	17616	4470	4311	4311
q2	2041	196	182	182
q3	11745	1004	1131	1004
q4	10527	711	755	711
q5	7740	2831	2837	2831
q6	231	143	142	142
q7	959	623	630	623
q8	9337	2097	2043	2043
q9	6930	6525	6570	6525
q10	7006	2160	2189	2160
q11	483	252	245	245
q12	397	228	231	228
q13	17777	3083	3053	3053
q14	290	234	241	234
q15	543	497	495	495
q16	528	427	429	427
q17	979	636	723	636
q18	7339	6801	6933	6801
q19	1398	1091	1012	1012
q20	673	340	363	340
q21	4139	3208	3067	3067
q22	1083	989	994	989
Total cold run time: 109761 ms
Total hot run time: 38059 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4355	4266	4297	4266
q2	396	271	280	271
q3	2895	2626	2652	2626
q4	1922	1688	1630	1630
q5	5653	5742	5755	5742
q6	225	145	153	145
q7	2247	1823	1888	1823
q8	3316	3400	3436	3400
q9	8852	8863	8802	8802
q10	3594	3387	3378	3378
q11	620	522	523	522
q12	821	674	673	673
q13	14188	3271	3289	3271
q14	334	309	286	286
q15	569	496	497	496
q16	569	472	507	472
q17	1868	1576	1557	1557
q18	8091	7754	7966	7754
q19	1700	1669	1562	1562
q20	2174	1927	1915	1915
q21	5743	5560	5477	5477
q22	1125	1024	1038	1024
Total cold run time: 71257 ms
Total hot run time: 57092 ms

doris-robot · 2024-09-11T02:52:10Z

TPC-DS: Total hot run time: 197703 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fabd24bf5eef2a9ed621a500a45ac88d6c4613a, data reload: false

query1	1256	872	916	872
query2	6315	1937	1882	1882
query3	10608	3948	4018	3948
query4	60274	26960	23202	23202
query5	5148	500	497	497
query6	391	174	161	161
query7	5623	311	297	297
query8	322	212	237	212
query9	7641	2509	2478	2478
query10	412	271	264	264
query11	16353	15143	15333	15143
query12	154	116	117	116
query13	1449	406	401	401
query14	10437	7173	7157	7157
query15	207	177	178	177
query16	6690	449	439	439
query17	1079	569	557	557
query18	1593	311	315	311
query19	202	165	145	145
query20	127	117	125	117
query21	205	101	108	101
query22	4723	4532	4972	4532
query23	34454	33526	33578	33526
query24	6056	2834	2851	2834
query25	485	396	392	392
query26	600	150	153	150
query27	1605	275	277	275
query28	3648	2034	2007	2007
query29	645	411	445	411
query30	247	150	167	150
query31	925	773	757	757
query32	73	52	55	52
query33	438	283	283	283
query34	878	483	457	457
query35	847	744	720	720
query36	1086	918	971	918
query37	142	88	82	82
query38	4010	3862	3984	3862
query39	1468	1405	1430	1405
query40	195	113	112	112
query41	46	45	46	45
query42	113	95	96	95
query43	513	486	475	475
query44	1094	762	740	740
query45	194	165	167	165
query46	1092	741	747	741
query47	1912	1802	1856	1802
query48	372	301	295	295
query49	763	440	475	440
query50	824	425	421	421
query51	7041	6846	6809	6809
query52	99	87	85	85
query53	253	178	181	178
query54	564	453	459	453
query55	74	74	77	74
query56	272	254	259	254
query57	1189	1069	1078	1069
query58	238	243	233	233
query59	3073	2883	2851	2851
query60	293	272	269	269
query61	105	99	96	96
query62	744	645	666	645
query63	214	191	191	191
query64	1364	672	678	672
query65	3222	3202	3149	3149
query66	657	341	357	341
query67	15831	15682	15373	15373
query68	1170	851	823	823
query69	420	336	331	331
query70	1161	1181	1154	1154
query71	338	342	335	335
query72	4635	3484	3545	3484
query73	587	579	583	579
query74	8983	8854	8994	8854
query75	3048	2989	2943	2943
query76	1057	842	829	829
query77	411	401	412	401
query78	9614	10055	9219	9219
query79	882	875	841	841
query80	821	836	825	825
query81	469	265	262	262
query82	262	267	265	265
query83	196	195	191	191
query84	202	112	110	110
query85	639	453	447	447
query86	311	324	288	288
query87	4359	4373	4462	4373
query88	4250	4122	4125	4122
query89	372	365	366	365
query90	739	315	310	310
query91	137	138	141	138
query92	79	80	81	80
query93	904	903	891	891
query94	399	376	358	358
query95	435	422	424	422
query96	470	473	470	470
query97	3077	3135	3118	3118
query98	232	237	233	233
query99	1317	1279	1287	1279
Total cold run time: 293143 ms
Total hot run time: 197703 ms

doris-robot · 2024-09-14T08:36:29Z

ClickBench: Total hot run time: 31.51 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 041e0cba480a59fb690d3943017532d57c3f5682, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.67	0.08	0.06
query5	0.50	0.50	0.49
query6	1.16	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.57	0.52	0.51
query10	0.56	0.58	0.56
query11	0.16	0.12	0.13
query12	0.16	0.13	0.12
query13	0.62	0.62	0.61
query14	1.48	1.44	1.48
query15	0.91	0.88	0.88
query16	0.36	0.36	0.36
query17	1.03	1.04	1.03
query18	0.16	0.16	0.16
query19	1.90	1.78	1.76
query20	0.01	0.02	0.01
query21	15.39	0.68	0.66
query22	3.72	6.90	1.77
query23	17.88	1.35	1.32
query24	2.29	0.22	0.22
query25	0.18	0.08	0.07
query26	0.29	0.19	0.18
query27	0.08	0.08	0.07
query28	13.18	1.14	1.11
query29	12.54	3.37	3.33
query30	0.25	0.06	0.06
query31	2.85	0.43	0.42
query32	3.24	0.50	0.50
query33	3.04	3.11	3.07
query34	15.45	4.36	4.29
query35	4.34	4.34	4.36
query36	0.68	0.50	0.50
query37	0.19	0.16	0.17
query38	0.16	0.15	0.14
query39	0.05	0.04	0.05
query40	0.17	0.14	0.13
query41	0.09	0.05	0.05
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.83 s
Total hot run time: 31.51 s

github-actions · 2024-09-18T08:33:00Z

PR approved by at least one committer and no changes requested.

…pache#40558) Refuse to execute insert overwrite on the same table: - Sometimes it is not possible to cancel a running insert overwrite - When executing insert overwrite in parallel, it causes temporary partitions to affect each other

#40992) …(#40558) pick: #40558

…40558) Refuse to execute insert overwrite on the same table: - Sometimes it is not possible to cancel a running insert overwrite - When executing insert overwrite in parallel, it causes temporary partitions to affect each other

…ecution (#41125) after PR #40558 insert overwrite is not allowed in the same table

…ecution (apache#41125) after PR apache#40558 insert overwrite is not allowed in the same table

…ecution (#41125) after PR #40558 insert overwrite is not allowed in the same table

…xternal table (#41575) pre pr: #40558 Allow parallel running of insert overwrite on the external table

…xternal table (apache#41575) pre pr: apache#40558 Allow parallel running of insert overwrite on the external table

…write execution (#48673) ### What problem does this PR solve? Related PR: #40558 Problem Summary: Previously, on #40558, concurrent execution of `INSERT OVERWRITE` was restricted for all OLAP tables to address potential issues caused by concurrent execution. In this current PR, the restriction has been modified to only apply to Materialized Views (MTMV), as it appears that users can tolerate the uncertainty in execution results caused by concurrency for OLAP tables. ### Release note Only restrict MTMV to not allow concurrent insert overwrite execution

…write execution (apache#48673) ### What problem does this PR solve? Related PR: apache#40558 Problem Summary: Previously, on apache#40558, concurrent execution of `INSERT OVERWRITE` was restricted for all OLAP tables to address potential issues caused by concurrent execution. In this current PR, the restriction has been modified to only apply to Materialized Views (MTMV), as it appears that users can tolerate the uncertainty in execution results caused by concurrency for OLAP tables. ### Release note Only restrict MTMV to not allow concurrent insert overwrite execution

apache#40992) …(apache#40558) pick: apache#40558

zddr added 10 commits September 9, 2024 15:31

1

4c48c70

1

0b64785

1

9c38f57

1

dcb24c3

1

af8def9

1

2df4044

1

f993a59

1

cc2f221

1

ffb4c59

1

8d35ec4

1

4337cbf

1

7b04297

morrySnow requested a review from morningman September 10, 2024 08:43

morrySnow reviewed Sep 10, 2024

View reviewed changes

...n/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertOverwriteTableCommand.java Outdated Show resolved Hide resolved

fe/fe-core/src/main/java/org/apache/doris/job/extensions/mtmv/MTMVTask.java Outdated Show resolved Hide resolved

morrySnow added the dev/2.1.x label Sep 10, 2024

zddr added 2 commits September 10, 2024 16:59

Merge branch 'master' into iot_cancel

85f32d4

comment

7fabd24

morningman approved these changes Sep 18, 2024

View reviewed changes

morrySnow approved these changes Sep 18, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2024

morrySnow merged commit 91d085c into apache:master Sep 19, 2024

zddr mentioned this pull request Sep 19, 2024

[enhance](mtmv) Refuse to execute insert overwrite on the same table … #40992

Merged

yiguolei pushed a commit that referenced this pull request Sep 19, 2024

[enhance](mtmv) Refuse to execute insert overwrite on the same table … (

7a24719

#40992) …(#40558) pick: #40558

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Sep 19, 2024

gavinchou added dev/3.0.2-merged and removed dev/3.0.x labels Sep 20, 2024

This was referenced Sep 23, 2024

[case](mtmv)Failed to fix the case of concurrent insert overwrite execution #41125

Merged

[doc](mtmv)Refuse to execute insert overwrite on the same table apache/doris-website#1139

Closed

morrySnow pushed a commit that referenced this pull request Sep 23, 2024

[case](mtmv) Failed to fix the case of concurrent insert overwrite ex…

19424d5

…ecution (#41125) after PR #40558 insert overwrite is not allowed in the same table

zddr added a commit to zddr/incubator-doris that referenced this pull request Sep 23, 2024

[case](mtmv) Failed to fix the case of concurrent insert overwrite ex…

6b0f4df

…ecution (apache#41125) after PR apache#40558 insert overwrite is not allowed in the same table

gavinchou pushed a commit that referenced this pull request Sep 28, 2024

[case](mtmv) Failed to fix the case of concurrent insert overwrite ex…

eb7152a

…ecution (#41125) after PR #40558 insert overwrite is not allowed in the same table

zddr mentioned this pull request Oct 9, 2024

[enhance](catalog)Allow parallel running of insert overwrite on the external table #41575

Merged

morningman pushed a commit that referenced this pull request Oct 11, 2024

[enhance](catalog)Allow parallel running of insert overwrite on the e…

5d4ab4d

…xternal table (#41575) pre pr: #40558 Allow parallel running of insert overwrite on the external table

zddr added a commit to zddr/incubator-doris that referenced this pull request Oct 12, 2024

[enhance](catalog)Allow parallel running of insert overwrite on the e…

d971b7a

…xternal table (apache#41575) pre pr: apache#40558 Allow parallel running of insert overwrite on the external table

zddr added a commit to zddr/incubator-doris that referenced this pull request Oct 15, 2024

[enhance](catalog)Allow parallel running of insert overwrite on the e…

ad91575

…xternal table (apache#41575) pre pr: apache#40558 Allow parallel running of insert overwrite on the external table

zddr mentioned this pull request Mar 5, 2025

[enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673

Merged

16 tasks

GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Sep 26, 2025

[enhance](mtmv) Refuse to execute insert overwrite on the same table … (

7dcc9e1

apache#40992) …(apache#40558) pick: apache#40558

[enhance](mtmv)Refuse to execute insert overwrite on the same table #40558

[enhance](mtmv)Refuse to execute insert overwrite on the same table #40558

Uh oh!

Conversation

zddr commented Sep 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

zddr commented Sep 9, 2024

Uh oh!

zddr commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

zddr commented Sep 10, 2024

Uh oh!

doris-robot commented Sep 10, 2024

Uh oh!

doris-robot commented Sep 10, 2024

Uh oh!

doris-robot commented Sep 10, 2024

Uh oh!

Uh oh!

Uh oh!

zddr commented Sep 11, 2024

Uh oh!

doris-robot commented Sep 11, 2024

Uh oh!

doris-robot commented Sep 11, 2024

Uh oh!

doris-robot commented Sep 14, 2024

Uh oh!

github-actions bot commented Sep 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

zddr commented Sep 9, 2024 •

edited

Loading