[branch-2.0](cherr-pick) schema change may cause mow duplicate key (33536)

[zhannngchen]

Proposed changes

Issue Number: close #xxx

cherry-pick #33536

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[zhannngchen]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[doris-robot]

TeamCity be ut coverage result:
Function Coverage: 37.84% (8064/21309)
Line Coverage: 29.52% (65874/223163)
Region Coverage: 28.98% (33898/116951)
Branch Coverage: 24.84% (17409/70080)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b_5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b/report/index.html

[doris-robot]

TPC-H: Total hot run time: 49724 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b, data reload: false

------ Round 1 ----------------------------------
q1	17737	4333	4337	4333
q2	2046	157	141	141
q3	10444	1911	1918	1911
q4	10367	1227	1330	1227
q5	8440	3913	3973	3913
q6	223	121	124	121
q7	2001	1590	1589	1589
q8	9302	2755	2698	2698
q9	10533	10398	10431	10398
q10	8609	3472	3472	3472
q11	422	233	244	233
q12	457	296	309	296
q13	18357	3962	4014	3962
q14	350	315	319	315
q15	506	451	460	451
q16	702	603	593	593
q17	1140	961	946	946
q18	7207	6967	6852	6852
q19	1644	1499	1504	1499
q20	525	307	304	304
q21	4421	4064	4091	4064
q22	508	406	407	406
Total cold run time: 115941 ms
Total hot run time: 49724 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4332	4289	4334	4289
q2	324	218	222	218
q3	4199	4183	4143	4143
q4	2772	2750	2738	2738
q5	7310	7252	7302	7252
q6	234	119	121	119
q7	3278	2850	2859	2850
q8	4403	4477	4528	4477
q9	17168	17011	17242	17011
q10	4442	4430	4426	4426
q11	763	722	686	686
q12	1080	908	938	908
q13	8448	3762	3818	3762
q14	454	440	438	438
q15	527	461	465	461
q16	796	701	705	701
q17	3834	3880	3859	3859
q18	8772	8808	8711	8711
q19	1708	1681	1642	1642
q20	2409	2156	2108	2108
q21	8509	8389	8476	8389
q22	1050	984	915	915
Total cold run time: 86812 ms
Total hot run time: 80103 ms

[doris-robot]

TPC-DS: Total hot run time: 201706 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b, data reload: false

query1	917	397	379	379
query2	6533	2071	2088	2071
query3	6922	197	197	197
query4	20453	17986	17977	17977
query5	19720	6505	6519	6505
query6	343	214	231	214
query7	4436	322	299	299
query8	257	264	222	222
query9	3125	2715	2631	2631
query10	388	287	298	287
query11	11302	10712	10659	10659
query12	118	73	72	72
query13	5572	653	641	641
query14	18678	13758	13667	13667
query15	373	224	235	224
query16	6427	273	254	254
query17	1491	1487	863	863
query18	2278	402	400	400
query19	204	145	144	144
query20	70	77	76	76
query21	188	91	92	91
query22	5162	4940	4925	4925
query23	32473	31974	31755	31755
query24	6784	6483	6532	6483
query25	531	442	427	427
query26	494	154	156	154
query27	1739	297	294	294
query28	6100	2253	2219	2219
query29	2854	2695	2754	2695
query30	239	162	160	160
query31	931	741	724	724
query32	71	58	57	57
query33	393	255	259	255
query34	848	457	483	457
query35	1129	1379	897	897
query36	1393	1381	1253	1253
query37	88	60	59	59
query38	3093	2949	2918	2918
query39	1374	1317	1320	1317
query40	204	92	94	92
query41	34	33	33	33
query42	88	85	86	85
query43	623	645	637	637
query44	1117	722	726	722
query45	241	228	224	224
query46	1231	972	952	952
query47	1929	1678	1941	1678
query48	982	683	663	663
query49	627	353	382	353
query50	871	619	604	604
query51	4755	4657	4608	4608
query52	89	73	72	72
query53	442	325	315	315
query54	2651	2440	2462	2440
query55	80	76	75	75
query56	207	183	199	183
query57	1290	1078	1168	1078
query58	217	205	199	199
query59	3314	3126	3318	3126
query60	199	194	183	183
query61	86	80	86	80
query62	802	499	509	499
query63	476	342	337	337
query64	2363	1467	1428	1428
query65	3694	3559	3521	3521
query66	751	384	352	352
query67	16114	16440	15647	15647
query68	9132	648	671	648
query69	582	334	335	334
query70	1617	1399	1419	1399
query71	415	298	300	298
query72	6466	3455	3439	3439
query73	737	325	333	325
query74	6217	5864	5890	5864
query75	5355	3660	3626	3626
query76	5653	1145	1223	1145
query77	996	250	254	250
query78	12620	11686	11589	11589
query79	7044	652	629	629
query80	947	387	388	387
query81	488	227	228	227
query82	1577	100	99	99
query83	171	132	133	132
query84	245	70	67	67
query85	833	279	271	271
query86	326	294	301	294
query87	3224	3032	2969	2969
query88	4733	2365	2356	2356
query89	352	281	283	281
query90	1956	207	210	207
query91	148	117	115	115
query92	59	48	50	48
query93	4131	576	601	576
query94	714	199	203	199
query95	1295	1060	1072	1060
query96	637	328	328	328
query97	6545	6361	6373	6361
query98	190	175	170	170
query99	2907	955	905	905
Total cold run time: 309280 ms
Total hot run time: 201706 ms

[doris-robot]

ClickBench: Total hot run time: 30.82 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.04
query4	1.79	0.07	0.06
query5	0.54	0.52	0.52
query6	1.29	0.63	0.66
query7	0.02	0.02	0.02
query8	0.04	0.02	0.02
query9	0.51	0.49	0.48
query10	0.52	0.53	0.54
query11	0.12	0.08	0.09
query12	0.12	0.09	0.10
query13	0.61	0.62	0.61
query14	0.80	0.78	0.79
query15	0.77	0.77	0.76
query16	0.36	0.37	0.35
query17	0.99	0.99	1.01
query18	0.22	0.25	0.26
query19	1.92	1.86	1.81
query20	0.02	0.01	0.02
query21	15.45	0.54	0.55
query22	2.43	2.26	2.09
query23	17.14	1.09	0.92
query24	6.88	0.79	0.66
query25	0.36	0.14	0.06
query26	0.60	0.16	0.17
query27	0.04	0.04	0.04
query28	6.80	0.74	0.78
query29	12.63	2.27	2.26
query30	0.58	0.54	0.48
query31	2.80	0.38	0.38
query32	3.40	0.50	0.49
query33	3.09	3.06	3.04
query34	15.23	4.78	4.77
query35	4.86	4.86	4.86
query36	1.06	1.01	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.02
query43	0.03	0.02	0.01
Total cold run time: 104.72 s
Total hot run time: 30.82 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5e5efd6ef4a2c7c37454d1b50d1e76b64a90d63b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.2 seconds inserted 10000000 Rows, about 495K ops/s

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.

[zhannngchen]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[doris-robot]

TPC-H: Total hot run time: 50460 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d9d89de21840ea82ebb64764f964b1245e4ef2d, data reload: false

------ Round 1 ----------------------------------
q1	19412	5292	4602	4602
q2	2862	162	160	160
q3	12350	1962	2001	1962
q4	10119	1231	1284	1231
q5	8387	4003	4012	4003
q6	232	127	122	122
q7	2084	1623	1642	1623
q8	9273	2727	2732	2727
q9	10965	10525	10434	10434
q10	8612	3517	3525	3517
q11	424	244	237	237
q12	477	299	302	299
q13	18323	3965	4019	3965
q14	352	329	328	328
q15	500	474	450	450
q16	703	601	609	601
q17	1126	962	946	946
q18	7293	6937	6914	6914
q19	1679	1534	1564	1534
q20	507	311	316	311
q21	4423	4108	4125	4108
q22	490	386	397	386
Total cold run time: 120593 ms
Total hot run time: 50460 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4343	4276	4292	4276
q2	320	214	215	214
q3	4181	4201	4161	4161
q4	2736	2755	2733	2733
q5	7260	7195	7209	7195
q6	237	121	117	117
q7	3217	2885	2830	2830
q8	4351	4515	4500	4500
q9	17170	16743	16969	16743
q10	4272	4225	4245	4225
q11	770	687	694	687
q12	1044	857	840	840
q13	7582	3768	3723	3723
q14	465	425	421	421
q15	499	457	450	450
q16	762	693	709	693
q17	3879	3827	3810	3810
q18	8817	8671	8804	8671
q19	1722	1701	1661	1661
q20	2435	2149	2125	2125
q21	8569	8476	8483	8476
q22	1017	966	919	919
Total cold run time: 85648 ms
Total hot run time: 79470 ms

[doris-robot]

TeamCity be ut coverage result:
Function Coverage: 37.84% (8066/21315)
Line Coverage: 29.52% (65894/223188)
Region Coverage: 29.00% (33912/116953)
Branch Coverage: 24.85% (17415/70074)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2d9d89de21840ea82ebb64764f964b1245e4ef2d_2d9d89de21840ea82ebb64764f964b1245e4ef2d/report/index.html

[doris-robot]

TPC-DS: Total hot run time: 201943 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d9d89de21840ea82ebb64764f964b1245e4ef2d, data reload: false

query1	939	392	379	379
query2	6517	2260	2140	2140
query3	6922	202	202	202
query4	21024	18022	18103	18022
query5	19730	6517	6538	6517
query6	285	218	235	218
query7	4163	302	296	296
query8	252	255	235	235
query9	3184	2765	2693	2693
query10	408	273	297	273
query11	11469	10801	10804	10801
query12	122	78	71	71
query13	5571	663	635	635
query14	18055	13828	13962	13828
query15	371	233	239	233
query16	6453	283	253	253
query17	1713	1453	866	866
query18	2326	398	400	398
query19	196	147	153	147
query20	80	79	74	74
query21	191	91	93	91
query22	5243	4874	5070	4874
query23	32687	31878	31863	31863
query24	6901	6552	6509	6509
query25	512	414	424	414
query26	525	157	161	157
query27	1881	291	290	290
query28	6185	2274	2246	2246
query29	2918	2587	2817	2587
query30	238	162	159	159
query31	926	732	738	732
query32	67	62	59	59
query33	399	249	237	237
query34	868	461	472	461
query35	1080	927	894	894
query36	1248	1359	1232	1232
query37	91	59	61	59
query38	3084	2884	2905	2884
query39	1366	1306	1323	1306
query40	203	92	99	92
query41	34	32	32	32
query42	77	88	81	81
query43	634	702	678	678
query44	1113	716	706	706
query45	240	226	228	226
query46	1230	966	954	954
query47	1816	1832	1659	1659
query48	973	674	657	657
query49	613	366	376	366
query50	882	578	600	578
query51	4710	4591	4702	4591
query52	88	77	78	77
query53	437	317	309	309
query54	2651	2485	2486	2485
query55	85	72	73	72
query56	209	208	194	194
query57	1241	1059	1129	1059
query58	211	198	200	198
query59	3333	3108	3173	3108
query60	207	199	193	193
query61	89	83	82	82
query62	748	462	511	462
query63	469	331	324	324
query64	2378	1503	1433	1433
query65	3643	3551	3576	3551
query66	820	372	369	369
query67	16077	15979	15430	15430
query68	9354	630	649	630
query69	568	343	327	327
query70	1750	1474	1411	1411
query71	409	294	310	294
query72	6441	3451	3450	3450
query73	736	312	332	312
query74	6282	5821	5815	5815
query75	5443	3678	3777	3678
query76	5938	1172	1223	1172
query77	1014	254	240	240
query78	12567	11795	11684	11684
query79	7266	653	647	647
query80	1104	394	372	372
query81	491	235	226	226
query82	1598	103	93	93
query83	168	128	132	128
query84	262	67	67	67
query85	858	288	289	288
query86	331	294	283	283
query87	3248	3045	3037	3037
query88	4811	2269	2266	2266
query89	380	278	311	278
query90	1958	202	214	202
query91	152	111	117	111
query92	55	48	54	48
query93	5250	623	578	578
query94	697	200	207	200
query95	1105	1068	1047	1047
query96	630	318	322	318
query97	6483	6328	6363	6328
query98	174	174	170	170
query99	2878	942	972	942
Total cold run time: 311802 ms
Total hot run time: 201943 ms

[doris-robot]

ClickBench: Total hot run time: 31.87 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2d9d89de21840ea82ebb64764f964b1245e4ef2d, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.78	0.08	0.09
query5	0.53	0.53	0.52
query6	1.24	0.62	0.63
query7	0.01	0.01	0.01
query8	0.03	0.03	0.03
query9	0.52	0.48	0.48
query10	0.54	0.54	0.54
query11	0.12	0.08	0.09
query12	0.11	0.10	0.09
query13	0.61	0.61	0.62
query14	0.77	0.78	0.80
query15	0.79	0.76	0.75
query16	0.38	0.38	0.36
query17	0.98	1.03	1.03
query18	0.21	0.26	0.23
query19	1.90	1.87	1.85
query20	0.02	0.01	0.02
query21	15.49	0.55	0.56
query22	2.02	2.54	1.98
query23	17.04	0.98	1.19
query24	5.09	1.65	1.75
query25	0.34	0.10	0.06
query26	0.66	0.15	0.16
query27	0.05	0.04	0.03
query28	7.51	0.76	0.73
query29	12.64	2.28	2.34
query30	0.60	0.48	0.53
query31	2.80	0.39	0.38
query32	3.38	0.50	0.49
query33	3.10	3.06	3.10
query34	15.24	4.81	4.80
query35	4.87	4.83	4.86
query36	1.05	1.00	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.17	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.15 s
Total hot run time: 31.87 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2d9d89de21840ea82ebb64764f964b1245e4ef2d with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.0 seconds inserted 10000000 Rows, about 500K ops/s

[dataroaring]

LGTM