[Fix](full compaction) Fix problems for full compaction #49825

bobhan1 · 2025-04-07T06:27:10Z

What problem does this PR solve?

in local mode, full compaction should update tablet's rowsets meta and delete bitmaps for incremental rowsets under same lock.
in local mode, full compaction should not preserve delete bitmap sentinel marks.
in cloud mode, full compaction should update correct delete bitmaps' version for incremental rowsets.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

bobhan1 · 2025-04-07T06:33:04Z

run buildall

bobhan1 · 2025-04-07T06:53:37Z

run buildall

doris-robot · 2025-04-07T07:12:18Z

TPC-H: Total hot run time: 34315 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2cb3a289aa60e58cc563a8915db33225362e133d, data reload: false

------ Round 1 ----------------------------------
q1	25770	5165	5193	5165
q2	2067	275	188	188
q3	10679	1224	680	680
q4	10262	1015	518	518
q5	8293	2380	2350	2350
q6	196	165	135	135
q7	932	768	608	608
q8	9311	1319	1088	1088
q9	6750	5117	5123	5117
q10	6817	2318	1901	1901
q11	480	295	271	271
q12	355	369	222	222
q13	17781	3739	3128	3128
q14	250	224	209	209
q15	533	491	472	472
q16	637	634	572	572
q17	605	868	347	347
q18	7707	7261	7122	7122
q19	1855	968	558	558
q20	330	344	233	233
q21	4251	3523	2481	2481
q22	1080	1034	950	950
Total cold run time: 116941 ms
Total hot run time: 34315 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5475	5318	5313	5313
q2	242	325	237	237
q3	2195	2761	2375	2375
q4	1490	1895	1389	1389
q5	4348	4371	4398	4371
q6	216	167	129	129
q7	2030	1932	1787	1787
q8	2598	2580	2546	2546
q9	7131	7094	7231	7094
q10	3065	3247	2792	2792
q11	592	518	493	493
q12	668	736	586	586
q13	3391	3929	3345	3345
q14	285	297	268	268
q15	543	484	474	474
q16	668	717	674	674
q17	1165	1544	1428	1428
q18	7807	7611	7445	7445
q19	880	829	824	824
q20	2024	2061	1894	1894
q21	5658	5015	4933	4933
q22	1094	1049	1002	1002
Total cold run time: 53565 ms
Total hot run time: 51399 ms

doris-robot · 2025-04-07T07:23:53Z

TPC-DS: Total hot run time: 186755 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2cb3a289aa60e58cc563a8915db33225362e133d, data reload: false

query1	984	500	486	486
query2	6546	1930	1927	1927
query3	6604	232	227	227
query4	25253	23189	22946	22946
query5	4343	625	465	465
query6	305	214	206	206
query7	4569	483	279	279
query8	306	233	227	227
query9	8568	2569	2562	2562
query10	462	316	268	268
query11	15423	15089	14832	14832
query12	157	111	108	108
query13	1668	519	395	395
query14	9237	6230	6272	6230
query15	203	186	159	159
query16	7162	650	469	469
query17	1079	732	571	571
query18	1990	405	331	331
query19	199	184	166	166
query20	122	118	120	118
query21	218	125	104	104
query22	4194	4332	4105	4105
query23	34200	33266	33256	33256
query24	8498	2430	2463	2430
query25	527	502	391	391
query26	1330	281	154	154
query27	2939	501	341	341
query28	4484	2460	2437	2437
query29	743	569	448	448
query30	305	230	200	200
query31	947	866	795	795
query32	73	67	69	67
query33	555	375	306	306
query34	794	875	524	524
query35	795	835	773	773
query36	1010	1003	939	939
query37	133	104	82	82
query38	4193	4091	4124	4091
query39	1517	1612	1464	1464
query40	222	124	115	115
query41	55	56	54	54
query42	128	116	115	115
query43	521	529	493	493
query44	1357	816	817	816
query45	184	183	175	175
query46	885	1061	626	626
query47	1899	1842	1717	1717
query48	384	408	316	316
query49	767	511	417	417
query50	678	694	398	398
query51	4104	4181	4062	4062
query52	108	108	99	99
query53	216	249	182	182
query54	571	567	512	512
query55	90	86	81	81
query56	301	304	292	292
query57	1114	1147	1063	1063
query58	264	262	263	262
query59	2599	2661	2547	2547
query60	325	325	311	311
query61	135	132	132	132
query62	817	722	645	645
query63	223	190	185	185
query64	4201	1024	738	738
query65	4308	4205	4205	4205
query66	1118	408	310	310
query67	15809	15310	15466	15310
query68	7926	888	520	520
query69	484	298	266	266
query70	1176	1120	1082	1082
query71	486	324	289	289
query72	5816	4769	4815	4769
query73	717	606	336	336
query74	8871	9074	8840	8840
query75	4022	3217	2690	2690
query76	3819	1187	776	776
query77	796	389	296	296
query78	10158	10063	9261	9261
query79	2048	814	572	572
query80	698	533	444	444
query81	513	258	222	222
query82	440	126	99	99
query83	287	257	238	238
query84	300	112	89	89
query85	783	405	317	317
query86	337	295	281	281
query87	4446	4510	4415	4415
query88	3393	2203	2177	2177
query89	384	307	283	283
query90	1963	216	210	210
query91	147	144	114	114
query92	79	66	56	56
query93	1216	960	592	592
query94	679	399	312	312
query95	371	295	291	291
query96	488	556	271	271
query97	3278	3240	3096	3096
query98	230	205	199	199
query99	1443	1363	1295	1295
Total cold run time: 273812 ms
Total hot run time: 186755 ms

doris-robot · 2025-04-07T07:29:12Z

ClickBench: Total hot run time: 31.65 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2cb3a289aa60e58cc563a8915db33225362e133d, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.11	0.10
query3	0.24	0.19	0.20
query4	1.60	0.20	0.20
query5	0.60	0.56	0.59
query6	1.18	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.57	0.51	0.50
query10	0.59	0.58	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.59	0.61
query14	2.65	2.83	2.79
query15	0.93	0.85	0.85
query16	0.39	0.39	0.38
query17	1.02	1.04	1.00
query18	0.22	0.19	0.19
query19	1.90	1.88	1.94
query20	0.02	0.01	0.01
query21	15.35	0.93	0.58
query22	0.76	1.14	0.62
query23	15.08	1.37	0.66
query24	7.16	1.59	1.00
query25	0.52	0.17	0.19
query26	0.54	0.15	0.14
query27	0.05	0.05	0.04
query28	10.08	0.88	0.42
query29	12.53	4.05	3.42
query30	0.25	0.08	0.07
query31	2.84	0.57	0.38
query32	3.22	0.55	0.46
query33	2.99	3.06	3.09
query34	15.85	5.12	4.48
query35	4.49	4.55	4.52
query36	0.67	0.51	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.13	0.14
query41	0.09	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.93 s
Total hot run time: 31.65 s

doris-robot · 2025-04-07T09:08:25Z

BE UT Coverage Report

Increment line coverage 0.00% (0/39) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.25% (14020/26830)
Line Coverage	41.04% (121037/294910)
Region Coverage	39.77% (61596/154890)
Branch Coverage	34.44% (30838/89544)

bobhan1 · 2025-04-07T09:34:10Z

run buildall

doris-robot · 2025-04-07T09:55:53Z

TPC-H: Total hot run time: 34341 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5f736f858daed616a7560571f8731936a5c3ce3a, data reload: false

------ Round 1 ----------------------------------
q1	26155	5100	4993	4993
q2	2060	277	197	197
q3	10396	1246	692	692
q4	10221	1006	539	539
q5	7460	2403	2407	2403
q6	191	163	132	132
q7	920	748	621	621
q8	9331	1286	1121	1121
q9	6926	5104	5045	5045
q10	6832	2330	1902	1902
q11	480	284	270	270
q12	347	357	220	220
q13	17737	3637	3139	3139
q14	236	221	204	204
q15	516	497	494	494
q16	622	607	604	604
q17	594	869	362	362
q18	7512	7281	7146	7146
q19	1093	955	565	565
q20	338	330	221	221
q21	4241	3456	2493	2493
q22	1070	1006	978	978
Total cold run time: 115278 ms
Total hot run time: 34341 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5124	5117	5066	5066
q2	237	324	227	227
q3	2141	2643	2255	2255
q4	1410	1818	1427	1427
q5	4495	4514	4367	4367
q6	206	163	123	123
q7	1971	1909	1732	1732
q8	2585	2514	2524	2514
q9	7193	7185	7213	7185
q10	3008	3194	2735	2735
q11	563	491	498	491
q12	698	724	595	595
q13	3516	3900	3218	3218
q14	275	293	268	268
q15	510	467	490	467
q16	648	682	635	635
q17	1123	1607	1345	1345
q18	7716	7444	7555	7444
q19	824	767	846	767
q20	1965	2078	1856	1856
q21	5079	4712	4603	4603
q22	1055	1031	1013	1013
Total cold run time: 52342 ms
Total hot run time: 50333 ms

doris-robot · 2025-04-07T10:07:28Z

TPC-DS: Total hot run time: 185862 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5f736f858daed616a7560571f8731936a5c3ce3a, data reload: false

query1	999	481	469	469
query2	6564	1979	1959	1959
query3	6747	221	215	215
query4	26469	23555	23187	23187
query5	4784	613	463	463
query6	317	207	207	207
query7	4621	482	302	302
query8	302	248	257	248
query9	8669	2612	2613	2612
query10	504	342	265	265
query11	15692	15062	14813	14813
query12	165	111	103	103
query13	1666	509	395	395
query14	9610	6148	6037	6037
query15	209	185	165	165
query16	7655	648	465	465
query17	1200	725	578	578
query18	2023	407	315	315
query19	197	188	160	160
query20	124	122	119	119
query21	216	123	109	109
query22	4131	4398	3958	3958
query23	34182	33009	33083	33009
query24	7457	2362	2339	2339
query25	517	448	392	392
query26	1230	270	153	153
query27	2579	482	349	349
query28	4319	2405	2375	2375
query29	715	563	418	418
query30	288	217	195	195
query31	957	855	770	770
query32	72	63	65	63
query33	563	355	301	301
query34	808	824	543	543
query35	772	817	754	754
query36	935	1012	910	910
query37	117	98	76	76
query38	4121	4087	4203	4087
query39	1461	1393	1407	1393
query40	215	120	111	111
query41	62	51	54	51
query42	121	103	107	103
query43	485	505	500	500
query44	1288	795	784	784
query45	181	170	165	165
query46	830	993	618	618
query47	1734	1817	1708	1708
query48	383	416	296	296
query49	783	500	440	440
query50	631	685	401	401
query51	4219	4178	4037	4037
query52	105	105	99	99
query53	230	259	189	189
query54	580	573	505	505
query55	84	81	81	81
query56	312	322	279	279
query57	1125	1150	1063	1063
query58	267	265	262	262
query59	2618	2786	2602	2602
query60	344	322	307	307
query61	131	127	129	127
query62	787	733	656	656
query63	225	182	181	181
query64	4272	1017	692	692
query65	4340	4248	4243	4243
query66	1071	415	318	318
query67	15847	15620	15365	15365
query68	8093	891	516	516
query69	470	305	266	266
query70	1186	1048	1087	1048
query71	488	322	286	286
query72	5593	4655	4633	4633
query73	699	568	336	336
query74	8914	9088	8684	8684
query75	3946	3204	2737	2737
query76	3699	1200	753	753
query77	809	376	287	287
query78	10187	10160	9313	9313
query79	2219	800	575	575
query80	589	516	445	445
query81	491	342	221	221
query82	486	125	97	97
query83	264	255	235	235
query84	247	101	82	82
query85	793	343	307	307
query86	390	301	276	276
query87	4456	4572	4357	4357
query88	3723	2231	2204	2204
query89	398	318	282	282
query90	1901	212	213	212
query91	140	139	109	109
query92	77	62	57	57
query93	1746	956	581	581
query94	673	430	318	318
query95	378	298	282	282
query96	488	561	370	370
query97	3133	3228	3085	3085
query98	242	203	203	203
query99	1433	1397	1251	1251
Total cold run time: 275578 ms
Total hot run time: 185862 ms

doris-robot · 2025-04-07T10:12:46Z

ClickBench: Total hot run time: 31.18 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5f736f858daed616a7560571f8731936a5c3ce3a, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.25	0.19	0.18
query4	1.59	0.20	0.18
query5	0.59	0.59	0.59
query6	1.16	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.53	0.53
query10	0.56	0.59	0.58
query11	0.16	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.61
query14	2.69	2.70	2.79
query15	0.92	0.85	0.84
query16	0.38	0.38	0.38
query17	1.10	1.05	0.99
query18	0.21	0.20	0.20
query19	1.93	2.02	1.76
query20	0.01	0.02	0.01
query21	15.35	0.86	0.53
query22	0.75	1.22	0.61
query23	14.98	1.38	0.62
query24	7.01	1.44	1.04
query25	0.52	0.25	0.09
query26	0.52	0.15	0.13
query27	0.05	0.04	0.04
query28	9.67	0.92	0.43
query29	12.55	3.89	3.29
query30	0.25	0.08	0.06
query31	2.84	0.58	0.38
query32	3.23	0.56	0.46
query33	3.04	3.06	3.02
query34	15.94	5.09	4.51
query35	4.49	4.52	4.52
query36	0.66	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.18	0.13	0.14
query41	0.07	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.03	0.02
Total cold run time: 105.43 s
Total hot run time: 31.18 s

bobhan1 · 2025-04-07T10:54:37Z

run external

bobhan1 · 2025-04-07T11:43:04Z

run buildall

doris-robot · 2025-04-07T12:01:38Z

TPC-H: Total hot run time: 34298 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eddda20b1a345f79abbc5ae5518cafc61895eaec, data reload: false

------ Round 1 ----------------------------------
q1	26306	5015	5008	5008
q2	2071	282	182	182
q3	10404	1260	674	674
q4	10233	1025	546	546
q5	7541	2362	2353	2353
q6	183	162	132	132
q7	911	754	610	610
q8	9350	1301	1149	1149
q9	6895	5123	5126	5123
q10	6825	2291	1872	1872
q11	482	284	266	266
q12	344	355	218	218
q13	17787	3801	3112	3112
q14	233	220	207	207
q15	536	494	491	491
q16	630	612	573	573
q17	567	857	358	358
q18	7607	7232	7180	7180
q19	1227	931	547	547
q20	332	348	220	220
q21	4098	3462	2496	2496
q22	1066	1007	981	981
Total cold run time: 115628 ms
Total hot run time: 34298 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5089	5060	5113	5060
q2	230	323	228	228
q3	2185	2633	2289	2289
q4	1445	1901	1498	1498
q5	4553	4414	4369	4369
q6	215	163	129	129
q7	1982	1905	1744	1744
q8	2564	2592	2446	2446
q9	7334	7272	7146	7146
q10	2977	3204	2727	2727
q11	559	503	498	498
q12	703	786	601	601
q13	3510	3883	3303	3303
q14	272	280	254	254
q15	525	482	478	478
q16	665	692	663	663
q17	1139	1542	1342	1342
q18	7806	7451	7414	7414
q19	779	795	936	795
q20	1867	1905	1802	1802
q21	5314	4811	4606	4606
q22	1046	1014	995	995
Total cold run time: 52759 ms
Total hot run time: 50387 ms

doris-robot · 2025-04-07T12:13:15Z

TPC-DS: Total hot run time: 186368 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eddda20b1a345f79abbc5ae5518cafc61895eaec, data reload: false

query1	1005	487	477	477
query2	6544	1930	1914	1914
query3	6758	217	219	217
query4	26080	23898	22968	22968
query5	4364	644	471	471
query6	304	210	210	210
query7	4648	493	290	290
query8	310	262	255	255
query9	8643	2596	2615	2596
query10	491	328	273	273
query11	15690	15055	14816	14816
query12	166	110	107	107
query13	1679	530	408	408
query14	8745	6219	6325	6219
query15	212	193	176	176
query16	7222	616	482	482
query17	1210	739	590	590
query18	1971	407	310	310
query19	195	187	169	169
query20	120	122	122	122
query21	215	132	104	104
query22	4273	4268	4166	4166
query23	33834	32896	33008	32896
query24	8050	2391	2368	2368
query25	548	448	402	402
query26	1229	268	149	149
query27	2711	499	333	333
query28	4315	2401	2383	2383
query29	806	572	421	421
query30	285	222	193	193
query31	935	843	745	745
query32	73	66	64	64
query33	568	361	305	305
query34	797	843	518	518
query35	789	849	740	740
query36	969	965	888	888
query37	118	103	77	77
query38	4323	4101	4145	4101
query39	1463	1413	1379	1379
query40	209	123	106	106
query41	57	60	53	53
query42	117	105	102	102
query43	491	510	493	493
query44	1268	790	791	790
query45	191	178	175	175
query46	844	1038	624	624
query47	1757	1782	1706	1706
query48	374	418	297	297
query49	792	518	404	404
query50	657	700	398	398
query51	4191	4137	4109	4109
query52	108	104	95	95
query53	247	260	190	190
query54	576	567	506	506
query55	84	78	81	78
query56	306	308	283	283
query57	1158	1139	1066	1066
query58	273	261	297	261
query59	2458	2657	2743	2657
query60	336	318	298	298
query61	132	129	133	129
query62	782	719	669	669
query63	233	188	192	188
query64	4493	1032	754	754
query65	4319	4237	4286	4237
query66	1173	419	348	348
query67	16018	15683	15457	15457
query68	9031	884	515	515
query69	473	302	262	262
query70	1177	1096	1107	1096
query71	452	306	290	290
query72	5741	4755	4918	4755
query73	731	641	349	349
query74	8851	8811	8898	8811
query75	4288	3222	2713	2713
query76	3695	1187	752	752
query77	800	394	279	279
query78	9882	10106	9222	9222
query79	3712	834	565	565
query80	664	527	458	458
query81	476	268	229	229
query82	567	131	104	104
query83	304	264	238	238
query84	300	111	86	86
query85	805	445	318	318
query86	336	323	291	291
query87	4484	4586	4275	4275
query88	3057	2226	2231	2226
query89	444	312	283	283
query90	1924	220	222	220
query91	146	143	112	112
query92	81	66	59	59
query93	2091	951	588	588
query94	685	425	316	316
query95	388	293	294	293
query96	494	569	283	283
query97	3265	3201	3115	3115
query98	230	202	207	202
query99	1427	1452	1288	1288
Total cold run time: 277303 ms
Total hot run time: 186368 ms

doris-robot · 2025-04-07T12:18:42Z

ClickBench: Total hot run time: 31.37 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eddda20b1a345f79abbc5ae5518cafc61895eaec, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.24	0.19	0.19
query4	1.59	0.19	0.20
query5	0.58	0.58	0.59
query6	1.19	0.72	0.73
query7	0.03	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.53	0.52
query10	0.60	0.59	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.59	0.60
query14	2.66	2.82	2.66
query15	0.92	0.84	0.84
query16	0.39	0.38	0.38
query17	1.03	1.05	1.02
query18	0.20	0.20	0.19
query19	1.91	1.94	1.84
query20	0.01	0.01	0.01
query21	15.36	0.93	0.56
query22	0.76	1.28	0.66
query23	14.83	1.37	0.61
query24	6.72	1.55	1.04
query25	0.46	0.26	0.10
query26	0.48	0.17	0.14
query27	0.05	0.05	0.05
query28	10.22	0.95	0.43
query29	12.56	3.96	3.35
query30	0.25	0.10	0.07
query31	2.82	0.57	0.39
query32	3.23	0.55	0.47
query33	3.01	3.06	3.04
query34	15.87	5.13	4.47
query35	4.47	4.56	4.49
query36	0.65	0.50	0.49
query37	0.09	0.06	0.07
query38	0.04	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.27 s
Total hot run time: 31.37 s

doris-robot · 2025-04-07T13:56:55Z

BE UT Coverage Report

Increment line coverage 0.00% (0/60) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.26% (14022/26830)
Line Coverage	41.04% (121047/294951)
Region Coverage	39.77% (61611/154899)
Branch Coverage	34.45% (30851/89554)

bobhan1 · 2025-04-08T01:48:21Z

run p0

bobhan1 · 2025-04-08T01:48:27Z

run cloud_p0

dataroaring

LGTM

github-actions · 2025-04-08T07:17:27Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-04-08T07:17:29Z

PR approved by anyone and no changes requested.

zhannngchen

LGTM

### What problem does this PR solve? 1. in local mode, full compaction should update tablet's rowsets meta and delete bitmaps for incremental rowsets under same lock. 2. in local mode, full compaction should not preserve delete bitmap sentinel marks. 3. in cloud mode, full compaction should update correct delete bitmaps' version for incremental rowsets.

…49825 (#49917) Cherry-picked from #49825 Co-authored-by: bobhan1 <baohan@selectdb.com>

…ion #49825 (#49919) pick #49825

### What problem does this PR solve? 1. in local mode, full compaction should update tablet's rowsets meta and delete bitmaps for incremental rowsets under same lock. 2. in local mode, full compaction should not preserve delete bitmap sentinel marks. 3. in cloud mode, full compaction should update correct delete bitmaps' version for incremental rowsets.

bobhan1 added 2 commits April 7, 2025 12:19

add case

740499e

fix and add case

49dcf01

bobhan1 force-pushed the fix-mow-full-compaction branch from 1145efc to 49dcf01 Compare April 7, 2025 06:32

remove marks

2cb3a28

bobhan1 requested review from dataroaring and gavinchou as code owners April 7, 2025 08:17

fix

ba11f20

bobhan1 force-pushed the fix-mow-full-compaction branch from ffa00b0 to ba11f20 Compare April 7, 2025 08:30

bobhan1 requested a review from w41ter as a code owner April 7, 2025 09:30

update

5f736f8

bobhan1 force-pushed the fix-mow-full-compaction branch from c82428f to 5f736f8 Compare April 7, 2025 09:33

bobhan1 changed the title ~~[Fix](full compaction) Should update tablet's rowsets and delete bitmaps under the same lock~~ [Fix](full compaction) Fix problems for full compaction Apr 7, 2025

update

eddda20

dataroaring approved these changes Apr 8, 2025

View reviewed changes

dataroaring added dev/3.0.x dev/2.1.x p0_w labels Apr 8, 2025

github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 8, 2025

github-actions bot added the reviewed label Apr 8, 2025

zhannngchen approved these changes Apr 9, 2025

View reviewed changes

zhannngchen merged commit 595b7c7 into apache:master Apr 9, 2025
26 of 29 checks passed

github-actions bot mentioned this pull request Apr 9, 2025

branch-3.0: [Fix](full compaction) Fix problems for full compaction #49825 #49917

Merged

github-actions bot added the dev/2.1.x-conflict label Apr 9, 2025

bobhan1 mentioned this pull request Apr 9, 2025

branch-2.1-pick: [Fix](full compaction) Fix problems for full compaction #49825 #49919

Merged

dataroaring pushed a commit that referenced this pull request Apr 10, 2025

branch-3.0: [Fix](full compaction) Fix problems for full compaction #…

3de3682

…49825 (#49917) Cherry-picked from #49825 Co-authored-by: bobhan1 <baohan@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Apr 10, 2025

yiguolei pushed a commit that referenced this pull request Apr 10, 2025

branch-2.1-pick: [Fix](full compaction) Fix problems for full compact…

308ede0

…ion #49825 (#49919) pick #49825

yiguolei added dev/2.1.10-merged and removed dev/2.1.x dev/2.1.x-conflict labels Apr 10, 2025

gavinchou mentioned this pull request Apr 23, 2025

Release Notes 3.0.5 #50333

Closed

yiguolei mentioned this pull request May 13, 2025

2.1.10 Release Note #50853

Closed

[Fix](full compaction) Fix problems for full compaction #49825

[Fix](full compaction) Fix problems for full compaction #49825

Uh oh!

Conversation

bobhan1 commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

bobhan1 commented Apr 7, 2025

Uh oh!

bobhan1 commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

BE UT Coverage Report

Uh oh!

bobhan1 commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

bobhan1 commented Apr 7, 2025

Uh oh!

bobhan1 commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

BE UT Coverage Report

Uh oh!

bobhan1 commented Apr 8, 2025

Uh oh!

bobhan1 commented Apr 8, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

github-actions bot commented Apr 8, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bobhan1 commented Apr 7, 2025 •

edited

Loading