Skip to content

branch-3.1: [fix](mow) fix check delete bitmap for mow tables with seq column #51763 #54417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Aug 7, 2025
Merged

branch-3.1: [fix](mow) fix check delete bitmap for mow tables with seq column #51763 #54417

morrySnow merged 1 commit into from
Aug 7, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 7, 2025

Cherry-picked from #51763

@mymeiyi
[fix](mow) fix check delete bitmap for mow tables with seq column (#5…
9d0eef2 
…1763)

### What problem does this PR solve?

when compcation for mow table with sequence column, if the output rowset
version is [start_version, end_version], it may generate delete bitmap
between [start_version, end_version], this pr modify the delete bitmap
checker to skip this situation
@github-actions github-actions bot requested a review from morrySnow as a code owner August 7, 2025 02:23
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Aug 7, 2025
@dataroaring dataroaring reopened this Aug 7, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 7, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 7, 2025

Cloud UT Coverage Report

Increment line coverage 100.00% (13/13) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.12% (1212/1494)
Line Coverage 65.40% (21461/32815)
Region Coverage 66.91% (10808/16154)
Branch Coverage 56.53% (5693/10070)

@doris-robot
Copy link

doris-robot commented Aug 7, 2025

TPC-H: Total hot run time: 32399 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d0eef26a1bf009088d7b3c7cc1a1e3afdbf0450, data reload: false

------ Round 1 ----------------------------------
q1	17608	5590	5415	5415
q2	2056	272	155	155
q3	10511	1280	769	769
q4	10297	875	448	448
q5	9110	2352	2123	2123
q6	180	161	130	130
q7	899	753	603	603
q8	9316	1443	1128	1128
q9	5234	4862	4912	4862
q10	6749	2254	1810	1810
q11	470	278	262	262
q12	340	354	211	211
q13	17768	3570	3034	3034
q14	226	223	208	208
q15	519	474	471	471
q16	417	422	366	366
q17	601	859	359	359
q18	7063	6469	6331	6331
q19	1213	959	545	545
q20	319	334	196	196
q21	2799	2188	1966	1966
q22	1082	1051	1007	1007
Total cold run time: 104777 ms
Total hot run time: 32399 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5570	5519	5470	5470
q2	230	338	234	234
q3	2251	2658	2323	2323
q4	1342	1812	1317	1317
q5	4384	4854	4948	4854
q6	166	161	127	127
q7	2086	2009	1805	1805
q8	2587	2809	2695	2695
q9	7282	7230	7245	7230
q10	3023	3308	2745	2745
q11	563	496	489	489
q12	658	779	608	608
q13	3411	3806	3199	3199
q14	282	292	267	267
q15	519	474	485	474
q16	469	498	424	424
q17	1206	1731	1246	1246
q18	7603	7474	7404	7404
q19	806	1120	1123	1120
q20	2008	2034	1884	1884
q21	5345	5050	4524	4524
q22	1110	1130	1024	1024
Total cold run time: 52901 ms
Total hot run time: 51463 ms

@doris-robot
Copy link

doris-robot commented Aug 7, 2025

TPC-DS: Total hot run time: 196783 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d0eef26a1bf009088d7b3c7cc1a1e3afdbf0450, data reload: false

query1	1296	910	887	887
query2	6210	2000	1933	1933
query3	10931	4465	4375	4375
query4	32638	23697	23346	23346
query5	3698	593	459	459
query6	265	187	180	180
query7	4003	486	310	310
query8	280	230	224	224
query9	9461	2642	2610	2610
query10	478	335	268	268
query11	18016	15712	15273	15273
query12	168	109	103	103
query13	1560	543	430	430
query14	9877	7228	7230	7228
query15	232	207	187	187
query16	7879	691	515	515
query17	1569	813	584	584
query18	2098	429	310	310
query19	201	193	165	165
query20	132	118	115	115
query21	205	133	112	112
query22	4733	4729	4443	4443
query23	35290	34422	33822	33822
query24	7900	2673	2690	2673
query25	551	492	419	419
query26	1304	287	179	179
query27	2183	495	352	352
query28	5054	2293	2238	2238
query29	751	605	465	465
query30	242	195	165	165
query31	994	923	860	860
query32	87	61	65	61
query33	518	383	315	315
query34	749	851	525	525
query35	799	825	723	723
query36	989	1058	951	951
query37	105	96	74	74
query38	4058	4037	3921	3921
query39	1529	1445	1485	1445
query40	207	108	104	104
query41	51	51	50	50
query42	131	111	102	102
query43	510	511	477	477
query44	1347	855	835	835
query45	182	180	188	180
query46	899	1077	679	679
query47	1980	1987	1924	1924
query48	414	430	338	338
query49	779	500	415	415
query50	670	697	418	418
query51	7515	7403	7257	7257
query52	104	98	95	95
query53	232	259	188	188
query54	551	558	473	473
query55	86	80	79	79
query56	261	267	266	266
query57	1279	1295	1229	1229
query58	246	216	214	214
query59	3213	3210	3097	3097
query60	328	282	268	268
query61	120	141	167	141
query62	803	770	693	693
query63	240	198	194	194
query64	4242	1009	652	652
query65	3376	3303	3254	3254
query66	1029	410	311	311
query67	15895	15879	15475	15475
query68	7928	820	552	552
query69	482	306	270	270
query70	1200	1144	1058	1058
query71	401	303	263	263
query72	5773	3791	3963	3791
query73	637	739	352	352
query74	10226	8908	9001	8908
query75	3209	3125	2670	2670
query76	3046	1147	784	784
query77	453	349	272	272
query78	10501	10330	9705	9705
query79	3824	826	591	591
query80	855	525	423	423
query81	508	265	220	220
query82	639	119	93	93
query83	171	155	146	146
query84	289	96	82	82
query85	788	364	297	297
query86	396	313	297	297
query87	4337	4380	4253	4253
query88	5220	2409	2391	2391
query89	409	340	288	288
query90	1756	194	191	191
query91	139	137	118	118
query92	70	56	51	51
query93	2748	901	553	553
query94	674	407	286	286
query95	342	275	264	264
query96	489	608	281	281
query97	3225	3323	3182	3182
query98	233	211	199	199
query99	1547	1423	1299	1299
Total cold run time: 298238 ms
Total hot run time: 196783 ms

@doris-robot
Copy link

doris-robot commented Aug 7, 2025

ClickBench: Total hot run time: 28.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d0eef26a1bf009088d7b3c7cc1a1e3afdbf0450, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.61	0.11	0.11
query5	0.54	0.49	0.52
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.50	0.50
query10	0.56	0.54	0.54
query11	0.13	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.60
query14	0.78	0.79	0.80
query15	0.86	0.82	0.82
query16	0.39	0.38	0.38
query17	1.08	0.99	0.99
query18	0.25	0.23	0.22
query19	1.98	1.86	1.80
query20	0.02	0.00	0.01
query21	15.41	1.00	0.62
query22	0.75	0.71	0.60
query23	15.24	1.44	0.54
query24	3.05	0.89	0.98
query25	0.22	0.12	0.16
query26	0.36	0.15	0.14
query27	0.06	0.04	0.06
query28	13.30	1.04	0.44
query29	12.58	3.85	3.22
query30	0.24	0.09	0.07
query31	2.83	0.59	0.39
query32	3.22	0.53	0.46
query33	2.94	2.97	3.01
query34	16.42	5.20	4.57
query35	4.52	4.56	4.63
query36	0.66	0.48	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.18	0.13	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.37 s
Total hot run time: 28.75 s

@doris-robot
Copy link

doris-robot commented Aug 7, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/11) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.39% (12642/27849)
Line Coverage 36.24% (112661/310832)
Region Coverage 35.29% (58226/165006)
Branch Coverage 32.48% (31662/97478)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 7, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/11) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.25% (20981/27517)
Line Coverage 69.62% (216251/310625)
Region Coverage 67.65% (129442/191347)
Branch Coverage 61.20% (67321/110008)

@morrySnow morrySnow merged commit e138957 into branch-3.1 Aug 7, 2025
21 of 23 checks passed
@github-actions github-actions bot deleted the auto-pick-51763-branch-3.1 branch August 7, 2025 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @morrySnow @dataroaring @mymeiyi