Skip to content

branch-3.0: [Fix](txn) Remove TabletTxnInfo if version exists when publish version #49564 #49588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Mar 28, 2025
Merged

branch-3.0: [Fix](txn) Remove TabletTxnInfo if version exists when publish version #49564 #49588

dataroaring merged 1 commit into from
Mar 28, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Mar 27, 2025

Cherry-picked from #49564

@bobhan1
[Fix](txn) Remove TabletTxnInfo if version exists when publish vers…
13753df 
…ion (#49564)

`TabletTxnInfo` should be removed if version exists when publish
version. Otherwise, the related txn info will not be deleted and occupy
large memories
@github-actions github-actions bot requested a review from dataroaring as a code owner March 27, 2025 11:10
@Thearas
Copy link
Contributor

Thearas commented Mar 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Mar 27, 2025
@Thearas
Copy link
Contributor

Thearas commented Mar 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Mar 27, 2025

TPC-H: Total hot run time: 39964 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 13753df01d7ca0e468c94d9b1b10405686da31b0, data reload: false

------ Round 1 ----------------------------------
q1	17570	6770	6590	6590
q2	2072	169	165	165
q3	10639	1073	1155	1073
q4	10559	762	695	695
q5	7747	2880	2805	2805
q6	221	134	134	134
q7	978	611	610	610
q8	9365	1970	2065	1970
q9	6529	6410	6378	6378
q10	6986	2261	2262	2261
q11	472	250	265	250
q12	401	208	208	208
q13	17768	2994	2985	2985
q14	248	204	208	204
q15	512	478	477	477
q16	653	596	592	592
q17	963	575	597	575
q18	7330	6636	6576	6576
q19	1387	1102	973	973
q20	479	204	206	204
q21	3949	3237	3250	3237
q22	1100	1002	1022	1002
Total cold run time: 107928 ms
Total hot run time: 39964 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6581	6587	6559	6559
q2	331	235	232	232
q3	2891	2744	2856	2744
q4	2051	1807	1815	1807
q5	5759	5724	5727	5724
q6	214	131	131	131
q7	2201	1803	1846	1803
q8	3402	3537	3509	3509
q9	8790	8872	8851	8851
q10	3571	3473	3511	3473
q11	599	486	482	482
q12	787	582	593	582
q13	7900	3200	3145	3145
q14	310	280	272	272
q15	506	465	474	465
q16	722	657	652	652
q17	1840	1643	1592	1592
q18	8209	7968	7776	7776
q19	1671	1531	1493	1493
q20	2077	1901	1881	1881
q21	5518	5477	5210	5210
q22	1105	1026	1054	1026
Total cold run time: 67035 ms
Total hot run time: 59409 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2025

BE UT Coverage Report

Increment line coverage 45.00% (9/20) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 38.88% (10178/26180)
Line Coverage 30.32% (86905/286605)
Region Coverage 29.34% (44621/152086)
Branch Coverage 25.86% (22694/87764)

@doris-robot
Copy link

doris-robot commented Mar 27, 2025

TPC-DS: Total hot run time: 197829 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 13753df01d7ca0e468c94d9b1b10405686da31b0, data reload: false

query1	1259	893	944	893
query2	6230	2049	1965	1965
query3	10813	4477	4191	4191
query4	60595	28861	23679	23679
query5	5242	486	474	474
query6	420	198	183	183
query7	5492	324	308	308
query8	313	237	225	225
query9	8791	2618	2600	2600
query10	474	267	254	254
query11	17948	15218	15627	15218
query12	163	105	103	103
query13	1446	479	441	441
query14	10184	7653	7341	7341
query15	201	175	178	175
query16	6750	436	490	436
query17	1228	599	592	592
query18	1794	333	353	333
query19	216	171	168	168
query20	124	111	119	111
query21	207	102	112	102
query22	4724	4436	4637	4436
query23	34576	34162	34091	34091
query24	6388	2927	2881	2881
query25	540	450	438	438
query26	667	175	178	175
query27	1819	368	357	357
query28	4262	2466	2433	2433
query29	707	450	429	429
query30	251	165	168	165
query31	976	825	833	825
query32	70	58	60	58
query33	406	298	295	295
query34	941	512	508	508
query35	848	726	744	726
query36	1120	976	995	976
query37	114	69	65	65
query38	4098	4002	4097	4002
query39	1532	1473	1472	1472
query40	203	97	97	97
query41	50	47	46	46
query42	114	104	105	104
query43	532	496	499	496
query44	1208	843	815	815
query45	187	168	169	168
query46	1149	716	749	716
query47	2023	1933	1888	1888
query48	478	399	391	391
query49	739	402	412	402
query50	862	426	423	423
query51	7320	7322	7091	7091
query52	106	92	93	92
query53	271	186	190	186
query54	586	465	467	465
query55	80	78	83	78
query56	273	246	252	246
query57	1291	1184	1153	1153
query58	219	202	205	202
query59	3147	2903	2887	2887
query60	282	259	257	257
query61	106	102	106	102
query62	763	664	701	664
query63	216	188	188	188
query64	1403	680	670	670
query65	3258	3194	3194	3194
query66	712	297	311	297
query67	15895	15647	15670	15647
query68	4244	583	576	576
query69	427	259	261	259
query70	1157	1140	1115	1115
query71	350	258	255	255
query72	6385	4047	3975	3975
query73	760	350	355	350
query74	10451	9239	9340	9239
query75	3361	2655	2669	2655
query76	1997	973	1068	973
query77	470	282	276	276
query78	10644	9536	9566	9536
query79	1875	613	616	613
query80	1390	433	446	433
query81	532	247	236	236
query82	1268	92	88	88
query83	288	143	147	143
query84	288	79	79	79
query85	1022	295	283	283
query86	384	295	293	293
query87	4386	4312	4401	4312
query88	3797	2392	2340	2340
query89	419	291	291	291
query90	1984	188	193	188
query91	182	148	149	148
query92	61	51	51	51
query93	2185	571	577	571
query94	792	306	303	303
query95	366	265	259	259
query96	626	285	281	281
query97	3300	3142	3166	3142
query98	210	197	203	197
query99	1559	1316	1280	1280
Total cold run time: 314562 ms
Total hot run time: 197829 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2025

ClickBench: Total hot run time: 31.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 13753df01d7ca0e468c94d9b1b10405686da31b0, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.03	0.04
query3	0.24	0.07	0.06
query4	1.61	0.10	0.10
query5	0.50	0.49	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.06	0.03	0.03
query9	0.55	0.49	0.49
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.60	0.59
query14	2.84	2.82	2.85
query15	0.89	0.83	0.82
query16	0.38	0.40	0.40
query17	1.04	1.02	0.99
query18	0.24	0.21	0.22
query19	1.94	1.81	2.00
query20	0.01	0.01	0.01
query21	15.36	0.59	0.60
query22	2.77	2.81	1.78
query23	17.02	1.08	0.77
query24	3.67	0.63	0.27
query25	0.16	0.16	0.10
query26	0.51	0.14	0.13
query27	0.03	0.05	0.05
query28	11.09	0.54	0.44
query29	12.56	3.22	3.19
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.28	0.46	0.45
query33	2.98	3.07	3.00
query34	16.87	4.43	4.49
query35	4.58	4.54	4.46
query36	0.66	0.50	0.48
query37	0.09	0.07	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 108.11 s
Total hot run time: 31.24 s

dataroaring
dataroaring approved these changes Mar 28, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 28, 2025
@github-actions
Copy link
Contributor Author

github-actions bot commented Mar 28, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor Author

github-actions bot commented Mar 28, 2025

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 86e96f5 into branch-3.0 Mar 28, 2025
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-49564-branch-3.0 branch March 28, 2025 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @dataroaring @bobhan1