Skip to content

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Mar 27, 2025

What problem does this PR solve?

TabletTxnInfo should be removed if version exists when publish version. Otherwise, the related txn info will not be deleted and occupy large memories

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1
Copy link
Contributor Author

bobhan1 commented Mar 27, 2025

run buildall

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 27, 2025
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 34159 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 81f6ec44815c8680f3bdb898b7bad4dba0547912, data reload: false

------ Round 1 ----------------------------------
q1	26215	5094	5079	5079
q2	2079	298	165	165
q3	10501	1264	683	683
q4	10237	1010	521	521
q5	7614	2378	2387	2378
q6	188	164	132	132
q7	927	751	600	600
q8	9321	1327	1093	1093
q9	6921	5160	5119	5119
q10	6822	2322	1892	1892
q11	490	273	266	266
q12	374	350	220	220
q13	17761	3703	3053	3053
q14	228	231	207	207
q15	537	494	486	486
q16	639	631	577	577
q17	576	871	348	348
q18	7737	7270	7213	7213
q19	1229	962	555	555
q20	318	335	188	188
q21	3986	2635	2384	2384
q22	1088	1030	1000	1000
Total cold run time: 115788 ms
Total hot run time: 34159 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5156	5116	5142	5116
q2	241	324	225	225
q3	2172	2680	2330	2330
q4	1459	1813	1468	1468
q5	4518	4492	4395	4395
q6	220	168	123	123
q7	1971	1939	1741	1741
q8	2630	2579	2568	2568
q9	7273	7170	7178	7170
q10	3039	3179	2751	2751
q11	581	512	480	480
q12	711	787	576	576
q13	3424	3992	3303	3303
q14	273	292	273	273
q15	514	481	495	481
q16	656	682	644	644
q17	1148	1576	1396	1396
q18	7647	7648	7391	7391
q19	833	793	887	793
q20	2006	2012	1873	1873
q21	5195	4711	4613	4613
q22	1096	1035	991	991
Total cold run time: 52763 ms
Total hot run time: 50701 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185901 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 81f6ec44815c8680f3bdb898b7bad4dba0547912, data reload: false

query1	1001	477	490	477
query2	6527	1889	1910	1889
query3	6796	218	216	216
query4	26156	23785	23184	23184
query5	4300	637	472	472
query6	287	186	189	186
query7	4617	480	266	266
query8	296	240	237	237
query9	8645	2556	2527	2527
query10	494	311	274	274
query11	15439	15204	14825	14825
query12	161	115	101	101
query13	1638	494	402	402
query14	8787	6142	6023	6023
query15	208	188	165	165
query16	7146	619	468	468
query17	1186	706	537	537
query18	1967	392	296	296
query19	182	174	149	149
query20	116	115	116	115
query21	214	118	98	98
query22	4172	4208	4138	4138
query23	33866	32941	33013	32941
query24	8388	2352	2340	2340
query25	532	440	387	387
query26	1243	265	146	146
query27	2770	503	311	311
query28	4385	2418	2388	2388
query29	753	540	408	408
query30	291	219	191	191
query31	926	881	761	761
query32	72	63	64	63
query33	555	369	319	319
query34	790	833	492	492
query35	831	822	769	769
query36	981	986	927	927
query37	123	101	79	79
query38	4097	4114	3953	3953
query39	1465	1402	1479	1402
query40	205	111	105	105
query41	54	53	52	52
query42	125	107	104	104
query43	501	505	475	475
query44	1305	790	795	790
query45	174	172	165	165
query46	841	1020	624	624
query47	1799	1885	1720	1720
query48	386	409	312	312
query49	788	493	417	417
query50	676	731	404	404
query51	4182	4181	4082	4082
query52	105	101	94	94
query53	220	246	174	174
query54	476	485	403	403
query55	85	80	83	80
query56	271	287	255	255
query57	1134	1146	1085	1085
query58	248	237	279	237
query59	2573	2818	2615	2615
query60	275	281	281	281
query61	135	129	130	129
query62	828	725	671	671
query63	212	184	185	184
query64	4361	1027	669	669
query65	4349	4223	4241	4223
query66	1130	401	309	309
query67	16103	15556	15371	15371
query68	5298	881	521	521
query69	481	296	255	255
query70	1190	1127	1087	1087
query71	389	296	264	264
query72	5646	4759	4827	4759
query73	641	615	340	340
query74	9009	9150	8939	8939
query75	3211	3228	2702	2702
query76	3171	1182	734	734
query77	490	361	296	296
query78	10027	10217	9363	9363
query79	971	879	572	572
query80	665	517	456	456
query81	476	259	223	223
query82	229	130	98	98
query83	178	175	165	165
query84	239	101	78	78
query85	742	349	324	324
query86	329	307	284	284
query87	4364	4543	4512	4512
query88	2830	2244	2318	2244
query89	384	310	282	282
query90	1780	211	212	211
query91	136	143	111	111
query92	64	59	56	56
query93	1122	1033	585	585
query94	608	426	305	305
query95	352	265	259	259
query96	478	548	276	276
query97	3144	3272	3116	3116
query98	230	207	207	207
query99	1342	1425	1325	1325
Total cold run time: 266160 ms
Total hot run time: 185901 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 81f6ec44815c8680f3bdb898b7bad4dba0547912, data reload: false

query1	0.04	0.03	0.04
query2	0.13	0.10	0.10
query3	0.24	0.20	0.19
query4	1.59	0.18	0.19
query5	0.59	0.58	0.59
query6	1.19	0.71	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.52	0.52
query10	0.59	0.58	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.59	0.59
query14	2.78	2.80	2.70
query15	0.92	0.86	0.83
query16	0.40	0.37	0.38
query17	1.08	1.04	1.01
query18	0.22	0.20	0.19
query19	1.94	1.95	1.88
query20	0.01	0.01	0.01
query21	15.36	0.89	0.56
query22	0.76	1.18	0.75
query23	14.79	1.38	0.66
query24	7.02	1.93	0.32
query25	0.28	0.08	0.11
query26	0.56	0.16	0.14
query27	0.04	0.05	0.06
query28	9.65	0.89	0.41
query29	12.54	3.93	3.28
query30	0.25	0.10	0.06
query31	2.83	0.60	0.39
query32	3.23	0.54	0.49
query33	2.96	2.98	3.06
query34	15.76	5.14	4.50
query35	4.52	4.56	4.55
query36	0.65	0.49	0.47
query37	0.08	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.14	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 104.93 s
Total hot run time: 30.72 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bobhan1
Copy link
Contributor Author

bobhan1 commented Mar 27, 2025

run cloud_p0

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 45.00% (9/20) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 50.89% (13623/26767)
Line Coverage 40.28% (118260/293603)
Region Coverage 38.95% (60079/154235)
Branch Coverage 33.87% (30222/89240)

@dataroaring dataroaring merged commit dc3afaa into apache:master Mar 27, 2025
28 of 30 checks passed
github-actions bot pushed a commit that referenced this pull request Mar 27, 2025
…ion (#49564)

`TabletTxnInfo` should be removed if version exists when publish
version. Otherwise, the related txn info will not be deleted and occupy
large memories
bobhan1 added a commit to bobhan1/doris that referenced this pull request Mar 27, 2025
…ion (apache#49564)

`TabletTxnInfo` should be removed if version exists when publish
version. Otherwise, the related txn info will not be deleted and occupy
large memories
bobhan1 added a commit to bobhan1/doris that referenced this pull request Mar 27, 2025
…ion (apache#49564)

`TabletTxnInfo` should be removed if version exists when publish
version. Otherwise, the related txn info will not be deleted and occupy
large memories
yiguolei pushed a commit that referenced this pull request Mar 28, 2025
@dataroaring dataroaring added the usercase Important user case type label label Mar 28, 2025
dataroaring pushed a commit that referenced this pull request Mar 28, 2025
…publish version #49564 (#49588)

Cherry-picked from #49564

Co-authored-by: bobhan1 <baohan@selectdb.com>
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…ion (apache#49564)

`TabletTxnInfo` should be removed if version exists when publish
version. Otherwise, the related txn info will not be deleted and occupy
large memories
deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.5-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants