Skip to content

Conversation

@liaoxin01
Copy link
Contributor

@liaoxin01 liaoxin01 commented Mar 17, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #48489

Problem Summary:

The _mem_table_ptr_lock will be locked in _reset_mem_table, so don't need to be acquired in MemTableWriter::write.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@liaoxin01
Copy link
Contributor Author

run buildall

@liaoxin01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32349 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bee98aa806e7b6a141c3dbe4daabe59143edaae5, data reload: false

------ Round 1 ----------------------------------
q1	24171	5244	5025	5025
q2	2039	309	186	186
q3	10381	1233	704	704
q4	10217	985	562	562
q5	7511	2356	2368	2356
q6	186	161	130	130
q7	906	741	604	604
q8	9308	1336	1051	1051
q9	5057	4603	4863	4603
q10	6808	2311	1906	1906
q11	485	275	261	261
q12	359	351	217	217
q13	17750	3686	3052	3052
q14	226	231	212	212
q15	539	498	491	491
q16	648	628	583	583
q17	561	851	353	353
q18	6989	6435	6360	6360
q19	1210	952	560	560
q20	317	336	203	203
q21	2754	2138	1925	1925
q22	1069	1010	1005	1005
Total cold run time: 109491 ms
Total hot run time: 32349 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5111	5110	5154	5110
q2	244	336	234	234
q3	2163	2686	2294	2294
q4	1471	1812	1398	1398
q5	4277	4122	4452	4122
q6	217	170	126	126
q7	2023	1943	1769	1769
q8	2622	2494	2507	2494
q9	7195	7152	7284	7152
q10	2991	3251	2783	2783
q11	580	509	512	509
q12	688	739	601	601
q13	3526	3925	3265	3265
q14	270	298	276	276
q15	542	514	521	514
q16	645	714	658	658
q17	1130	1523	1371	1371
q18	7768	7598	7475	7475
q19	792	759	900	759
q20	1997	2056	1886	1886
q21	5368	5041	4672	4672
q22	1085	1043	1004	1004
Total cold run time: 52705 ms
Total hot run time: 50472 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185172 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bee98aa806e7b6a141c3dbe4daabe59143edaae5, data reload: false

query1	995	489	471	471
query2	6558	1931	1902	1902
query3	6789	220	217	217
query4	26387	23669	23394	23394
query5	4314	688	469	469
query6	290	189	172	172
query7	4595	496	295	295
query8	304	232	222	222
query9	8585	2527	2527	2527
query10	451	299	243	243
query11	15624	15152	14842	14842
query12	161	109	103	103
query13	1649	518	394	394
query14	8839	6742	6125	6125
query15	211	187	164	164
query16	7120	629	487	487
query17	909	707	558	558
query18	1954	388	291	291
query19	184	178	155	155
query20	113	136	120	120
query21	208	120	101	101
query22	4103	4327	4273	4273
query23	33819	33037	32964	32964
query24	7918	2372	2371	2371
query25	519	445	387	387
query26	1240	274	152	152
query27	2502	472	331	331
query28	4226	2402	2369	2369
query29	751	561	425	425
query30	277	213	188	188
query31	928	843	810	810
query32	75	66	63	63
query33	555	352	313	313
query34	790	844	511	511
query35	807	818	766	766
query36	960	986	893	893
query37	121	100	75	75
query38	4236	4157	4128	4128
query39	1450	1370	1408	1370
query40	208	118	102	102
query41	55	50	51	50
query42	115	103	103	103
query43	479	493	488	488
query44	1277	783	790	783
query45	174	167	160	160
query46	836	1037	611	611
query47	1760	1800	1720	1720
query48	367	425	300	300
query49	783	503	428	428
query50	681	735	407	407
query51	4122	4169	4074	4074
query52	108	112	112	112
query53	242	254	193	193
query54	495	485	418	418
query55	83	81	83	81
query56	256	252	270	252
query57	1140	1157	1072	1072
query58	267	242	242	242
query59	2428	2677	2353	2353
query60	274	275	254	254
query61	119	123	122	122
query62	792	710	663	663
query63	228	193	183	183
query64	4307	1012	713	713
query65	4389	4332	4368	4332
query66	1135	414	299	299
query67	15815	15513	15250	15250
query68	5249	887	512	512
query69	475	335	268	268
query70	1230	1084	1105	1084
query71	373	296	273	273
query72	5565	3553	3741	3553
query73	735	718	347	347
query74	9011	9067	8723	8723
query75	3165	3161	2721	2721
query76	3161	1180	751	751
query77	511	378	295	295
query78	9960	10316	9346	9346
query79	1127	856	597	597
query80	1538	563	498	498
query81	548	261	221	221
query82	414	130	100	100
query83	286	263	159	159
query84	296	89	74	74
query85	776	357	311	311
query86	374	304	278	278
query87	4365	4450	4392	4392
query88	2972	2251	2274	2251
query89	383	320	281	281
query90	1740	227	225	225
query91	139	142	109	109
query92	65	62	57	57
query93	1192	1072	584	584
query94	671	408	305	305
query95	367	279	261	261
query96	482	561	366	366
query97	3312	3354	3282	3282
query98	229	202	198	198
query99	1304	1377	1344	1344
Total cold run time: 266467 ms
Total hot run time: 185172 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.48 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bee98aa806e7b6a141c3dbe4daabe59143edaae5, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.11	0.10
query3	0.25	0.18	0.19
query4	1.58	0.19	0.19
query5	0.59	0.58	0.58
query6	1.21	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.52	0.51
query10	0.60	0.59	0.57
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.60	0.61
query14	2.78	2.68	2.84
query15	0.92	0.83	0.85
query16	0.36	0.38	0.38
query17	1.02	1.00	1.05
query18	0.21	0.20	0.21
query19	1.88	1.87	1.87
query20	0.02	0.01	0.02
query21	15.35	0.89	0.56
query22	0.75	1.08	0.71
query23	15.02	1.38	0.60
query24	6.89	1.75	0.92
query25	0.52	0.19	0.22
query26	0.60	0.16	0.14
query27	0.05	0.05	0.05
query28	9.74	0.91	0.43
query29	12.53	3.96	3.28
query30	0.25	0.09	0.06
query31	2.83	0.58	0.39
query32	3.23	0.55	0.47
query33	3.02	3.04	3.04
query34	15.87	5.15	4.55
query35	4.60	4.60	4.60
query36	0.69	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.57 s
Total hot run time: 31.48 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 70.00% (14/20) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 48.86% (13088/26784)
Line Coverage 38.41% (112709/293466)
Region Coverage 37.24% (57360/154034)
Branch Coverage 32.31% (28806/89166)

@liaoxin01
Copy link
Contributor Author

run buildall

@liaoxin01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32112 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aae939840649ff8b330e1da8e02cb4b1a833d92c, data reload: false

------ Round 1 ----------------------------------
q1	24205	5078	5040	5040
q2	2044	296	178	178
q3	10581	1240	703	703
q4	10288	991	535	535
q5	8539	2433	2367	2367
q6	184	162	130	130
q7	910	765	631	631
q8	9311	1279	1039	1039
q9	4848	4558	4650	4558
q10	6866	2313	1896	1896
q11	494	266	253	253
q12	343	353	219	219
q13	17778	3678	3089	3089
q14	225	230	205	205
q15	539	498	470	470
q16	637	604	597	597
q17	580	847	342	342
q18	6958	6381	6266	6266
q19	1890	966	552	552
q20	326	326	192	192
q21	2767	2110	1886	1886
q22	1064	1044	964	964
Total cold run time: 111377 ms
Total hot run time: 32112 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5195	5076	5081	5076
q2	233	334	225	225
q3	2151	2644	2281	2281
q4	1401	1830	1361	1361
q5	4245	4122	4377	4122
q6	211	169	130	130
q7	1993	1890	1800	1800
q8	2615	2713	2525	2525
q9	7318	7219	7287	7219
q10	3032	3129	2670	2670
q11	573	504	491	491
q12	726	796	594	594
q13	3528	3966	3233	3233
q14	289	327	264	264
q15	540	488	484	484
q16	649	691	648	648
q17	1124	1574	1361	1361
q18	7967	7648	7614	7614
q19	808	782	823	782
q20	1994	2042	2016	2016
q21	5307	4883	4688	4688
q22	1136	1106	1031	1031
Total cold run time: 53035 ms
Total hot run time: 50615 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192021 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aae939840649ff8b330e1da8e02cb4b1a833d92c, data reload: false

query1	1421	1070	1046	1046
query2	6418	1851	1842	1842
query3	11131	4613	4514	4514
query4	55178	26175	22955	22955
query5	5301	571	487	487
query6	368	219	190	190
query7	5010	489	297	297
query8	335	251	250	250
query9	6320	2490	2479	2479
query10	422	298	254	254
query11	15262	15053	14793	14793
query12	179	110	106	106
query13	1138	523	410	410
query14	10108	6765	6981	6765
query15	240	201	170	170
query16	7056	601	514	514
query17	1089	724	559	559
query18	1518	406	331	331
query19	220	207	158	158
query20	133	139	121	121
query21	212	120	109	109
query22	4602	4573	4477	4477
query23	34093	33323	33317	33317
query24	5765	2464	2479	2464
query25	453	450	400	400
query26	672	276	166	166
query27	1741	507	357	357
query28	2727	2422	2408	2408
query29	574	569	427	427
query30	282	220	202	202
query31	877	873	844	844
query32	74	71	62	62
query33	461	373	295	295
query34	762	869	514	514
query35	816	851	770	770
query36	950	1004	915	915
query37	122	105	80	80
query38	4267	4223	4133	4133
query39	1494	1480	1416	1416
query40	203	120	104	104
query41	52	52	53	52
query42	128	106	103	103
query43	489	505	490	490
query44	1292	794	798	794
query45	184	180	174	174
query46	863	1034	660	660
query47	1876	1868	1765	1765
query48	401	418	317	317
query49	724	516	428	428
query50	713	754	426	426
query51	4242	4342	4222	4222
query52	113	111	102	102
query53	244	271	195	195
query54	501	515	433	433
query55	87	82	90	82
query56	291	279	260	260
query57	1174	1157	1120	1120
query58	242	238	246	238
query59	2618	2719	2708	2708
query60	304	284	295	284
query61	148	149	151	149
query62	753	736	687	687
query63	236	199	201	199
query64	1537	1118	684	684
query65	4600	4331	4333	4331
query66	744	394	290	290
query67	15829	15524	15427	15427
query68	7534	816	511	511
query69	550	298	268	268
query70	1198	1122	1087	1087
query71	504	297	292	292
query72	5592	3623	3724	3623
query73	1361	761	358	358
query74	9057	8967	8744	8744
query75	3689	3143	2702	2702
query76	4256	1165	727	727
query77	627	383	285	285
query78	10035	10243	9301	9301
query79	2122	815	582	582
query80	854	538	456	456
query81	483	258	220	220
query82	661	124	96	96
query83	277	176	155	155
query84	280	99	79	79
query85	782	349	306	306
query86	356	311	374	311
query87	4489	4469	4454	4454
query88	2908	2247	2235	2235
query89	410	310	277	277
query90	1952	218	221	218
query91	157	145	109	109
query92	71	59	55	55
query93	1208	1059	590	590
query94	699	415	293	293
query95	349	274	268	268
query96	494	552	279	279
query97	3395	3461	3338	3338
query98	221	220	208	208
query99	1413	1362	1273	1273
Total cold run time: 298363 ms
Total hot run time: 192021 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aae939840649ff8b330e1da8e02cb4b1a833d92c, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.11	0.10
query3	0.25	0.19	0.19
query4	1.59	0.20	0.11
query5	0.55	0.54	0.55
query6	1.18	0.72	0.70
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.60	0.51	0.54
query10	0.62	0.61	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.62	0.60
query14	2.67	2.70	2.72
query15	0.90	0.85	0.85
query16	0.38	0.39	0.38
query17	1.03	1.04	0.99
query18	0.22	0.19	0.20
query19	1.97	1.95	1.87
query20	0.02	0.01	0.01
query21	15.35	0.87	0.53
query22	0.77	1.19	0.59
query23	14.98	1.38	0.64
query24	6.89	1.82	1.11
query25	0.51	0.19	0.13
query26	0.50	0.16	0.13
query27	0.05	0.05	0.06
query28	9.72	0.79	0.42
query29	12.53	3.97	3.32
query30	0.25	0.10	0.06
query31	2.82	0.57	0.39
query32	3.23	0.54	0.46
query33	2.98	3.05	3.02
query34	15.74	5.13	4.50
query35	4.55	4.54	4.53
query36	0.67	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.14 s
Total hot run time: 31.27 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 70.00% (14/20) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 48.88% (13093/26784)
Line Coverage 38.45% (112866/293539)
Region Coverage 37.26% (57400/154045)
Branch Coverage 32.33% (28832/89174)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 19, 2025
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 327e87c into apache:master Mar 19, 2025
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Mar 19, 2025
Related PR: #48489 

Problem Summary:

The _mem_table_ptr_lock will be locked in _reset_mem_table, so don't
need to be acquired in MemTableWriter::write.
dataroaring pushed a commit that referenced this pull request Mar 20, 2025
… (#49230)

Cherry-picked from #49170

Co-authored-by: Xin Liao <liaoxin@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
Related PR: apache#48489 

Problem Summary:

The _mem_table_ptr_lock will be locked in _reset_mem_table, so don't
need to be acquired in MemTableWriter::write.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.5-merged p0_b reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants