Skip to content

Conversation

@freemandealer
Copy link
Contributor

keep _storage last so it will deconstruct first
otherwise, load_cache_info_into_memory might crash coz it will use other members of BlockFileCache
so join this async load thread first

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

keep _storage last so it will deconstruct first
otherwise, load_cache_info_into_memory might crash
coz it will use other members of BlockFileCache
so join this async load thread first

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@Thearas
Copy link
Contributor

Thearas commented Jun 12, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 12, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@freemandealer
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33872 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b2db666e25e65725d0ac65027150ca1776dd022, data reload: false

------ Round 1 ----------------------------------
q1	17605	5236	5093	5093
q2	1941	283	170	170
q3	10537	1254	771	771
q4	10215	1002	504	504
q5	7566	2334	2375	2334
q6	178	162	131	131
q7	915	769	638	638
q8	9318	1280	1082	1082
q9	6844	5082	5064	5064
q10	6857	2309	1891	1891
q11	488	280	286	280
q12	343	350	213	213
q13	17777	3637	3081	3081
q14	237	226	213	213
q15	578	482	491	482
q16	432	450	374	374
q17	598	854	357	357
q18	7622	7252	7155	7155
q19	2218	1032	545	545
q20	330	332	218	218
q21	3842	3149	2320	2320
q22	1008	1011	956	956
Total cold run time: 107449 ms
Total hot run time: 33872 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5236	5057	5065	5057
q2	239	318	219	219
q3	2168	2655	2323	2323
q4	1413	1822	1352	1352
q5	4250	4199	4348	4199
q6	227	182	134	134
q7	2042	1986	1849	1849
q8	2630	2639	2547	2547
q9	7131	7160	7202	7160
q10	3010	3121	2679	2679
q11	576	522	508	508
q12	674	792	641	641
q13	3507	3935	3252	3252
q14	293	314	270	270
q15	518	481	467	467
q16	449	490	455	455
q17	1159	1594	1346	1346
q18	7867	7557	7666	7557
q19	823	791	932	791
q20	1997	2060	1913	1913
q21	4983	4348	4412	4348
q22	1097	1064	1019	1019
Total cold run time: 52289 ms
Total hot run time: 50086 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193475 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b2db666e25e65725d0ac65027150ca1776dd022, data reload: false

query1	1393	1013	1006	1006
query2	6275	1976	1938	1938
query3	11079	4424	4546	4424
query4	54562	24467	23386	23386
query5	5205	513	470	470
query6	338	211	209	209
query7	4896	512	301	301
query8	290	234	235	234
query9	5663	2661	2673	2661
query10	458	351	288	288
query11	15157	15128	14920	14920
query12	157	116	110	110
query13	1037	523	410	410
query14	10254	6401	6479	6401
query15	212	203	183	183
query16	7117	674	513	513
query17	1092	771	610	610
query18	1574	432	326	326
query19	202	209	192	192
query20	182	122	121	121
query21	215	125	107	107
query22	4404	4384	4258	4258
query23	34278	33575	33477	33477
query24	6518	2455	2435	2435
query25	472	471	397	397
query26	705	281	160	160
query27	2215	524	359	359
query28	2947	2212	2183	2183
query29	576	594	426	426
query30	275	222	194	194
query31	923	862	808	808
query32	71	62	65	62
query33	452	364	331	331
query34	796	885	553	553
query35	791	843	758	758
query36	946	1000	929	929
query37	113	102	79	79
query38	4336	4360	4365	4360
query39	1554	1472	1475	1472
query40	223	125	112	112
query41	63	61	59	59
query42	136	134	110	110
query43	521	540	510	510
query44	1377	847	851	847
query45	189	184	178	178
query46	847	1056	668	668
query47	1821	1868	1757	1757
query48	393	437	328	328
query49	665	491	411	411
query50	680	714	423	423
query51	4327	4266	4271	4266
query52	108	108	101	101
query53	229	267	189	189
query54	592	590	527	527
query55	90	85	88	85
query56	323	308	310	308
query57	1219	1225	1136	1136
query58	280	285	253	253
query59	2903	2915	2877	2877
query60	332	315	311	311
query61	126	121	128	121
query62	733	748	692	692
query63	231	191	191	191
query64	1712	1020	676	676
query65	4209	4150	4200	4150
query66	705	393	321	321
query67	15962	15589	15235	15235
query68	7082	892	536	536
query69	544	312	275	275
query70	1195	1155	1138	1138
query71	514	312	300	300
query72	5926	4846	4784	4784
query73	1453	635	352	352
query74	9035	9100	9014	9014
query75	3863	3193	2707	2707
query76	4081	1214	751	751
query77	625	375	292	292
query78	10285	9981	9234	9234
query79	5758	809	572	572
query80	854	521	455	455
query81	479	270	219	219
query82	614	121	96	96
query83	379	252	233	233
query84	294	104	91	91
query85	794	356	309	309
query86	409	307	300	300
query87	4454	4419	4335	4335
query88	3696	2305	2274	2274
query89	448	321	293	293
query90	1804	217	227	217
query91	149	138	110	110
query92	72	66	64	64
query93	3448	959	583	583
query94	661	394	309	309
query95	376	300	302	300
query96	502	572	285	285
query97	2684	2750	2676	2676
query98	241	217	247	217
query99	1468	1399	1261	1261
Total cold run time: 304913 ms
Total hot run time: 193475 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.1 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1b2db666e25e65725d0ac65027150ca1776dd022, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.25	0.06	0.07
query4	1.61	0.10	0.10
query5	0.44	0.42	0.43
query6	1.19	0.64	0.67
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.57	0.53	0.51
query10	0.58	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.13
query13	0.62	0.60	0.60
query14	0.80	0.82	0.82
query15	0.87	0.86	0.87
query16	0.38	0.37	0.39
query17	1.07	1.06	1.08
query18	0.24	0.22	0.22
query19	1.94	1.84	1.89
query20	0.02	0.01	0.01
query21	15.39	0.91	0.55
query22	0.74	1.11	0.94
query23	14.71	1.40	0.68
query24	6.92	2.07	0.69
query25	0.52	0.15	0.20
query26	0.60	0.15	0.13
query27	0.05	0.06	0.04
query28	8.98	0.84	0.44
query29	12.54	3.91	3.33
query30	0.26	0.10	0.06
query31	2.80	0.59	0.39
query32	3.23	0.55	0.47
query33	3.08	3.13	3.10
query34	15.85	5.10	4.45
query35	4.53	4.51	4.49
query36	0.65	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 102.4 s
Total hot run time: 29.1 s

@freemandealer
Copy link
Contributor Author

run vault_p0

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.38% (15048/26692)
Line Coverage 45.14% (134566/298132)
Region Coverage 44.25% (67678/152932)
Branch Coverage 38.82% (34714/89420)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.83% (20987/26288)
Line Coverage 72.79% (216984/298090)
Region Coverage 71.07% (127918/179977)
Branch Coverage 64.79% (66247/102254)

@dataroaring dataroaring merged commit 0c38e5e into apache:master Jun 18, 2025
28 of 31 checks passed
github-actions bot pushed a commit that referenced this pull request Jun 18, 2025
keep _storage last so it will deconstruct first
otherwise, load_cache_info_into_memory might crash coz it will use other
members of BlockFileCache
so join this async load thread first

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
dataroaring pushed a commit that referenced this pull request Jun 19, 2025
…1684 (#51904)

Cherry-picked from #51684

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
github-actions bot pushed a commit that referenced this pull request Jun 19, 2025
keep _storage last so it will deconstruct first
otherwise, load_cache_info_into_memory might crash coz it will use other
members of BlockFileCache
so join this async load thread first

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
morrySnow pushed a commit that referenced this pull request Jun 20, 2025
…1684 (#51957)

Cherry-picked from #51684

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. cloud dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants