[feature](cloud) Read peer be cache when balance in same cluster #56384

deardeng · 2025-09-24T06:54:25Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-09-24T06:54:30Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

deardeng · 2025-09-24T06:54:39Z

run buildall

hello-stephen · 2025-09-24T07:18:40Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.75% (1567/1871)
Line Coverage	67.88% (27895/41096)
Region Coverage	68.23% (13726/20117)
Branch Coverage	58.51% (7321/12512)

hello-stephen · 2025-09-24T08:04:20Z

FE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-09-24T08:56:13Z

ClickBench: Total hot run time: 30.78 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1d683ca5a7ee0fe76a2c7738439de2e28791dde9, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.26	0.09	0.09
query4	1.61	0.12	0.12
query5	0.29	0.27	0.25
query6	1.18	0.65	0.66
query7	0.04	0.03	0.03
query8	0.07	0.05	0.05
query9	0.63	0.53	0.53
query10	0.61	0.59	0.58
query11	0.17	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.64	0.61
query14	1.02	1.04	1.04
query15	0.87	0.86	0.87
query16	0.40	0.39	0.39
query17	1.06	1.05	1.08
query18	0.21	0.21	0.21
query19	2.24	2.01	2.11
query20	0.02	0.01	0.02
query21	15.57	0.93	0.59
query22	0.76	1.28	0.71
query23	14.78	1.37	0.63
query24	6.95	2.18	0.78
query25	0.49	0.31	0.09
query26	0.54	0.16	0.14
query27	0.06	0.05	0.06
query28	9.54	1.35	0.93
query29	12.60	3.86	3.34
query30	0.31	0.17	0.14
query31	2.84	0.59	0.38
query32	3.24	0.56	0.48
query33	3.20	3.06	3.10
query34	16.30	5.48	4.83
query35	4.96	4.93	4.91
query36	0.70	0.52	0.50
query37	0.11	0.08	0.07
query38	0.07	0.05	0.04
query39	0.04	0.04	0.04
query40	0.19	0.17	0.15
query41	0.10	0.03	0.04
query42	0.05	0.04	0.03
query43	0.05	0.04	0.04
Total cold run time: 105.06 s
Total hot run time: 30.78 s

hello-stephen · 2025-09-24T10:05:57Z

BE UT Coverage Report

Increment line coverage 24.40% (91/373) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.41% (17660/33696)
Line Coverage	37.65% (160471/426210)
Region Coverage	32.15% (122237/380187)
Branch Coverage	33.51% (53574/159872)

hello-stephen · 2025-09-24T10:50:44Z

BE Regression && UT Coverage Report

Increment line coverage 26.77% (125/467) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.20% (23431/32909)
Line Coverage	57.58% (244821/425183)
Region Coverage	52.92% (203907/385317)
Branch Coverage	54.57% (87603/160547)

hello-stephen · 2025-09-24T11:03:05Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

gavinchou

path to tablet_id needs an UT

deardeng · 2025-09-30T11:06:27Z

run buildall

hello-stephen · 2025-09-30T11:34:07Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.35% (1632/1958)
Line Coverage	67.86% (28834/42488)
Region Coverage	68.05% (14206/20876)
Branch Coverage	58.41% (7570/12960)

hello-stephen · 2025-09-30T12:21:21Z

FE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-09-30T13:09:03Z

TPC-DS: Total hot run time: 190362 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26c718796e1430de60a803ef6ede929a04170791, data reload: false

query1	1105	441	405	405
query2	6563	1695	1697	1695
query3	6758	221	221	221
query4	26923	23783	23571	23571
query5	5177	636	492	492
query6	352	248	245	245
query7	4665	508	298	298
query8	323	272	253	253
query9	8685	2591	2576	2576
query10	551	348	290	290
query11	15541	15279	14757	14757
query12	180	124	117	117
query13	1687	559	423	423
query14	11546	9228	9255	9228
query15	226	192	178	178
query16	8507	724	566	566
query17	1633	831	661	661
query18	2070	453	366	366
query19	248	220	187	187
query20	138	135	127	127
query21	233	134	127	127
query22	4736	4778	4690	4690
query23	34885	34014	33713	33713
query24	8476	2553	2550	2550
query25	614	538	551	538
query26	1265	276	165	165
query27	3032	525	398	398
query28	4383	2158	2130	2130
query29	783	602	479	479
query30	302	219	193	193
query31	912	830	750	750
query32	79	68	66	66
query33	596	376	321	321
query34	793	837	534	534
query35	793	832	728	728
query36	989	1000	931	931
query37	125	111	90	90
query38	3506	3533	3462	3462
query39	1470	1408	1418	1408
query40	220	130	122	122
query41	68	61	62	61
query42	119	118	116	116
query43	489	491	474	474
query44	1359	834	833	833
query45	194	180	178	178
query46	847	992	652	652
query47	1839	1812	1744	1744
query48	387	436	309	309
query49	775	523	417	417
query50	658	692	407	407
query51	3933	4004	3903	3903
query52	108	109	100	100
query53	239	274	219	219
query54	600	603	525	525
query55	90	89	84	84
query56	335	318	308	308
query57	1172	1228	1115	1115
query58	282	284	278	278
query59	2625	2554	2536	2536
query60	342	346	340	340
query61	163	151	154	151
query62	792	717	660	660
query63	225	203	197	197
query64	4413	1160	844	844
query65	4039	3937	3971	3937
query66	1058	431	338	338
query67	15609	15292	15087	15087
query68	7471	946	602	602
query69	471	317	291	291
query70	1440	1322	1322	1322
query71	455	348	320	320
query72	5793	4928	4840	4840
query73	639	574	366	366
query74	8854	9146	8917	8917
query75	3424	3319	2844	2844
query76	3303	1163	755	755
query77	738	413	327	327
query78	9640	9766	8871	8871
query79	2404	798	580	580
query80	694	649	495	495
query81	524	262	231	231
query82	224	162	130	130
query83	257	261	245	245
query84	256	116	94	94
query85	858	463	430	430
query86	368	328	312	312
query87	3734	3723	3649	3649
query88	3698	2238	2230	2230
query89	388	326	295	295
query90	2049	222	212	212
query91	166	163	142	142
query92	85	65	139	65
query93	2451	977	645	645
query94	675	444	324	324
query95	400	324	313	313
query96	489	568	274	274
query97	2915	2967	2889	2889
query98	245	218	214	214
query99	1327	1395	1274	1274
Total cold run time: 280265 ms
Total hot run time: 190362 ms

doris-robot · 2025-09-30T13:13:19Z

ClickBench: Total hot run time: 0.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26c718796e1430de60a803ef6ede929a04170791, data reload: false

query1	0.05	0.01	0.01
query2	0.13	0.01	0.00
query3	0.31	0.01	0.01
query4	1.72	0.01	0.01
query5	0.28	0.00	0.01
query6	1.68	0.01	0.01
query7	0.05	0.00	0.01
query8	0.08	0.00	0.00
query9	0.68	0.00	0.01
query10	0.61	0.00	0.00
query11	0.27	0.00	0.00
query12	0.26	0.00	0.00
query13	0.65	0.00	0.00
query14	1.04	0.01	0.01
query15	0.97	0.00	0.00
query16	0.40	0.00	0.00
query17	1.08	0.00	0.01
query18	0.23	0.00	0.01
query19	1.99	0.00	0.01
query20	0.02	0.00	0.01
query21	15.36	0.00	0.00
query22	5.73	0.00	0.00
query23	15.55	0.00	0.01
query24	1.79	0.01	0.00
query25	0.17	0.00	0.00
query26	0.20	0.01	0.00
query27	0.11	0.00	0.00
query28	1.78	0.01	0.00
query29	12.75	0.00	0.00
query30	0.35	0.00	0.00
query31	2.62	0.00	0.00
query32	6.34	0.00	0.00
query33	4.32	0.01	0.00
query34	8.59	0.00	0.00
query35	7.47	0.00	0.01
query36	0.66	0.01	0.00
query37	0.22	0.00	0.00
query38	0.21	0.00	0.00
query39	0.06	0.00	0.00
query40	0.22	0.00	0.00
query41	0.13	0.00	0.00
query42	0.09	0.00	0.00
query43	0.08	0.00	0.00
Total cold run time: 97.3 s
Total hot run time: 0.05 s

doris-robot · 2025-09-30T14:18:09Z

BE UT Coverage Report

Increment line coverage 28.82% (117/406) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.43% (17671/33703)
Line Coverage	37.63% (160492/426458)
Region Coverage	32.13% (122551/381407)
Branch Coverage	33.51% (53687/160221)

hello-stephen · 2025-09-30T15:16:33Z

BE Regression && UT Coverage Report

Increment line coverage 31.76% (161/507) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.08% (23378/32889)
Line Coverage	57.51% (244631/425342)
Region Coverage	52.72% (203425/385882)
Branch Coverage	54.47% (87600/160828)

hello-stephen · 2025-09-30T15:28:23Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

deardeng · 2025-10-24T12:03:31Z

path to tablet_id needs an UT

this pr has , ut in ParseTabletIdFromPath

deardeng · 2025-10-24T12:03:38Z

run buildall

deardeng · 2025-10-24T12:14:44Z

run buildall

doris-robot · 2025-10-30T16:28:46Z

BE UT Coverage Report

Increment line coverage 24.77% (109/440) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.69% (18044/34246)
Line Coverage	37.94% (163809/431712)
Region Coverage	32.30% (124942/386855)
Branch Coverage	33.68% (54610/162135)

hello-stephen · 2025-10-30T17:40:58Z

BE Regression && UT Coverage Report

Increment line coverage 26.42% (144/545) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.45% (24045/33655)
Line Coverage	57.85% (250126/432401)
Region Coverage	52.84% (207348/392409)
Branch Coverage	54.64% (89195/163236)

hello-stephen · 2025-10-30T17:52:31Z

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

…che#56384)

Related PR： #56384 modify the `enable_cache_read_from_peer` configuration from false to true.

…che#56384)

…cluster #56384 (#57672) cherry pick from #56384

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

…che#56384)

…cluster #56384 (#58089) cherry pick from #56384

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

deardeng requested review from dataroaring and gavinchou as code owners September 24, 2025 06:54

gavinchou reviewed Sep 30, 2025

View reviewed changes

deardeng force-pushed the balance-peer-read-cache branch from 8ddd953 to 26c7187 Compare September 30, 2025 11:06

deardeng force-pushed the balance-peer-read-cache branch from 26c7187 to 456a64c Compare October 20, 2025 10:55

deardeng added 5 commits October 24, 2025 20:14

[feature-WIP](cloud) Read file cache from peer be when cache miss

ec0ccb7

[feature](cloud) Read peer be cache when balance in same cluster

4acf54f

fix review

7fd19a0

use timer

1892e69

add compaction case

d389567

deardeng force-pushed the balance-peer-read-cache branch from 65d4a56 to d389567 Compare October 24, 2025 12:14

hello-stephen approved these changes Oct 31, 2025

View reviewed changes

hello-stephen merged commit ee6480f into apache:master Oct 31, 2025
32 of 40 checks passed

dwdwqfwe pushed a commit to dwdwqfwe/doris that referenced this pull request Oct 31, 2025

[feature](cloud) Read peer be cache when balance in same cluster (apa…

11c3fe3

…che#56384)

deardeng mentioned this pull request Nov 1, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio #57587

Merged

hello-stephen pushed a commit that referenced this pull request Nov 2, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio (#57587)

77380ff

Related PR： #56384 modify the `enable_cache_read_from_peer` configuration from false to true.

morrySnow added p0_b dev/3.1.x labels Nov 4, 2025

github-actions bot added the dev/3.1.x-conflict label Nov 4, 2025

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Nov 4, 2025

[feature](cloud) Read peer be cache when balance in same cluster (apa…

407f489

…che#56384)

deardeng mentioned this pull request Nov 4, 2025

branch-3.1: [feature](cloud) Read peer be cache when balance in same cluster #56384 #57672

Merged

morrySnow pushed a commit that referenced this pull request Nov 7, 2025

branch-3.1: [feature](cloud) Read peer be cache when balance in same …

53758f8

…cluster #56384 (#57672) cherry pick from #56384

morrySnow added dev/3.1.3-merged and removed dev/3.1.x dev/3.1.x-conflict labels Nov 7, 2025

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Nov 7, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio (apache#57587

d1ae6a8

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

deardeng mentioned this pull request Nov 7, 2025

branch-3.1: [fix](cloud)Fix read from peer use thread pool not asyncio #57587 #57807

Merged

gavinchou added the dev/4.0.x label Nov 15, 2025

github-actions bot added the dev/4.0.x-conflict label Nov 15, 2025

deardeng mentioned this pull request Nov 17, 2025

branch-4.0: [feature](cloud) Read peer be cache when balance in same cluster #56384 #58089

Merged

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Nov 17, 2025

[feature](cloud) Read peer be cache when balance in same cluster (apa…

bd543b4

…che#56384)

yiguolei pushed a commit that referenced this pull request Nov 18, 2025

branch-4.0: [feature](cloud) Read peer be cache when balance in same …

f05a582

…cluster #56384 (#58089) cherry pick from #56384

yiguolei added dev/4.0.2-merged and removed dev/4.0.x dev/4.0.x-conflict labels Nov 18, 2025

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Dec 12, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio (apache#57587

ba08ca3

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Dec 12, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio (apache#57587

6166210

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

deardeng added a commit to deardeng/incubator-doris that referenced this pull request Dec 18, 2025

[fix](cloud)Fix read from peer use thread pool not asyncio (apache#57587

1d2a0da

) Related PR： apache#56384 modify the `enable_cache_read_from_peer` configuration from false to true.

[feature](cloud) Read peer be cache when balance in same cluster #56384

[feature](cloud) Read peer be cache when balance in same cluster #56384

Uh oh!

Conversation

deardeng commented Sep 24, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 24, 2025

Uh oh!

deardeng commented Sep 24, 2025

Uh oh!

hello-stephen commented Sep 24, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Sep 24, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Sep 24, 2025

Uh oh!

hello-stephen commented Sep 24, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 24, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Sep 24, 2025

FE Regression Coverage Report

Uh oh!

gavinchou left a comment

Choose a reason for hiding this comment

Uh oh!

deardeng commented Sep 30, 2025

Uh oh!

hello-stephen commented Sep 30, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Sep 30, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Sep 30, 2025

Uh oh!

doris-robot commented Sep 30, 2025

Uh oh!

doris-robot commented Sep 30, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 30, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Sep 30, 2025

FE Regression Coverage Report

Uh oh!

deardeng commented Oct 24, 2025

Uh oh!

deardeng commented Oct 24, 2025

Uh oh!

deardeng commented Oct 24, 2025

Uh oh!

doris-robot commented Oct 30, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 30, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Oct 30, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants