Skip to content

Conversation

@sollhui
Copy link
Contributor

@sollhui sollhui commented Jun 2, 2025

pick #51240

Accelerate query information_schema.tables by scanning table information from FE master when sending query request from follower node. It can reduces rpc call from follower to meta service to improve query performance for master getting version is memory operation, but it will slightly increase the pressure on the FE master.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

… node in cloud mode (apache#51240)

Accelerate query information_schema.tables by scanning table information
from FE master when sending query request from follower node. It can
reduces rpc call from follower to meta service to improve query
performance for master getting version is memory operation, but it will
slightly increase the pressure on the FE master.
@sollhui sollhui requested a review from dataroaring as a code owner June 2, 2025 13:04
@Thearas
Copy link
Contributor

Thearas commented Jun 2, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@sollhui
Copy link
Contributor Author

sollhui commented Jun 2, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41576 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41cb453f6d363366a08c76fa47926a6e0cee5a53, data reload: false

------ Round 1 ----------------------------------
q1	17570	7552	7461	7461
q2	2066	168	158	158
q3	10790	1095	1258	1095
q4	10526	808	802	802
q5	7803	2985	2958	2958
q6	238	135	130	130
q7	1062	615	607	607
q8	9355	2006	2061	2006
q9	7074	6769	6748	6748
q10	7070	2290	2324	2290
q11	470	267	259	259
q12	430	209	210	209
q13	17779	2968	2958	2958
q14	250	201	201	201
q15	504	470	455	455
q16	553	375	367	367
q17	998	536	591	536
q18	7233	6611	6674	6611
q19	1391	1246	1181	1181
q20	496	197	197	197
q21	4219	3443	3388	3388
q22	1087	959	989	959
Total cold run time: 108964 ms
Total hot run time: 41576 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7473	7448	7322	7322
q2	352	229	228	228
q3	3173	3042	3032	3032
q4	2137	1890	1845	1845
q5	5843	5934	6089	5934
q6	233	127	126	126
q7	2350	1818	1787	1787
q8	3714	3880	3727	3727
q9	9072	9083	8919	8919
q10	3693	3660	3651	3651
q11	601	506	503	503
q12	819	632	633	632
q13	11315	3118	3186	3118
q14	307	269	280	269
q15	511	453	457	453
q16	507	472	458	458
q17	1994	1725	1697	1697
q18	8097	7663	7671	7663
q19	1804	1624	1683	1624
q20	2147	1840	1876	1840
q21	5778	5512	5361	5361
q22	1146	1019	1029	1019
Total cold run time: 73066 ms
Total hot run time: 61208 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195606 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41cb453f6d363366a08c76fa47926a6e0cee5a53, data reload: false

query1	1296	909	913	909
query2	6206	1921	1921	1921
query3	10833	4210	4268	4210
query4	65923	28081	23319	23319
query5	5056	450	440	440
query6	413	187	197	187
query7	5479	313	309	309
query8	314	237	219	219
query9	8775	2572	2565	2565
query10	444	287	266	266
query11	17287	15567	15753	15567
query12	156	104	102	102
query13	1443	441	430	430
query14	10220	6685	6776	6685
query15	210	188	176	176
query16	7246	474	481	474
query17	1073	593	611	593
query18	1812	329	323	323
query19	211	160	170	160
query20	123	107	111	107
query21	208	107	103	103
query22	4666	4304	4359	4304
query23	34492	34131	33831	33831
query24	6686	2914	2947	2914
query25	540	442	414	414
query26	663	174	175	174
query27	2153	369	342	342
query28	4344	2159	2125	2125
query29	684	423	461	423
query30	240	150	151	150
query31	974	801	844	801
query32	67	55	58	55
query33	412	281	302	281
query34	908	523	540	523
query35	843	731	717	717
query36	1082	925	952	925
query37	109	70	67	67
query38	4016	3967	3946	3946
query39	1499	1444	1455	1444
query40	217	96	101	96
query41	48	47	49	47
query42	119	103	103	103
query43	522	485	500	485
query44	1181	819	814	814
query45	180	167	165	165
query46	1147	758	774	758
query47	1981	1885	1882	1882
query48	485	375	380	375
query49	737	414	374	374
query50	855	419	430	419
query51	7286	7209	7220	7209
query52	97	88	85	85
query53	255	181	186	181
query54	561	444	454	444
query55	75	72	75	72
query56	248	234	247	234
query57	1223	1137	1071	1071
query58	215	201	206	201
query59	3035	2998	2914	2914
query60	283	251	249	249
query61	113	108	110	108
query62	798	669	668	668
query63	221	190	190	190
query64	1412	661	609	609
query65	3243	3158	3159	3158
query66	694	285	287	285
query67	15808	15498	15398	15398
query68	4025	585	592	585
query69	426	264	264	264
query70	1149	1138	1107	1107
query71	359	261	257	257
query72	6410	4067	4059	4059
query73	740	353	359	353
query74	10206	9233	9158	9158
query75	3330	2630	2658	2630
query76	1912	1159	1066	1066
query77	473	275	276	275
query78	10659	9605	9449	9449
query79	1323	595	598	595
query80	861	419	414	414
query81	480	221	215	215
query82	1246	84	86	84
query83	239	149	143	143
query84	283	84	78	78
query85	883	301	286	286
query86	354	284	298	284
query87	4366	4281	4200	4200
query88	3659	2367	2336	2336
query89	409	294	293	293
query90	1989	187	185	185
query91	185	146	145	145
query92	68	47	57	47
query93	1568	556	552	552
query94	799	303	289	289
query95	352	251	248	248
query96	600	277	282	277
query97	3309	3183	3159	3159
query98	207	192	197	192
query99	1531	1298	1286	1286
Total cold run time: 316545 ms
Total hot run time: 195606 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41cb453f6d363366a08c76fa47926a6e0cee5a53, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.03
query3	0.23	0.06	0.06
query4	1.63	0.11	0.11
query5	0.54	0.51	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.03
query8	0.03	0.03	0.03
query9	0.55	0.52	0.49
query10	0.55	0.55	0.56
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	0.77	0.78	0.81
query15	0.83	0.82	0.82
query16	0.38	0.37	0.37
query17	0.99	1.07	1.04
query18	0.24	0.20	0.21
query19	1.96	1.85	1.85
query20	0.02	0.00	0.01
query21	15.40	0.61	0.60
query22	2.99	2.80	1.55
query23	17.00	0.99	0.78
query24	2.98	1.59	0.47
query25	0.19	0.18	0.13
query26	0.45	0.13	0.14
query27	0.05	0.04	0.03
query28	10.63	0.46	0.45
query29	12.63	3.18	3.17
query30	0.24	0.07	0.05
query31	2.87	0.39	0.37
query32	3.22	0.46	0.45
query33	2.93	2.96	3.05
query34	16.88	4.50	4.56
query35	4.51	4.50	4.55
query36	0.65	0.47	0.49
query37	0.10	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.15	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.95 s
Total hot run time: 29.32 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 41ce18b into apache:branch-3.0 Jun 11, 2025
22 checks passed
dataroaring pushed a commit that referenced this pull request Jun 18, 2025
… node in cloud mode (#51240) (#51405)

pick #51240

Accelerate query information_schema.tables by scanning table information
from FE master when sending query request from follower node. It can
reduces rpc call from follower to meta service to improve query
performance for master getting version is memory operation, but it will
slightly increase the pressure on the FE master.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants