Skip to content

branch-3.0: [feature](metrics) add catalog/db/table num metrics #47891 #47945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Feb 24, 2025
Merged

branch-3.0: [feature](metrics) add catalog/db/table num metrics #47891 #47945

dataroaring merged 1 commit into from
Feb 24, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Feb 15, 2025

Cherry-picked from #47891

@morningman
[feature](metrics) add catalog/db/table num metrics (#47891)
d646ed8 
### What problem does this PR solve?

Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
# HELP doris_fe_catalog_num total catalog num
# TYPE doris_fe_catalog_num gauge
doris_fe_catalog_num 1

# HELP doris_fe_internal_database_num total internal database num
# TYPE doris_fe_internal_database_num gauge
doris_fe_internal_database_num 4

# HELP doris_fe_internal_table_num total internal table num
# TYPE doris_fe_internal_table_num gauge
doris_fe_internal_table_num 44
```
@Thearas
Copy link
Contributor

Thearas commented Feb 15, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 15, 2025
@Thearas
Copy link
Contributor

Thearas commented Feb 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 15, 2025

TPC-H: Total hot run time: 40775 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d646ed83088fcc2622c6be1f49bde859d79af0af, data reload: false

------ Round 1 ----------------------------------
q1	17565	7395	7206	7206
q2	2071	162	169	162
q3	11100	1076	1206	1076
q4	10564	762	712	712
q5	7731	2884	2866	2866
q6	237	149	149	149
q7	975	623	603	603
q8	9358	1899	2040	1899
q9	6563	6388	6452	6388
q10	7055	2264	2375	2264
q11	467	286	263	263
q12	401	220	212	212
q13	17806	2990	3036	2990
q14	237	209	216	209
q15	569	529	517	517
q16	686	575	576	575
q17	975	551	536	536
q18	7353	6673	6687	6673
q19	1391	1050	1030	1030
q20	466	201	196	196
q21	4055	3259	3700	3259
q22	1130	997	990	990
Total cold run time: 108755 ms
Total hot run time: 40775 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7255	7216	7256	7216
q2	336	237	230	230
q3	2949	2936	2945	2936
q4	2072	1849	1817	1817
q5	5717	5753	5730	5730
q6	228	142	145	142
q7	2242	1797	1800	1797
q8	3327	3566	3528	3528
q9	8838	8881	8875	8875
q10	3608	3589	3558	3558
q11	600	501	508	501
q12	780	656	618	618
q13	9379	3161	3165	3161
q14	319	303	279	279
q15	585	527	542	527
q16	705	655	645	645
q17	1827	1638	1603	1603
q18	8261	7688	7572	7572
q19	1666	1525	1504	1504
q20	2083	1820	1862	1820
q21	5507	5360	5414	5360
q22	1177	1030	1041	1030
Total cold run time: 69461 ms
Total hot run time: 60449 ms

@doris-robot
Copy link

doris-robot commented Feb 15, 2025

TPC-DS: Total hot run time: 197458 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d646ed83088fcc2622c6be1f49bde859d79af0af, data reload: false

query1	1303	946	910	910
query2	6248	2016	2012	2012
query3	10937	4254	4484	4254
query4	66377	28500	23636	23636
query5	4992	470	454	454
query6	419	186	188	186
query7	5458	310	321	310
query8	305	225	224	224
query9	8544	2674	2665	2665
query10	450	269	263	263
query11	17294	15120	15947	15120
query12	163	107	110	107
query13	1446	467	458	458
query14	10417	6925	7608	6925
query15	195	182	175	175
query16	7166	500	515	500
query17	1107	583	593	583
query18	1851	331	327	327
query19	223	163	171	163
query20	126	116	112	112
query21	206	106	104	104
query22	4839	4472	4475	4472
query23	34744	34050	34397	34050
query24	6101	2963	2868	2868
query25	553	399	405	399
query26	664	163	166	163
query27	2047	347	347	347
query28	4524	2465	2421	2421
query29	694	439	441	439
query30	246	159	168	159
query31	999	848	832	832
query32	77	57	61	57
query33	398	283	292	283
query34	916	502	512	502
query35	835	725	721	721
query36	1094	978	957	957
query37	128	77	76	76
query38	4160	3963	4070	3963
query39	1508	1475	1488	1475
query40	211	98	103	98
query41	48	47	49	47
query42	115	103	99	99
query43	524	485	488	485
query44	1201	833	827	827
query45	185	172	173	172
query46	1151	696	743	696
query47	2038	1939	1914	1914
query48	481	382	390	382
query49	738	401	408	401
query50	836	430	432	430
query51	7387	7205	7195	7195
query52	99	87	86	86
query53	252	174	181	174
query54	550	444	449	444
query55	79	73	76	73
query56	262	244	245	244
query57	1255	1099	1092	1092
query58	207	203	200	200
query59	3022	3016	3020	3016
query60	270	249	252	249
query61	106	110	122	110
query62	830	726	726	726
query63	219	195	199	195
query64	1391	678	637	637
query65	3224	3197	3177	3177
query66	710	294	308	294
query67	15810	15495	15406	15406
query68	3723	594	563	563
query69	419	261	295	261
query70	1148	1085	1037	1037
query71	358	260	266	260
query72	6394	4198	4171	4171
query73	757	343	357	343
query74	10245	9056	9153	9056
query75	3350	2660	2702	2660
query76	1980	1099	1150	1099
query77	490	279	279	279
query78	10512	9609	9577	9577
query79	1379	589	599	589
query80	877	425	433	425
query81	506	239	230	230
query82	1254	117	117	117
query83	245	148	152	148
query84	285	80	76	76
query85	875	310	292	292
query86	349	302	298	298
query87	4401	4394	4262	4262
query88	3598	2384	2371	2371
query89	416	292	289	289
query90	2023	188	193	188
query91	188	149	156	149
query92	72	52	49	49
query93	1627	544	544	544
query94	812	280	300	280
query95	363	251	260	251
query96	613	282	282	282
query97	3289	3177	3172	3172
query98	213	198	200	198
query99	1697	1405	1401	1401
Total cold run time: 317413 ms
Total hot run time: 197458 ms

@doris-robot
Copy link

doris-robot commented Feb 15, 2025

ClickBench: Total hot run time: 32.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d646ed83088fcc2622c6be1f49bde859d79af0af, data reload: false

query1	0.04	0.03	0.02
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.63	0.10	0.10
query5	0.50	0.52	0.52
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.03	0.02
query9	0.57	0.50	0.48
query10	0.55	0.56	0.56
query11	0.14	0.09	0.10
query12	0.14	0.11	0.10
query13	0.60	0.60	0.60
query14	2.73	2.85	2.75
query15	0.90	0.82	0.84
query16	0.39	0.36	0.38
query17	1.04	1.07	1.06
query18	0.24	0.22	0.21
query19	1.86	1.82	2.02
query20	0.02	0.01	0.01
query21	15.35	0.61	0.60
query22	3.32	2.18	1.44
query23	17.00	0.83	1.00
query24	2.72	0.85	1.72
query25	0.39	0.18	0.12
query26	0.24	0.13	0.13
query27	0.04	0.04	0.04
query28	10.61	1.12	1.07
query29	12.56	3.20	3.24
query30	0.26	0.06	0.06
query31	2.86	0.38	0.38
query32	3.24	0.46	0.45
query33	2.99	3.00	2.98
query34	17.03	4.52	4.59
query35	4.54	4.51	4.58
query36	0.66	0.51	0.49
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.16 s
Total hot run time: 32.34 s

dataroaring
dataroaring approved these changes Feb 24, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 20976a5 into branch-3.0 Feb 24, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-47891-branch-3.0 branch February 24, 2025 03:30
@dataroaring dataroaring mentioned this pull request Feb 24, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @doris-robot @dataroaring @morningman