Skip to content

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Feb 13, 2025

What problem does this PR solve?

Add FE metrics for

  1. num of catalogs
  2. num of internal databases
  3. num of internal tables
# HELP doris_fe_catalog_num total catalog num
# TYPE doris_fe_catalog_num gauge
doris_fe_catalog_num 1

# HELP doris_fe_internal_database_num total internal database num
# TYPE doris_fe_internal_database_num gauge
doris_fe_internal_database_num 4

# HELP doris_fe_internal_table_num total internal table num
# TYPE doris_fe_internal_table_num gauge
doris_fe_internal_table_num 44

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman marked this pull request as ready for review February 13, 2025 16:01
@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31108 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34e9b8ba4730fbb1335e5fa664f0d7ae07eb9a7a, data reload: false

------ Round 1 ----------------------------------
q1	17641	5153	5092	5092
q2	2049	291	165	165
q3	10427	1210	740	740
q4	10213	989	517	517
q5	7538	2288	2320	2288
q6	180	160	129	129
q7	900	740	596	596
q8	9297	1264	1053	1053
q9	4913	4389	4712	4389
q10	6805	2305	1887	1887
q11	466	278	251	251
q12	335	353	211	211
q13	17784	3675	3097	3097
q14	232	236	207	207
q15	492	467	475	467
q16	659	623	567	567
q17	568	839	340	340
q18	6581	6222	6202	6202
q19	1218	925	538	538
q20	305	311	189	189
q21	2772	2042	1891	1891
q22	363	326	292	292
Total cold run time: 101738 ms
Total hot run time: 31108 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5058	5032	5060	5032
q2	229	325	233	233
q3	2163	2673	2287	2287
q4	1385	1803	1379	1379
q5	4239	4089	4175	4089
q6	214	163	122	122
q7	1842	1767	1666	1666
q8	2486	2626	2523	2523
q9	7144	7125	7077	7077
q10	2987	3172	2793	2793
q11	570	508	474	474
q12	733	768	614	614
q13	3602	3765	3372	3372
q14	274	297	271	271
q15	499	475	446	446
q16	662	686	639	639
q17	1119	1570	1346	1346
q18	7573	7384	7396	7384
q19	755	800	911	800
q20	1933	2038	1862	1862
q21	5437	4939	4780	4780
q22	649	601	553	553
Total cold run time: 51553 ms
Total hot run time: 49742 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189282 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34e9b8ba4730fbb1335e5fa664f0d7ae07eb9a7a, data reload: false

query1	1345	947	932	932
query2	6225	1802	1789	1789
query3	10964	4418	4352	4352
query4	55917	24401	23048	23048
query5	5089	576	478	478
query6	355	197	187	187
query7	4951	505	285	285
query8	324	251	233	233
query9	6030	2573	2567	2567
query10	437	297	253	253
query11	15232	15003	15005	15003
query12	154	107	104	104
query13	1069	511	377	377
query14	10521	6340	6624	6340
query15	209	200	177	177
query16	7018	630	495	495
query17	1050	721	556	556
query18	1532	407	305	305
query19	203	193	161	161
query20	128	121	124	121
query21	212	124	113	113
query22	4554	4435	4524	4435
query23	34073	33302	33553	33302
query24	5671	2461	2420	2420
query25	485	478	437	437
query26	680	284	164	164
query27	1679	485	347	347
query28	2862	2479	2436	2436
query29	593	552	412	412
query30	215	186	156	156
query31	874	849	805	805
query32	74	66	59	59
query33	435	346	298	298
query34	794	859	508	508
query35	815	820	761	761
query36	980	985	902	902
query37	123	92	70	70
query38	4236	4241	4241	4241
query39	1501	1446	1436	1436
query40	202	115	102	102
query41	50	48	51	48
query42	120	111	104	104
query43	493	518	500	500
query44	1295	824	817	817
query45	176	176	164	164
query46	879	1075	660	660
query47	1854	1874	1804	1804
query48	386	427	309	309
query49	716	517	418	418
query50	731	772	425	425
query51	4252	4289	4196	4196
query52	116	110	105	105
query53	243	270	191	191
query54	498	496	426	426
query55	89	92	83	83
query56	269	320	271	271
query57	1183	1183	1130	1130
query58	253	244	244	244
query59	2923	2822	2801	2801
query60	291	285	254	254
query61	121	117	121	117
query62	714	756	684	684
query63	236	192	197	192
query64	1474	1027	732	732
query65	3243	3141	3130	3130
query66	703	433	300	300
query67	15690	15552	15485	15485
query68	5400	770	512	512
query69	530	301	260	260
query70	1264	1138	1054	1054
query71	429	298	259	259
query72	5311	3491	3691	3491
query73	1265	706	343	343
query74	9202	9144	8698	8698
query75	3192	3134	2664	2664
query76	3820	1171	725	725
query77	535	363	295	295
query78	9985	10143	9264	9264
query79	1358	863	585	585
query80	714	527	434	434
query81	498	267	233	233
query82	441	189	96	96
query83	166	167	157	157
query84	283	96	71	71
query85	721	342	317	317
query86	362	276	281	276
query87	4498	4371	4381	4371
query88	2802	2166	2144	2144
query89	382	317	302	302
query90	1610	189	186	186
query91	131	136	108	108
query92	64	59	57	57
query93	1378	990	581	581
query94	632	401	304	304
query95	344	264	247	247
query96	475	578	262	262
query97	2745	2843	2761	2761
query98	227	217	202	202
query99	1263	1368	1266	1266
Total cold run time: 290836 ms
Total hot run time: 189282 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.16 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34e9b8ba4730fbb1335e5fa664f0d7ae07eb9a7a, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.02
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.40	0.42	0.40
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.59	0.53	0.51
query10	0.58	0.58	0.59
query11	0.15	0.11	0.10
query12	0.14	0.10	0.11
query13	0.62	0.60	0.59
query14	2.79	2.70	2.74
query15	0.91	0.87	0.86
query16	0.37	0.41	0.37
query17	1.08	1.06	1.03
query18	0.22	0.20	0.20
query19	1.95	1.76	1.95
query20	0.01	0.02	0.01
query21	15.38	0.87	0.54
query22	0.76	1.19	0.67
query23	14.89	1.37	0.66
query24	10.96	0.93	0.39
query25	0.32	0.28	0.14
query26	0.90	0.18	0.13
query27	0.05	0.05	0.05
query28	6.36	0.83	0.44
query29	12.61	3.94	3.27
query30	0.24	0.08	0.05
query31	2.82	0.58	0.38
query32	3.27	0.54	0.47
query33	2.95	3.07	3.02
query34	15.86	5.11	4.52
query35	4.52	4.49	4.49
query36	0.67	0.51	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.02 s
Total hot run time: 30.16 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 14, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit a14e95e into apache:master Feb 15, 2025
32 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 15, 2025
### What problem does this PR solve?

Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
# HELP doris_fe_catalog_num total catalog num
# TYPE doris_fe_catalog_num gauge
doris_fe_catalog_num 1

# HELP doris_fe_internal_database_num total internal database num
# TYPE doris_fe_internal_database_num gauge
doris_fe_internal_database_num 4

# HELP doris_fe_internal_table_num total internal table num
# TYPE doris_fe_internal_table_num gauge
doris_fe_internal_table_num 44
```
morningman added a commit to morningman/doris that referenced this pull request Feb 18, 2025
Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
doris_fe_catalog_num 1

doris_fe_internal_database_num 4

doris_fe_internal_table_num 44
```
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
### What problem does this PR solve?

Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
# HELP doris_fe_catalog_num total catalog num
# TYPE doris_fe_catalog_num gauge
doris_fe_catalog_num 1

# HELP doris_fe_internal_database_num total internal database num
# TYPE doris_fe_internal_database_num gauge
doris_fe_internal_database_num 4

# HELP doris_fe_internal_table_num total internal table num
# TYPE doris_fe_internal_table_num gauge
doris_fe_internal_table_num 44
```
dataroaring pushed a commit that referenced this pull request Feb 24, 2025
#47945)

Cherry-picked from #47891

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
morningman added a commit to morningman/doris that referenced this pull request Feb 24, 2025
Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
doris_fe_catalog_num 1

doris_fe_internal_database_num 4

doris_fe_internal_table_num 44
```
@gsvitak-ionna
Copy link

@morningman are these metric expose to Prometheus?

@gavinchou gavinchou mentioned this pull request Apr 23, 2025
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
### What problem does this PR solve?

Add FE metrics for
1. num of catalogs
2. num of internal databases
3. num of internal tables

```
# HELP doris_fe_catalog_num total catalog num
# TYPE doris_fe_catalog_num gauge
doris_fe_catalog_num 1

# HELP doris_fe_internal_database_num total internal database num
# TYPE doris_fe_internal_database_num gauge
doris_fe_internal_database_num 4

# HELP doris_fe_internal_table_num total internal table num
# TYPE doris_fe_internal_table_num gauge
doris_fe_internal_table_num 44
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.5-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants