Skip to content

[opt](cpu-profile) enable cpu profile in BE webui (#40330) #41849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Oct 15, 2024
Merged

[opt](cpu-profile) enable cpu profile in BE webui (#40330) #41849

morningman merged 1 commit into from
Oct 15, 2024

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Oct 15, 2024

bp #40330

@doris-robot
Copy link

doris-robot commented Oct 15, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

morningman commented Oct 15, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 15, 2024

TPC-H: Total hot run time: 40316 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a880175971eda0ce5b97bee7523e312a781d9e04, data reload: false

------ Round 1 ----------------------------------
q1	17587	7390	7551	7390
q2	2052	168	150	150
q3	10714	1073	1158	1073
q4	10541	784	767	767
q5	7722	2808	2836	2808
q6	228	149	148	148
q7	1027	634	612	612
q8	9570	1882	1975	1882
q9	7728	6373	6418	6373
q10	6956	2319	2280	2280
q11	448	246	253	246
q12	400	221	221	221
q13	17768	3005	2971	2971
q14	255	210	209	209
q15	559	503	515	503
q16	494	419	411	411
q17	967	534	545	534
q18	7201	6486	6542	6486
q19	1342	972	1005	972
q20	567	275	271	271
q21	3935	3060	3115	3060
q22	1093	1029	949	949
Total cold run time: 109154 ms
Total hot run time: 40316 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7353	7285	7244	7244
q2	326	228	219	219
q3	2824	2891	2898	2891
q4	1978	1804	1850	1804
q5	5671	5729	5716	5716
q6	223	141	148	141
q7	2221	1775	1772	1772
q8	3344	3556	3501	3501
q9	8700	8891	8837	8837
q10	3542	3469	3491	3469
q11	570	494	493	493
q12	796	610	601	601
q13	16591	3199	3137	3137
q14	320	267	274	267
q15	564	524	523	523
q16	493	466	471	466
q17	1823	1629	1609	1609
q18	8159	7772	7749	7749
q19	4064	1453	1394	1394
q20	2036	1857	1859	1857
q21	5534	5277	5285	5277
q22	1102	1028	999	999
Total cold run time: 78234 ms
Total hot run time: 59966 ms

@doris-robot
Copy link

doris-robot commented Oct 15, 2024

TPC-DS: Total hot run time: 193988 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a880175971eda0ce5b97bee7523e312a781d9e04, data reload: false

query1	1248	865	861	861
query2	6263	2015	2005	2005
query3	10807	3680	3821	3680
query4	63115	29413	23402	23402
query5	5158	454	442	442
query6	386	166	159	159
query7	5458	304	299	299
query8	267	212	199	199
query9	8541	2645	2624	2624
query10	441	294	292	292
query11	17148	15151	15645	15151
query12	161	109	102	102
query13	1498	462	446	446
query14	10677	7333	7254	7254
query15	208	170	173	170
query16	7075	507	518	507
query17	1136	618	582	582
query18	1805	322	313	313
query19	205	167	153	153
query20	125	113	112	112
query21	204	111	113	111
query22	4698	4500	4495	4495
query23	34454	34292	34209	34209
query24	6031	2863	2823	2823
query25	501	392	412	392
query26	655	162	158	158
query27	1670	294	292	292
query28	3965	2162	2171	2162
query29	655	437	440	437
query30	229	152	147	147
query31	978	757	813	757
query32	69	52	54	52
query33	410	290	297	290
query34	884	490	487	487
query35	870	745	749	745
query36	1080	954	929	929
query37	131	74	77	74
query38	3953	3890	3889	3889
query39	1475	1423	1398	1398
query40	205	96	103	96
query41	44	43	44	43
query42	114	93	96	93
query43	499	492	488	488
query44	1111	763	759	759
query45	199	168	168	168
query46	1131	719	696	696
query47	1873	1824	1819	1819
query48	423	350	339	339
query49	686	397	393	393
query50	817	429	399	399
query51	7147	6875	6806	6806
query52	94	94	81	81
query53	253	183	179	179
query54	558	456	446	446
query55	74	73	77	73
query56	274	236	246	236
query57	1219	1101	1133	1101
query58	217	229	223	223
query59	3259	2977	2974	2974
query60	274	267	265	265
query61	103	103	108	103
query62	761	657	646	646
query63	209	184	183	183
query64	1290	621	596	596
query65	3252	3177	3159	3159
query66	674	306	300	300
query67	15921	15375	15323	15323
query68	3674	534	519	519
query69	726	280	291	280
query70	1155	1117	1102	1102
query71	501	268	274	268
query72	8035	3926	3883	3883
query73	763	330	336	330
query74	10454	8881	8926	8881
query75	4897	2698	2685	2685
query76	3965	836	920	836
query77	804	291	277	277
query78	9979	9217	9062	9062
query79	2743	598	582	582
query80	2818	435	423	423
query81	563	219	225	219
query82	803	130	126	126
query83	317	132	137	132
query84	302	82	78	78
query85	1608	297	279	279
query86	448	302	306	302
query87	4465	4345	4230	4230
query88	4695	2388	2395	2388
query89	402	285	278	278
query90	2294	181	181	181
query91	144	102	104	102
query92	59	46	46	46
query93	4087	529	528	528
query94	1238	295	275	275
query95	347	248	252	248
query96	620	279	274	274
query97	3268	3137	3127	3127
query98	211	196	191	191
query99	1509	1269	1282	1269
Total cold run time: 325209 ms
Total hot run time: 193988 ms

@morningman
Copy link
Contributor Author

morningman commented Oct 15, 2024

run buildall

@morningman
[opt](cpu-profile) enable cpu profile in BE webui (apache#40330)
9ab3048 
Visit `be_ip:webserver_port/cpu`
You can get cpu profile like this:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/05da73ca-c02a-4ec3-a2fb-83ef80ff9553">

or

![img_v3_02ec_e8d45934-ddaf-45de-818e-a7e4899ec7cg](https://github.com/user-attachments/assets/6b1a4e15-a219-456b-9d1f-e3be88177db5)
@morningman
Copy link
Contributor Author

morningman commented Oct 15, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Oct 15, 2024

TPC-H: Total hot run time: 40092 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ab30480e9123a12553d937a0112b80f1a048b8c, data reload: false

------ Round 1 ----------------------------------
q1	18182	7386	7300	7300
q2	2030	162	141	141
q3	10595	1103	1090	1090
q4	10213	742	646	646
q5	7711	2816	2712	2712
q6	226	146	144	144
q7	978	624	602	602
q8	9339	1875	1919	1875
q9	6589	6350	6328	6328
q10	6982	2266	2275	2266
q11	444	243	254	243
q12	408	206	205	205
q13	17800	2980	2957	2957
q14	246	213	220	213
q15	556	524	515	515
q16	478	408	405	405
q17	965	620	465	465
q18	7335	6642	6644	6642
q19	2600	956	1010	956
q20	587	263	280	263
q21	4032	3181	3150	3150
q22	1102	981	974	974
Total cold run time: 109398 ms
Total hot run time: 40092 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7316	7254	7238	7238
q2	332	222	229	222
q3	2831	2751	2707	2707
q4	1861	1644	1643	1643
q5	5329	5398	5349	5349
q6	216	137	137	137
q7	2091	1634	1659	1634
q8	3184	3348	3367	3348
q9	8523	8479	8471	8471
q10	3431	3397	3371	3371
q11	568	490	485	485
q12	745	561	560	560
q13	16899	2992	2941	2941
q14	290	273	260	260
q15	560	509	510	509
q16	479	434	440	434
q17	1767	1567	1521	1521
q18	7703	7300	7234	7234
q19	1647	1565	1546	1546
q20	2001	1765	1785	1765
q21	5010	4960	5060	4960
q22	1090	987	983	983
Total cold run time: 73873 ms
Total hot run time: 57318 ms

@doris-robot
Copy link

doris-robot commented Oct 15, 2024

TPC-DS: Total hot run time: 188286 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ab30480e9123a12553d937a0112b80f1a048b8c, data reload: false

query1	989	375	359	359
query2	6533	2062	1952	1952
query3	6701	211	220	211
query4	33781	23350	23508	23350
query5	4310	462	437	437
query6	255	162	166	162
query7	4615	296	294	294
query8	246	207	194	194
query9	9709	2637	2630	2630
query10	474	308	283	283
query11	18018	15162	14997	14997
query12	152	96	98	96
query13	1682	446	424	424
query14	10055	7170	7134	7134
query15	231	176	168	168
query16	7997	455	451	451
query17	1619	588	552	552
query18	1996	318	323	318
query19	217	144	146	144
query20	114	107	108	107
query21	212	101	101	101
query22	4173	4176	4291	4176
query23	34706	33942	33692	33692
query24	11860	2800	2849	2800
query25	689	407	411	407
query26	1743	157	160	157
query27	2804	296	288	288
query28	7817	2133	2109	2109
query29	1039	447	426	426
query30	334	151	145	145
query31	1059	767	781	767
query32	101	55	54	54
query33	775	295	287	287
query34	990	488	498	488
query35	868	740	717	717
query36	1081	933	938	933
query37	276	79	79	79
query38	3977	3790	3841	3790
query39	1469	1447	1417	1417
query40	302	100	96	96
query41	46	47	43	43
query42	121	96	96	96
query43	525	484	489	484
query44	1264	777	758	758
query45	195	171	172	171
query46	1151	699	702	699
query47	1919	1816	1840	1816
query48	418	344	343	343
query49	1261	399	400	399
query50	816	394	400	394
query51	7052	6957	7049	6957
query52	104	85	90	85
query53	262	187	188	187
query54	1211	471	463	463
query55	80	77	75	75
query56	285	251	256	251
query57	1238	1148	1140	1140
query58	253	240	247	240
query59	3141	2835	2889	2835
query60	301	268	252	252
query61	104	106	100	100
query62	866	654	669	654
query63	207	177	176	176
query64	5133	603	585	585
query65	3216	3139	3149	3139
query66	1421	292	300	292
query67	15701	15537	15194	15194
query68	4979	549	542	542
query69	523	281	288	281
query70	1216	1127	1099	1099
query71	349	264	264	264
query72	7201	3905	3835	3835
query73	768	330	345	330
query74	10450	8877	8936	8877
query75	3385	2645	2682	2645
query76	3434	859	833	833
query77	601	291	287	287
query78	10066	9184	9149	9149
query79	3446	585	575	575
query80	1982	429	430	429
query81	575	219	225	219
query82	1029	125	127	125
query83	324	133	156	133
query84	309	79	70	70
query85	2314	296	274	274
query86	521	288	288	288
query87	4482	4168	4212	4168
query88	4391	2364	2396	2364
query89	422	290	289	289
query90	2092	189	183	183
query91	144	108	104	104
query92	69	46	50	46
query93	4965	536	533	533
query94	1144	274	284	274
query95	350	252	249	249
query96	629	274	281	274
query97	3313	3124	3064	3064
query98	222	198	200	198
query99	1832	1265	1288	1265
Total cold run time: 311787 ms
Total hot run time: 188286 ms

@morningman morningman merged commit b2d245c into apache:branch-3.0 Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@morningman @doris-robot