Skip to content

[chore](log) print query id before logging profile in be.INFO #32915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 2 commits into from
Mar 27, 2024
Merged

[chore](log) print query id before logging profile in be.INFO #32915

yiguolei merged 2 commits into from
Mar 27, 2024

Conversation

@zhiqiang-hhhh
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh commented Mar 27, 2024

No description provided.

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Mar 27, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-H: Total hot run time: 49754 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 89b733b421f659a361d08424a598216426e0b9d0, data reload: false

------ Round 1 ----------------------------------
q1	17955	4369	4361	4361
q2	2032	154	138	138
q3	10434	1920	1942	1920
q4	10315	1271	1332	1271
q5	8487	3950	3950	3950
q6	219	123	124	123
q7	2047	1600	1574	1574
q8	9343	2703	2709	2703
q9	10781	10527	10344	10344
q10	8637	3536	3469	3469
q11	412	230	240	230
q12	464	303	301	301
q13	18371	3991	4035	3991
q14	364	326	320	320
q15	506	459	463	459
q16	689	602	588	588
q17	1110	963	958	958
q18	7249	6769	6874	6769
q19	1655	1576	1573	1573
q20	542	321	300	300
q21	4485	4203	4030	4030
q22	510	382	386	382
Total cold run time: 116607 ms
Total hot run time: 49754 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4313	4352	4310	4310
q2	324	227	214	214
q3	4162	4164	4179	4164
q4	2766	2755	2746	2746
q5	7272	7166	7206	7166
q6	231	120	122	120
q7	3270	2842	2893	2842
q8	4339	4437	4499	4437
q9	16980	16991	16989	16989
q10	4236	4257	4295	4257
q11	749	698	685	685
q12	1026	849	833	833
q13	6780	3758	3762	3758
q14	463	419	418	418
q15	492	459	448	448
q16	771	705	706	705
q17	3859	3880	3915	3880
q18	8911	8764	8730	8730
q19	1706	1689	1626	1626
q20	2390	2105	2160	2105
q21	8562	8559	8547	8547
q22	1023	950	962	950
Total cold run time: 84625 ms
Total hot run time: 79930 ms

yiguolei
yiguolei reviewed Mar 27, 2024
View reviewed changes
be/src/runtime/plan_fragment_executor.cpp Outdated

if (_is_report_success) {
std::stringstream ss;
ss << "Query : " << print_id(this->_query_id) << " instance "
Copy link
Contributor

@yiguolei yiguolei Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要这样写
把代码写到542 行跟随LOG(INFO)一起输出就可以了。

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TeamCity be ut coverage result:
Function Coverage: 37.79% (8044/21285)
Line Coverage: 29.46% (65694/223016)
Region Coverage: 28.92% (33803/116903)
Branch Coverage: 24.77% (17354/70066)
Coverage Report: http://coverage.selectdb-in.cc/coverage/89b733b421f659a361d08424a598216426e0b9d0_89b733b421f659a361d08424a598216426e0b9d0/report/index.html

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-DS: Total hot run time: 202922 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 89b733b421f659a361d08424a598216426e0b9d0, data reload: false

query1	940	393	389	389
query2	6551	2171	2255	2171
query3	6924	200	198	198
query4	20627	18041	17935	17935
query5	19734	6568	6484	6484
query6	285	214	225	214
query7	4163	285	293	285
query8	266	273	231	231
query9	3148	2704	2632	2632
query10	413	280	307	280
query11	11327	10728	10753	10728
query12	122	73	71	71
query13	5566	653	669	653
query14	17184	13551	13579	13551
query15	388	231	238	231
query16	6460	283	260	260
query17	1717	1453	860	860
query18	2329	406	386	386
query19	204	148	147	147
query20	77	73	74	73
query21	181	94	92	92
query22	5289	5077	5025	5025
query23	32536	31969	32048	31969
query24	7009	6499	6548	6499
query25	507	428	412	412
query26	524	166	158	158
query27	1898	290	300	290
query28	6108	2249	2244	2244
query29	2926	2643	2763	2643
query30	239	159	160	159
query31	886	732	746	732
query32	65	61	55	55
query33	378	253	248	248
query34	841	476	482	476
query35	1101	878	917	878
query36	1351	1316	1248	1248
query37	88	59	58	58
query38	3100	2936	2885	2885
query39	1360	1317	1310	1310
query40	199	94	93	93
query41	35	32	33	32
query42	85	96	81	81
query43	613	525	588	525
query44	1143	708	716	708
query45	245	226	231	226
query46	1233	1000	969	969
query47	1943	1811	1858	1811
query48	971	665	660	660
query49	615	353	372	353
query50	863	621	573	573
query51	4691	4739	4740	4739
query52	98	75	87	75
query53	437	323	315	315
query54	2665	2419	2433	2419
query55	104	79	87	79
query56	216	199	191	191
query57	1219	1161	1047	1047
query58	212	192	186	186
query59	3362	3446	3315	3315
query60	204	192	198	192
query61	82	80	85	80
query62	786	464	508	464
query63	471	332	336	332
query64	2402	1474	1373	1373
query65	3620	3570	3559	3559
query66	767	366	372	366
query67	15807	15637	14967	14967
query68	10351	677	691	677
query69	573	333	342	333
query70	1905	1478	1465	1465
query71	391	289	310	289
query72	6447	3475	3452	3452
query73	740	324	318	318
query74	6289	5947	5766	5766
query75	5377	3695	3780	3695
query76	6893	1152	1216	1152
query77	1189	247	250	247
query78	12580	12901	13510	12901
query79	10097	662	680	662
query80	673	398	396	396
query81	497	233	235	233
query82	320	104	96	96
query83	187	138	125	125
query84	254	69	71	69
query85	726	281	277	277
query86	348	290	304	290
query87	3278	3042	3026	3026
query88	4998	2331	2329	2329
query89	337	292	290	290
query90	2567	217	212	212
query91	152	117	117	117
query92	54	50	53	50
query93	3059	588	592	588
query94	1170	209	204	204
query95	1128	1093	1080	1080
query96	637	326	325	325
query97	6488	6356	6244	6244
query98	181	178	171	171
query99	2913	955	953	953
Total cold run time: 312699 ms
Total hot run time: 202922 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

ClickBench: Total hot run time: 31.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 89b733b421f659a361d08424a598216426e0b9d0, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.04
query4	1.80	0.07	0.07
query5	0.53	0.52	0.52
query6	1.27	0.60	0.60
query7	0.01	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.49
query10	0.54	0.54	0.55
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.60
query14	0.77	0.80	0.80
query15	0.81	0.76	0.76
query16	0.36	0.39	0.38
query17	0.98	1.02	1.02
query18	0.21	0.27	0.26
query19	1.92	1.86	1.77
query20	0.02	0.01	0.01
query21	15.46	0.56	0.56
query22	1.99	2.59	2.00
query23	17.36	0.90	0.98
query24	4.93	1.51	1.48
query25	0.33	0.08	0.05
query26	0.70	0.17	0.17
query27	0.04	0.05	0.04
query28	7.70	0.76	0.71
query29	12.62	2.35	2.34
query30	0.58	0.52	0.47
query31	2.82	0.39	0.37
query32	3.39	0.51	0.49
query33	3.07	3.07	3.03
query34	15.25	4.79	4.79
query35	4.84	4.86	4.84
query36	1.08	1.01	1.02
query37	0.06	0.04	0.04
query38	0.04	0.02	0.03
query39	0.02	0.01	0.01
query40	0.16	0.13	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.02
query43	0.02	0.01	0.02
Total cold run time: 103.53 s
Total hot run time: 31.6 s

@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 89b733b421f659a361d08424a598216426e0b9d0 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.8 seconds inserted 10000000 Rows, about 480K ops/s

@zhiqiang-hhhh zhiqiang-hhhh mentioned this pull request Mar 27, 2024
Merged
@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Mar 27, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-H: Total hot run time: 49844 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34beb8935268e133724eb30ba9a2f725c7aa7059, data reload: false

------ Round 1 ----------------------------------
q1	17936	4456	4394	4394
q2	2028	153	147	147
q3	10328	1887	1937	1887
q4	10103	1242	1302	1242
q5	8535	4011	3988	3988
q6	235	122	121	121
q7	1999	1580	1595	1580
q8	9244	2723	2719	2719
q9	11098	10840	10385	10385
q10	8644	3506	3524	3506
q11	416	243	237	237
q12	460	296	293	293
q13	18336	3981	4004	3981
q14	346	323	329	323
q15	504	457	452	452
q16	703	590	580	580
q17	1123	979	976	976
q18	7298	6823	6894	6823
q19	1702	1537	1455	1455
q20	503	287	294	287
q21	4412	4080	4144	4080
q22	499	401	388	388
Total cold run time: 116452 ms
Total hot run time: 49844 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4389	4301	4297	4297
q2	315	227	220	220
q3	4160	4153	4137	4137
q4	2720	2741	2729	2729
q5	7286	7241	7190	7190
q6	231	121	116	116
q7	3204	2857	2800	2800
q8	4313	4463	4473	4463
q9	17635	17401	17079	17079
q10	4260	4223	4306	4223
q11	755	665	696	665
q12	1014	824	866	824
q13	6995	3758	3752	3752
q14	462	426	428	426
q15	501	450	458	450
q16	750	686	697	686
q17	3836	3814	3834	3814
q18	8846	8777	8923	8777
q19	1709	1695	1668	1668
q20	2395	2143	2133	2133
q21	8359	8353	8474	8353
q22	1015	942	980	942
Total cold run time: 85150 ms
Total hot run time: 79744 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TeamCity be ut coverage result:
Function Coverage: 37.80% (8046/21285)
Line Coverage: 29.47% (65720/223016)
Region Coverage: 28.93% (33819/116903)
Branch Coverage: 24.78% (17363/70066)
Coverage Report: http://coverage.selectdb-in.cc/coverage/34beb8935268e133724eb30ba9a2f725c7aa7059_34beb8935268e133724eb30ba9a2f725c7aa7059/report/index.html

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

TPC-DS: Total hot run time: 201070 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34beb8935268e133724eb30ba9a2f725c7aa7059, data reload: false

query1	952	391	378	378
query2	6537	2264	2142	2142
query3	6920	197	200	197
query4	20351	17884	17888	17884
query5	19740	6511	6514	6511
query6	290	210	224	210
query7	4150	294	300	294
query8	252	252	261	252
query9	3154	2718	2646	2646
query10	418	286	306	286
query11	11258	10681	10662	10662
query12	117	83	68	68
query13	5580	638	623	623
query14	17817	13512	13694	13512
query15	367	227	242	227
query16	6452	272	253	253
query17	1582	1447	847	847
query18	2308	404	395	395
query19	222	139	144	139
query20	77	78	75	75
query21	191	93	90	90
query22	5349	5097	5060	5060
query23	32386	31792	31939	31792
query24	6962	6510	6507	6507
query25	528	421	408	408
query26	526	159	156	156
query27	1935	292	291	291
query28	6156	2268	2224	2224
query29	2847	2656	2714	2656
query30	241	164	157	157
query31	905	726	702	702
query32	67	59	62	59
query33	398	253	242	242
query34	840	458	456	456
query35	1147	941	947	941
query36	1235	1253	1214	1214
query37	89	64	58	58
query38	3115	2929	2931	2929
query39	1356	1318	1316	1316
query40	209	90	97	90
query41	39	36	31	31
query42	79	80	82	80
query43	635	618	625	618
query44	1133	708	715	708
query45	237	224	223	223
query46	1235	972	979	972
query47	1874	1889	1676	1676
query48	1000	659	651	651
query49	623	364	375	364
query50	873	626	597	597
query51	4748	4633	4673	4633
query52	78	79	72	72
query53	436	314	310	310
query54	2672	2440	2466	2440
query55	85	79	83	79
query56	203	212	206	206
query57	1145	1035	1196	1035
query58	210	184	203	184
query59	3482	3310	3249	3249
query60	204	188	183	183
query61	86	84	87	84
query62	844	458	521	458
query63	458	326	329	326
query64	2397	1485	1334	1334
query65	3604	3522	3554	3522
query66	752	372	363	363
query67	16089	15364	15804	15364
query68	8365	646	673	646
query69	574	319	338	319
query70	1503	1472	1313	1313
query71	403	300	302	300
query72	6500	3447	3468	3447
query73	741	312	313	312
query74	6281	5820	5930	5820
query75	4547	3683	3635	3635
query76	4617	1148	1155	1148
query77	606	244	249	244
query78	12833	11649	12172	11649
query79	8111	641	645	641
query80	1316	396	389	389
query81	497	230	229	229
query82	1536	98	98	98
query83	164	133	129	129
query84	258	70	66	66
query85	863	295	286	286
query86	304	285	290	285
query87	3307	2961	3039	2961
query88	4750	2299	2311	2299
query89	371	267	310	267
query90	1853	210	219	210
query91	149	117	119	117
query92	55	50	51	50
query93	5528	601	552	552
query94	646	201	201	201
query95	1119	1072	1050	1050
query96	638	320	320	320
query97	6431	6431	6364	6364
query98	187	179	175	175
query99	2928	881	837	837
Total cold run time: 308228 ms
Total hot run time: 201070 ms

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

ClickBench: Total hot run time: 30.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34beb8935268e133724eb30ba9a2f725c7aa7059, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.24	0.05	0.05
query4	1.78	0.06	0.06
query5	0.54	0.51	0.52
query6	1.24	0.62	0.63
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.48	0.48
query10	0.55	0.53	0.54
query11	0.12	0.08	0.10
query12	0.12	0.09	0.09
query13	0.61	0.61	0.60
query14	0.77	0.78	0.80
query15	0.77	0.75	0.76
query16	0.36	0.37	0.37
query17	0.99	1.02	0.98
query18	0.20	0.26	0.24
query19	1.90	1.81	1.79
query20	0.01	0.01	0.01
query21	15.50	0.56	0.56
query22	2.01	1.97	1.65
query23	17.19	1.01	0.94
query24	4.89	1.31	1.33
query25	0.39	0.12	0.05
query26	0.50	0.16	0.17
query27	0.05	0.04	0.04
query28	7.61	0.75	0.76
query29	12.64	2.17	2.38
query30	0.57	0.52	0.51
query31	2.80	0.38	0.37
query32	3.39	0.49	0.48
query33	3.02	3.06	3.10
query34	15.25	4.80	4.83
query35	4.87	4.84	4.85
query36	1.07	1.00	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.02
query42	0.02	0.02	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.02 s
Total hot run time: 30.95 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 27, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 27, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Mar 27, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 34beb8935268e133724eb30ba9a2f725c7aa7059 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

@yiguolei yiguolei merged commit 8650ac6 into apache:branch-2.0 Mar 27, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@zhiqiang-hhhh
[chore](log) print query id before logging profile in be.INFO (apache…
2c34b72 
…#32915)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.8-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@zhiqiang-hhhh @doris-robot @yiguolei @wm1581066