Skip to content

[feat](nereids)print physical plan in profile #35144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 1 commit into from
May 22, 2024
Merged

[feat](nereids)print physical plan in profile #35144

englefly merged 1 commit into from
May 22, 2024

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented May 21, 2024 •
edited
Loading

Proposed changes

print physical plan in profile summary

Summary:
   - Profile ID: f3c4842af5f64dd6-a2b4cd00e16e596f
   - Task Type: QUERY
   - Start Time: 2024-05-21 18:38:32
   - End Time: 2024-05-21 18:38:32
   - Total: 70ms
   - Task State: EOF
   - User: root
   - Default Db: tpch_sf1
   - Sql Statement: select * from region
   - Physical Plan: PhysicalResultSink[54] ( outputExprs=[r_regionkey#0, r_name#1, r_comment#2] )
     +--PhysicalDistribute[51]@0 ( stats=5, distributionSpec=DistributionSpecGather )
        +--PhysicalOlapScan[region]@0 ( stats=5 )

image

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

doris-robot commented May 21, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

englefly commented May 21, 2024

run buildall

@englefly
Copy link
Contributor Author

englefly commented May 21, 2024

run buildall

@doris-robot
Copy link

doris-robot commented May 21, 2024

TPC-H: Total hot run time: 40968 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9922a572b51aacae850c4da0a95c1688f29b43fe, data reload: false

------ Round 1 ----------------------------------
q1	17613	4235	4178	4178
q2	2006	190	183	183
q3	10515	1202	1197	1197
q4	10651	765	809	765
q5	7533	2671	2661	2661
q6	218	132	134	132
q7	1006	591	570	570
q8	9365	2107	2092	2092
q9	9781	7211	7107	7107
q10	9220	3684	3700	3684
q11	466	243	228	228
q12	457	211	209	209
q13	17766	2945	2929	2929
q14	249	224	211	211
q15	502	481	476	476
q16	511	376	377	376
q17	961	657	717	657
q18	7974	7521	7334	7334
q19	1961	1530	1523	1523
q20	643	299	297	297
q21	5021	3896	3987	3896
q22	340	263	281	263
Total cold run time: 114759 ms
Total hot run time: 40968 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4266	4213	4186	4186
q2	356	281	262	262
q3	2946	2721	2699	2699
q4	1850	1511	1541	1511
q5	5294	5269	5265	5265
q6	204	121	126	121
q7	2225	1844	1820	1820
q8	3169	3299	3299	3299
q9	8905	8949	8882	8882
q10	3845	3657	3620	3620
q11	573	492	507	492
q12	752	597	569	569
q13	16511	2939	2967	2939
q14	290	255	261	255
q15	508	475	462	462
q16	481	400	417	400
q17	1759	1494	1453	1453
q18	7665	7577	7361	7361
q19	1705	1559	1519	1519
q20	1962	1770	1797	1770
q21	4966	4778	4837	4778
q22	563	479	485	479
Total cold run time: 70795 ms
Total hot run time: 54142 ms

@doris-robot
Copy link

doris-robot commented May 21, 2024

TPC-DS: Total hot run time: 180486 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9922a572b51aacae850c4da0a95c1688f29b43fe, data reload: false

query1	930	388	368	368
query2	6456	2542	2254	2254
query3	6676	210	219	210
query4	24894	21105	21178	21105
query5	4186	419	423	419
query6	256	173	173	173
query7	4588	298	289	289
query8	261	198	198	198
query9	8514	2409	2378	2378
query10	456	259	278	259
query11	14730	14108	14316	14108
query12	136	91	87	87
query13	1642	377	373	373
query14	9795	7420	7012	7012
query15	223	169	169	169
query16	7901	256	253	253
query17	1892	558	547	547
query18	1999	274	263	263
query19	192	148	146	146
query20	92	87	83	83
query21	197	135	132	132
query22	4976	4764	4861	4764
query23	34376	33757	33545	33545
query24	11889	2888	2829	2829
query25	658	373	359	359
query26	1778	160	154	154
query27	3011	325	314	314
query28	7632	2037	2017	2017
query29	1030	603	607	603
query30	338	171	181	171
query31	988	735	725	725
query32	93	52	56	52
query33	750	263	244	244
query34	1017	489	481	481
query35	807	684	668	668
query36	1081	903	905	903
query37	155	73	70	70
query38	2855	2732	2786	2732
query39	1625	1550	1559	1550
query40	272	130	130	130
query41	48	43	45	43
query42	104	96	97	96
query43	583	546	558	546
query44	1261	724	754	724
query45	269	252	253	252
query46	1083	687	724	687
query47	1941	1864	1881	1864
query48	369	290	290	290
query49	1157	399	397	397
query50	770	383	385	383
query51	6848	6739	6812	6739
query52	108	92	93	92
query53	367	295	289	289
query54	894	431	437	431
query55	77	77	72	72
query56	237	218	216	216
query57	1287	1140	1131	1131
query58	228	204	197	197
query59	3546	3103	3295	3103
query60	251	231	235	231
query61	93	88	88	88
query62	667	493	472	472
query63	308	281	289	281
query64	9803	2257	1755	1755
query65	3177	3075	3125	3075
query66	1391	349	340	340
query67	15354	15065	14988	14988
query68	4603	534	525	525
query69	472	303	298	298
query70	1169	1160	1128	1128
query71	414	266	269	266
query72	7712	2670	2346	2346
query73	700	315	316	315
query74	6535	6214	6216	6214
query75	3479	2674	2600	2600
query76	2814	985	939	939
query77	405	264	264	264
query78	10794	10342	10111	10111
query79	2429	508	517	508
query80	1015	428	427	427
query81	512	243	245	243
query82	714	92	100	92
query83	237	166	168	166
query84	238	86	86	86
query85	1443	293	266	266
query86	448	318	277	277
query87	3294	3140	3145	3140
query88	4037	2339	2321	2321
query89	458	379	380	379
query90	1997	186	185	185
query91	129	99	99	99
query92	62	50	49	49
query93	1557	491	477	477
query94	1231	181	179	179
query95	385	298	300	298
query96	586	269	264	264
query97	3193	3006	3024	3006
query98	234	223	212	212
query99	1126	904	887	887
Total cold run time: 287990 ms
Total hot run time: 180486 ms

@doris-robot
Copy link

doris-robot commented May 21, 2024

ClickBench: Total hot run time: 30.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9922a572b51aacae850c4da0a95c1688f29b43fe, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.24	0.06	0.06
query4	1.68	0.10	0.10
query5	0.50	0.50	0.50
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.52	0.49	0.49
query10	0.55	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.79	0.78	0.77
query15	0.83	0.79	0.80
query16	0.36	0.37	0.35
query17	0.93	0.95	0.96
query18	0.20	0.23	0.22
query19	1.78	1.73	1.63
query20	0.02	0.01	0.01
query21	15.57	0.67	0.66
query22	4.66	7.29	1.83
query23	18.31	1.39	1.28
query24	1.94	0.21	0.21
query25	0.16	0.08	0.09
query26	0.27	0.17	0.18
query27	0.08	0.08	0.07
query28	13.30	1.01	1.00
query29	13.23	3.22	3.26
query30	0.24	0.06	0.07
query31	2.85	0.37	0.39
query32	3.31	0.47	0.48
query33	2.88	2.95	2.86
query34	17.07	4.39	4.40
query35	4.49	4.48	4.45
query36	0.64	0.46	0.46
query37	0.18	0.16	0.15
query38	0.17	0.16	0.13
query39	0.05	0.03	0.03
query40	0.16	0.14	0.15
query41	0.08	0.05	0.04
query42	0.06	0.04	0.05
query43	0.05	0.03	0.04
Total cold run time: 110.38 s
Total hot run time: 30.21 s

@morrySnow
Copy link
Contributor

morrySnow commented May 21, 2024

could u replace screenshot by code block? screeshot is hard to put into git message and not firendly for search engine

@englefly
Copy link
Contributor Author

englefly commented May 22, 2024

run p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 22, 2024
@github-actions
Copy link
Contributor

github-actions bot commented May 22, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented May 22, 2024

PR approved by anyone and no changes requested.

@englefly englefly merged commit 675ac80 into apache:master May 22, 2024
@englefly englefly deleted the prof-plan branch May 22, 2024 06:34
dataroaring pushed a commit that referenced this pull request May 26, 2024
@englefly @dataroaring
[feat](nereids)print physical plan in profile #35144
c853ce9 
print physical plan in profile summary

Summary:
   - Profile ID: f3c4842af5f64dd6-a2b4cd00e16e596f
   - Task Type: QUERY
   - Start Time: 2024-05-21 18:38:32
   - End Time: 2024-05-21 18:38:32
   - Total: 70ms
   - Task State: EOF
   - User: root
   - Default Db: tpch_sf1
   - Sql Statement: select * from region
   - Physical Plan: PhysicalResultSink[54] ( outputExprs=[r_regionkey#0, r_name#1, r_comment#2] )
     +--PhysicalDistribute[51]@0 ( stats=5, distributionSpec=DistributionSpecGather )
        +--PhysicalOlapScan[region]@0 ( stats=5 )
@englefly englefly mentioned this pull request Oct 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@starocean999 starocean999 starocean999 approved these changes

+1 more reviewer

@Mryange Mryange Mryange approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@englefly @doris-robot @morrySnow @gavinchou @starocean999 @Mryange