Skip to content

[Improvement](profile) add catalog info in profile #38298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
liutang123 merged 1 commit into from
Jul 26, 2024
Merged

[Improvement](profile) add catalog info in profile #38298

liutang123 merged 1 commit into from
Jul 26, 2024

Conversation

@Yulei-Yang
Copy link
Contributor

@Yulei-Yang Yulei-Yang commented Jul 24, 2024

bp #38283

@doris-robot
Copy link

doris-robot commented Jul 24, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yulei-Yang
Copy link
Contributor Author

Yulei-Yang commented Jul 24, 2024

run buildall

@github-actions github-actions bot added area/load Issues or PRs related to all kinds of load kind/test labels Jul 24, 2024
@doris-robot
Copy link

doris-robot commented Jul 24, 2024

TPC-H: Total hot run time: 49948 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e, data reload: false

------ Round 1 ----------------------------------
q1	17641	4364	4353	4353
q2	2094	156	144	144
q3	10469	1919	1950	1919
q4	10327	1236	1325	1236
q5	8404	3862	3842	3842
q6	235	125	124	124
q7	2045	1624	1608	1608
q8	9289	2709	2675	2675
q9	10517	10255	10313	10255
q10	8624	3504	3496	3496
q11	418	250	242	242
q12	472	307	304	304
q13	18334	3987	4007	3987
q14	356	324	320	320
q15	494	460	454	454
q16	683	569	568	568
q17	1127	957	987	957
q18	7272	7053	6983	6983
q19	1787	1643	1597	1597
q20	538	322	311	311
q21	4523	4227	4096	4096
q22	539	477	484	477
Total cold run time: 116188 ms
Total hot run time: 49948 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4427	4460	4420	4420
q2	330	236	234	234
q3	4204	4172	4139	4139
q4	2749	2749	2745	2745
q5	7126	7088	7117	7088
q6	241	122	118	118
q7	3163	2820	2847	2820
q8	4368	4457	4479	4457
q9	16784	16884	16706	16706
q10	4257	4229	4298	4229
q11	751	709	689	689
q12	1011	852	854	852
q13	7046	3733	3718	3718
q14	448	424	431	424
q15	491	456	456	456
q16	734	687	681	681
q17	3860	3834	3793	3793
q18	8710	8702	8728	8702
q19	1721	1679	1607	1607
q20	2363	2125	2097	2097
q21	8548	8421	8547	8421
q22	1041	967	961	961
Total cold run time: 84373 ms
Total hot run time: 79357 ms

@doris-robot
Copy link

doris-robot commented Jul 24, 2024

TPC-DS: Total hot run time: 204763 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e, data reload: false

query1	936	421	385	385
query2	6554	3004	2940	2940
query3	6914	211	214	211
query4	20195	17999	18012	17999
query5	19740	6516	6476	6476
query6	274	212	225	212
query7	4153	293	310	293
query8	420	462	439	439
query9	3152	2722	2665	2665
query10	408	316	294	294
query11	11420	10674	10736	10674
query12	131	72	80	72
query13	5602	701	697	697
query14	17579	13370	13814	13370
query15	372	254	230	230
query16	6489	282	260	260
query17	1712	1478	880	880
query18	2312	415	411	411
query19	216	157	154	154
query20	82	83	83	83
query21	193	98	94	94
query22	5337	5014	5065	5014
query23	32527	32105	31977	31977
query24	7039	6452	6495	6452
query25	517	423	433	423
query26	534	167	162	162
query27	1870	301	299	299
query28	6143	2386	2318	2318
query29	2920	2728	2945	2728
query30	244	174	171	171
query31	932	730	764	730
query32	70	63	61	61
query33	398	247	253	247
query34	849	467	491	467
query35	1093	920	894	894
query36	1333	1039	1106	1039
query37	89	59	62	59
query38	3043	2936	2914	2914
query39	1365	1326	1304	1304
query40	203	99	92	92
query41	49	43	45	43
query42	91	81	86	81
query43	734	767	837	767
query44	1126	715	711	711
query45	247	233	226	226
query46	1239	968	985	968
query47	2262	1837	1689	1689
query48	1028	703	731	703
query49	636	372	365	365
query50	847	618	644	618
query51	4745	4699	4760	4699
query52	93	79	82	79
query53	449	320	319	319
query54	2652	2431	2464	2431
query55	86	75	81	75
query56	236	213	222	213
query57	1185	1224	1076	1076
query58	223	210	206	206
query59	4229	4193	4269	4193
query60	217	222	206	206
query61	95	91	95	91
query62	843	471	491	471
query63	489	348	351	348
query64	2584	1547	1525	1525
query65	3626	3561	3571	3561
query66	777	394	372	372
query67	17296	15616	15673	15616
query68	9099	641	676	641
query69	591	352	369	352
query70	1635	1444	1576	1444
query71	419	316	318	316
query72	6666	3456	3463	3456
query73	742	331	322	322
query74	6260	5991	5784	5784
query75	5489	3739	3679	3679
query76	5473	1145	1232	1145
query77	950	261	261	261
query78	12455	11720	11760	11720
query79	7008	641	622	622
query80	989	403	401	401
query81	493	237	234	234
query82	1498	104	99	99
query83	163	135	132	132
query84	253	69	72	69
query85	885	333	340	333
query86	336	290	300	290
query87	3302	3042	3015	3015
query88	4868	2353	2347	2347
query89	360	288	315	288
query90	1949	214	226	214
query91	175	138	145	138
query92	58	54	54	54
query93	3680	588	564	564
query94	707	210	217	210
query95	1106	1081	1050	1050
query96	659	340	332	332
query97	6390	6338	6346	6338
query98	200	178	167	167
query99	2980	926	904	904
Total cold run time: 311282 ms
Total hot run time: 204763 ms

@doris-robot
Copy link

doris-robot commented Jul 24, 2024

ClickBench: Total hot run time: 30.79 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e, data reload: false

query1	0.02	0.02	0.03
query2	0.08	0.03	0.02
query3	0.25	0.05	0.05
query4	1.80	0.06	0.06
query5	0.53	0.52	0.52
query6	1.23	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.54	0.48	0.48
query10	0.54	0.54	0.52
query11	0.13	0.09	0.09
query12	0.12	0.09	0.09
query13	0.63	0.63	0.61
query14	0.79	0.77	0.80
query15	0.78	0.76	0.76
query16	0.39	0.37	0.38
query17	1.02	1.03	1.01
query18	0.23	0.25	0.26
query19	1.94	1.82	1.80
query20	0.02	0.01	0.01
query21	15.92	0.55	0.57
query22	1.96	2.43	1.50
query23	17.15	0.92	0.94
query24	6.47	1.12	1.12
query25	0.40	0.07	0.06
query26	0.70	0.16	0.17
query27	0.04	0.03	0.04
query28	6.43	0.77	0.76
query29	12.75	2.36	2.30
query30	0.63	0.49	0.51
query31	2.81	0.39	0.38
query32	3.36	0.51	0.49
query33	3.05	3.12	3.06
query34	15.26	4.82	4.80
query35	4.85	4.83	4.85
query36	1.07	1.02	1.02
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.17	0.15	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.35 s
Total hot run time: 30.79 s

@doris-robot
Copy link

doris-robot commented Jul 24, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1d785a78e1f0fb948f448ab50eac873f7152550e with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

nextdreamblue
nextdreamblue approved these changes Jul 25, 2024
View reviewed changes
Copy link
Contributor

@nextdreamblue nextdreamblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 25, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jul 25, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 25, 2024

PR approved by anyone and no changes requested.

liutang123
liutang123 approved these changes Jul 25, 2024
View reviewed changes
Copy link
Contributor

@liutang123 liutang123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liutang123 liutang123 merged commit bdbbcb7 into apache:branch-2.0 Jul 26, 2024
@Yulei-Yang Yulei-Yang deleted the add_catalog_for_profile_for_v2 branch July 26, 2024 02:28
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@Yulei-Yang
[Improvement](profile) add catalog info in profile (apache#38298)
63dfe3a 
bp apache#38283
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liutang123 liutang123 liutang123 approved these changes

@nextdreamblue nextdreamblue nextdreamblue approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. area/load Issues or PRs related to all kinds of load kind/test reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Yulei-Yang @doris-robot @liutang123 @nextdreamblue