Skip to content

Conversation

@Yulei-Yang
Copy link
Contributor

@Yulei-Yang Yulei-Yang commented Jun 25, 2024

lakefs is a inner filesystem used in tencent dlc, it's a simple wrap of tencent cosn.

docs: apache/doris-website#887

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yulei-Yang
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39667 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1c00bd8bad739c8da04edb64805f8b174626539d, data reload: false

------ Round 1 ----------------------------------
q1	18152	4527	4454	4454
q2	2507	199	186	186
q3	11673	1093	1032	1032
q4	10997	744	745	744
q5	7685	2699	2655	2655
q6	225	137	137	137
q7	965	610	619	610
q8	9239	2085	2049	2049
q9	8787	6519	6464	6464
q10	8927	3707	3736	3707
q11	443	242	232	232
q12	485	215	222	215
q13	18016	2957	2973	2957
q14	274	215	218	215
q15	517	479	475	475
q16	505	369	364	364
q17	952	704	618	618
q18	8108	7385	7346	7346
q19	3123	1451	1424	1424
q20	651	323	316	316
q21	5182	3132	3912	3132
q22	406	342	335	335
Total cold run time: 117819 ms
Total hot run time: 39667 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4372	4291	4283	4283
q2	363	262	269	262
q3	2966	2774	2734	2734
q4	1903	1610	1652	1610
q5	5273	5287	5284	5284
q6	217	131	127	127
q7	2145	1705	1765	1705
q8	3172	3319	3300	3300
q9	8266	8289	8306	8289
q10	3834	3610	3641	3610
q11	580	477	481	477
q12	769	629	624	624
q13	17477	2971	2965	2965
q14	280	267	260	260
q15	532	467	481	467
q16	480	413	408	408
q17	1742	1479	1464	1464
q18	7634	7410	7372	7372
q19	1703	1646	1568	1568
q20	1979	1796	1805	1796
q21	5015	4727	4588	4588
q22	648	558	546	546
Total cold run time: 71350 ms
Total hot run time: 53739 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170695 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1c00bd8bad739c8da04edb64805f8b174626539d, data reload: false

query1	924	400	399	399
query2	6444	2461	2334	2334
query3	6647	203	200	200
query4	18634	17316	17398	17316
query5	4165	493	475	475
query6	247	153	161	153
query7	4604	300	292	292
query8	331	323	297	297
query9	8637	2398	2385	2385
query10	599	283	281	281
query11	10552	10092	10223	10092
query12	133	80	81	80
query13	1627	400	372	372
query14	10207	7342	6942	6942
query15	267	188	195	188
query16	8112	270	262	262
query17	1908	551	527	527
query18	2050	284	275	275
query19	196	151	155	151
query20	90	83	80	80
query21	209	127	122	122
query22	4522	4178	4270	4178
query23	33751	33053	33072	33053
query24	11849	2928	2877	2877
query25	668	384	362	362
query26	1724	149	156	149
query27	2918	308	313	308
query28	7516	2091	2104	2091
query29	1018	605	624	605
query30	286	150	147	147
query31	943	735	734	734
query32	97	54	54	54
query33	768	283	285	283
query34	976	462	466	462
query35	737	638	612	612
query36	1106	946	929	929
query37	293	71	70	70
query38	2874	2778	2767	2767
query39	877	801	791	791
query40	269	128	125	125
query41	53	52	52	52
query42	122	102	98	98
query43	568	542	534	534
query44	1168	749	747	747
query45	193	162	162	162
query46	1081	712	725	712
query47	1857	1795	1791	1791
query48	365	299	297	297
query49	1197	426	417	417
query50	772	415	388	388
query51	6898	6755	6803	6755
query52	106	90	97	90
query53	366	300	292	292
query54	984	440	448	440
query55	75	74	74	74
query56	282	265	271	265
query57	1183	1056	1053	1053
query58	255	248	244	244
query59	3603	3106	3093	3093
query60	296	280	282	280
query61	94	95	122	95
query62	652	446	468	446
query63	318	290	292	290
query64	9821	2246	1744	1744
query65	3197	3099	3090	3090
query66	1368	336	339	336
query67	15505	14948	14981	14948
query68	4595	545	548	545
query69	550	422	392	392
query70	1169	1117	1168	1117
query71	408	273	271	271
query72	7093	2762	2631	2631
query73	763	331	322	322
query74	5896	5563	5570	5563
query75	3611	2673	2667	2667
query76	3144	980	929	929
query77	661	305	301	301
query78	10481	9898	9721	9721
query79	2398	511	521	511
query80	861	477	474	474
query81	577	219	222	219
query82	785	107	100	100
query83	266	172	166	166
query84	241	85	92	85
query85	1578	291	277	277
query86	492	321	315	315
query87	3282	3082	3087	3082
query88	3812	2394	2378	2378
query89	484	394	378	378
query90	1806	197	197	197
query91	142	114	111	111
query92	63	52	51	51
query93	2221	519	520	519
query94	1285	190	185	185
query95	415	308	321	308
query96	591	272	270	270
query97	3198	3047	3014	3014
query98	235	197	207	197
query99	1264	846	866	846
Total cold run time: 276521 ms
Total hot run time: 170695 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1c00bd8bad739c8da04edb64805f8b174626539d, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.70	0.07	0.09
query5	0.49	0.52	0.49
query6	1.14	0.71	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.54	0.53
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.59	0.58	0.58
query14	0.76	0.77	0.80
query15	0.84	0.81	0.81
query16	0.35	0.38	0.36
query17	1.03	1.00	1.01
query18	0.23	0.24	0.23
query19	1.79	1.67	1.72
query20	0.01	0.01	0.01
query21	15.45	0.65	0.66
query22	3.85	8.01	1.77
query23	18.29	1.49	1.42
query24	2.09	0.25	0.24
query25	0.14	0.08	0.08
query26	0.28	0.17	0.18
query27	0.08	0.09	0.08
query28	13.19	1.03	1.00
query29	12.59	3.31	3.29
query30	0.25	0.06	0.06
query31	2.89	0.39	0.38
query32	3.28	0.46	0.46
query33	2.83	2.92	2.83
query34	17.25	4.40	4.39
query35	4.48	4.50	4.46
query36	0.64	0.47	0.48
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.10	0.04	0.04
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.05 s
Total hot run time: 30.46 s

@Yulei-Yang
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39538 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 39786816e0cc3729006d0f588a8a5f8e6a693f29, data reload: false

------ Round 1 ----------------------------------
q1	17605	4318	4256	4256
q2	2006	185	188	185
q3	10481	1099	1121	1099
q4	10178	875	707	707
q5	7444	2584	2632	2584
q6	223	133	133	133
q7	945	596	595	595
q8	9241	2047	2045	2045
q9	8866	6546	6435	6435
q10	8951	3692	3715	3692
q11	442	233	228	228
q12	477	219	232	219
q13	17771	2975	2990	2975
q14	274	222	218	218
q15	510	481	480	480
q16	511	370	371	370
q17	950	637	675	637
q18	7968	7498	7398	7398
q19	3829	1529	1439	1439
q20	638	318	312	312
q21	4925	3194	3958	3194
q22	389	337	340	337
Total cold run time: 114624 ms
Total hot run time: 39538 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4349	4299	4256	4256
q2	360	273	265	265
q3	3009	2715	2865	2715
q4	1977	1673	1751	1673
q5	5461	5602	5419	5419
q6	233	126	125	125
q7	2200	1782	1826	1782
q8	3234	3406	3365	3365
q9	8568	8634	8698	8634
q10	4101	3832	3852	3832
q11	594	495	502	495
q12	798	622	635	622
q13	17111	3175	3150	3150
q14	290	288	267	267
q15	534	508	495	495
q16	487	438	413	413
q17	1807	1530	1477	1477
q18	8070	7870	7718	7718
q19	2373	1652	1642	1642
q20	2145	1864	1881	1864
q21	5064	4852	4955	4852
q22	618	536	552	536
Total cold run time: 73383 ms
Total hot run time: 55597 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174088 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 39786816e0cc3729006d0f588a8a5f8e6a693f29, data reload: false

query1	945	406	367	367
query2	6441	2334	2375	2334
query3	6626	219	205	205
query4	19511	17079	17371	17079
query5	3565	476	479	476
query6	240	157	158	157
query7	4587	303	305	303
query8	312	299	296	296
query9	8496	2390	2365	2365
query10	584	331	299	299
query11	10455	9960	9993	9960
query12	113	81	83	81
query13	1638	368	386	368
query14	9415	7800	7094	7094
query15	233	188	193	188
query16	7754	283	274	274
query17	1860	555	541	541
query18	1959	285	288	285
query19	197	166	163	163
query20	93	85	85	85
query21	207	131	140	131
query22	4189	4084	4069	4069
query23	33902	33705	33691	33691
query24	8963	2887	2836	2836
query25	616	422	400	400
query26	724	165	163	163
query27	2238	327	334	327
query28	5891	2139	2183	2139
query29	896	671	634	634
query30	257	153	147	147
query31	986	797	785	785
query32	95	54	58	54
query33	624	316	301	301
query34	897	486	491	486
query35	777	627	632	627
query36	1115	993	1003	993
query37	151	79	79	79
query38	3001	2815	2820	2815
query39	913	872	834	834
query40	223	134	131	131
query41	56	53	55	53
query42	111	101	98	98
query43	595	573	543	543
query44	1115	730	731	730
query45	195	164	167	164
query46	1073	731	737	731
query47	1848	1777	1776	1776
query48	389	316	295	295
query49	836	422	420	420
query50	780	384	393	384
query51	6877	6774	6845	6774
query52	111	92	97	92
query53	364	294	304	294
query54	902	453	442	442
query55	73	72	74	72
query56	286	268	268	268
query57	1161	1056	1033	1033
query58	241	258	261	258
query59	3310	3241	3115	3115
query60	315	272	277	272
query61	99	95	95	95
query62	606	450	451	450
query63	323	300	300	300
query64	8525	2260	1783	1783
query65	3184	3097	3074	3074
query66	742	336	337	336
query67	15317	14927	14836	14836
query68	4736	553	565	553
query69	649	434	404	404
query70	1207	1147	1138	1138
query71	434	277	290	277
query72	8944	5294	5523	5294
query73	776	335	323	323
query74	5914	5475	5540	5475
query75	3957	2629	2682	2629
query76	3069	928	945	928
query77	734	307	318	307
query78	10479	9902	9931	9902
query79	2822	522	539	522
query80	1108	475	483	475
query81	559	221	218	218
query82	1426	108	114	108
query83	490	172	178	172
query84	277	94	87	87
query85	1369	301	279	279
query86	465	335	289	289
query87	3264	3079	3047	3047
query88	4024	2372	2369	2369
query89	483	384	394	384
query90	1810	194	192	192
query91	185	101	102	101
query92	59	48	50	48
query93	2713	524	527	524
query94	1202	192	186	186
query95	409	314	328	314
query96	596	281	271	271
query97	3212	3059	3019	3019
query98	218	201	193	193
query99	1179	810	846	810
Total cold run time: 269892 ms
Total hot run time: 174088 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 39786816e0cc3729006d0f588a8a5f8e6a693f29, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.66	0.07	0.07
query5	0.51	0.49	0.50
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.54	0.49	0.48
query10	0.54	0.53	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.13
query13	0.60	0.58	0.59
query14	0.75	0.83	0.79
query15	0.85	0.80	0.82
query16	0.36	0.36	0.35
query17	1.05	1.02	1.04
query18	0.20	0.24	0.25
query19	1.82	1.81	1.75
query20	0.01	0.00	0.01
query21	15.43	0.68	0.64
query22	4.09	6.39	2.38
query23	18.24	1.44	1.31
query24	2.13	0.23	0.22
query25	0.15	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.08	0.09
query28	13.30	1.02	0.99
query29	12.64	3.34	3.29
query30	0.25	0.06	0.05
query31	2.86	0.40	0.38
query32	3.29	0.46	0.48
query33	2.92	2.89	2.92
query34	17.22	4.37	4.38
query35	4.54	4.50	4.46
query36	0.65	0.48	0.47
query37	0.17	0.16	0.15
query38	0.15	0.14	0.14
query39	0.05	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.51 s
Total hot run time: 31.05 s

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 27, 2024
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lide-reed lide-reed merged commit 507dfd8 into apache:master Jun 27, 2024
@Yulei-Yang Yulei-Yang deleted the support_iceberg_on_lakefs_for_master branch June 27, 2024 04:03
morningman pushed a commit to apache/doris-website that referenced this pull request Jul 23, 2024
doris pr: apache/doris#36823

---------

Co-authored-by: yuleiyang <yuleiyang@tencent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants