[chore](dependencies)upgrade hive catalog to 2.0.0 #33424

CalvinKirs · 2024-04-09T08:01:01Z

Changes

Version 2.0.0 of the shade package eliminates potential jar conflicts, resolves dependency component issues, and significantly reduces package size.
Utilize the directly-dependent guava library instead of relying on transitively included libraries.
Adapt paimon 0.7.0

doris-robot · 2024-04-09T08:01:06Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

CalvinKirs · 2024-04-09T08:01:18Z

run buildall

CalvinKirs · 2024-04-09T08:44:25Z

run buildall

doris-robot · 2024-04-09T09:18:16Z

TPC-H: Total hot run time: 49839 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 190cf676194f3ebef6cdf2789a36efeaa77f5367, data reload: false

------ Round 1 ----------------------------------
q1	17636	4357	4305	4305
q2	2054	146	141	141
q3	10424	1873	1914	1873
q4	10358	1276	1317	1276
q5	8470	3915	3965	3915
q6	231	125	123	123
q7	2032	1582	1592	1582
q8	9294	2705	2697	2697
q9	10670	10455	10494	10455
q10	8652	3480	3518	3480
q11	418	237	245	237
q12	470	300	302	300
q13	18368	3961	4009	3961
q14	348	326	319	319
q15	508	446	465	446
q16	722	605	598	598
q17	1115	953	973	953
q18	7288	6850	6882	6850
q19	1674	1551	1527	1527
q20	539	325	302	302
q21	4514	4120	4105	4105
q22	506	417	394	394
Total cold run time: 116291 ms
Total hot run time: 49839 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4304	4283	4279	4279
q2	319	221	219	219
q3	4173	4160	4125	4125
q4	2729	2725	2736	2725
q5	7277	7153	7200	7153
q6	229	124	119	119
q7	3205	2835	2899	2835
q8	4365	4472	4476	4472
q9	17077	17097	16961	16961
q10	4201	4246	4226	4226
q11	771	681	707	681
q12	1033	852	846	846
q13	7059	3725	3758	3725
q14	443	409	421	409
q15	497	459	452	452
q16	769	698	712	698
q17	3863	3937	3814	3814
q18	8752	8856	8702	8702
q19	1719	1690	1638	1638
q20	2401	2129	2088	2088
q21	8640	8568	8513	8513
q22	1004	980	944	944
Total cold run time: 84830 ms
Total hot run time: 79624 ms

doris-robot · 2024-04-09T09:30:49Z

TPC-DS: Total hot run time: 200691 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 190cf676194f3ebef6cdf2789a36efeaa77f5367, data reload: false

query1	917	394	388	388
query2	6558	2067	2106	2067
query3	6929	199	206	199
query4	20038	17968	17953	17953
query5	19731	6470	6501	6470
query6	288	214	226	214
query7	4154	289	297	289
query8	278	248	234	234
query9	3143	2689	2601	2601
query10	403	280	304	280
query11	11392	10686	10796	10686
query12	123	72	69	69
query13	5569	631	623	623
query14	17272	13602	13650	13602
query15	371	222	233	222
query16	6458	292	256	256
query17	1699	1466	853	853
query18	2338	403	400	400
query19	203	144	141	141
query20	76	77	77	77
query21	182	97	99	97
query22	5225	5004	5068	5004
query23	32398	32127	31892	31892
query24	7025	6490	6537	6490
query25	515	411	403	403
query26	531	158	158	158
query27	1906	296	296	296
query28	6112	2275	2238	2238
query29	2933	2680	2692	2680
query30	249	160	163	160
query31	913	721	716	716
query32	65	61	59	59
query33	399	245	231	231
query34	856	461	471	461
query35	1103	890	889	889
query36	1329	1349	1060	1060
query37	87	61	58	58
query38	3073	2945	2914	2914
query39	1355	1327	1323	1323
query40	203	90	87	87
query41	35	32	33	32
query42	82	84	82	82
query43	626	591	572	572
query44	1121	724	720	720
query45	235	233	222	222
query46	1218	964	963	963
query47	1881	1694	1839	1694
query48	974	673	664	664
query49	609	366	366	366
query50	854	646	599	599
query51	4754	4634	4673	4634
query52	89	86	86	86
query53	437	319	305	305
query54	2654	2446	2472	2446
query55	101	69	84	69
query56	213	213	197	197
query57	1252	1084	1057	1057
query58	208	208	204	204
query59	3324	3296	3214	3214
query60	203	189	198	189
query61	87	81	87	81
query62	808	512	487	487
query63	478	331	326	326
query64	2524	1467	1442	1442
query65	3648	3975	3560	3560
query66	759	379	368	368
query67	15401	15028	14953	14953
query68	9609	663	647	647
query69	570	328	322	322
query70	1672	1527	1289	1289
query71	425	298	303	298
query72	6487	3415	3387	3387
query73	733	322	331	322
query74	6316	5867	5807	5807
query75	5389	3707	3720	3707
query76	6093	1120	1181	1120
query77	1043	251	253	251
query78	12619	11516	23040	11516
query79	4958	632	633	632
query80	826	391	384	384
query81	491	227	229	227
query82	265	94	102	94
query83	173	137	147	137
query84	254	68	67	67
query85	839	295	274	274
query86	333	291	294	291
query87	3182	3001	3004	3001
query88	3577	2331	2341	2331
query89	340	275	272	272
query90	1964	192	203	192
query91	153	121	115	115
query92	56	51	52	51
query93	983	537	559	537
query94	706	204	198	198
query95	1108	1057	1052	1052
query96	630	327	318	318
query97	6476	6401	6507	6401
query98	205	183	181	181
query99	2848	945	898	898
Total cold run time: 300297 ms
Total hot run time: 200691 ms

doris-robot · 2024-04-09T09:36:15Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 190cf676194f3ebef6cdf2789a36efeaa77f5367, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.02	0.02
query3	0.25	0.04	0.04
query4	1.84	0.11	0.07
query5	0.53	0.53	0.52
query6	1.24	0.66	0.62
query7	0.01	0.01	0.01
query8	0.03	0.03	0.02
query9	0.53	0.49	0.49
query10	0.54	0.53	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.63	0.61	0.62
query14	0.78	0.78	0.79
query15	0.78	0.77	0.76
query16	0.37	0.39	0.36
query17	1.04	1.01	1.01
query18	0.22	0.27	0.24
query19	1.92	1.85	1.82
query20	0.02	0.01	0.01
query21	15.46	0.54	0.54
query22	2.26	2.57	1.21
query23	17.44	1.01	0.88
query24	4.38	1.32	2.65
query25	0.38	0.11	0.04
query26	0.72	0.15	0.15
query27	0.05	0.04	0.04
query28	6.85	0.73	0.76
query29	12.77	2.29	2.27
query30	0.59	0.51	0.51
query31	2.82	0.38	0.37
query32	3.40	0.49	0.49
query33	3.07	3.07	3.04
query34	15.24	4.80	4.77
query35	4.86	4.87	4.83
query36	1.06	1.00	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.13
query41	0.07	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 102.8 s
Total hot run time: 30.55 s

doris-robot · 2024-04-09T09:39:28Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 190cf676194f3ebef6cdf2789a36efeaa77f5367 with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.6 seconds inserted 10000000 Rows, about 485K ops/s

CalvinKirs · 2024-04-09T10:17:52Z

run buildall

doris-robot · 2024-04-09T10:49:16Z

TPC-H: Total hot run time: 49670 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit db592740d338db6ec614309984f23c6e994b0e90, data reload: false

------ Round 1 ----------------------------------
q1	17615	4363	4340	4340
q2	2053	151	143	143
q3	10448	1875	1920	1875
q4	10344	1216	1324	1216
q5	8469	3881	3894	3881
q6	220	123	125	123
q7	2008	1600	1577	1577
q8	9286	2707	2701	2701
q9	10864	10546	10394	10394
q10	8601	3511	3540	3511
q11	428	240	235	235
q12	454	300	298	298
q13	18358	3957	4021	3957
q14	359	337	339	337
q15	506	461	451	451
q16	688	597	587	587
q17	1127	948	958	948
q18	7352	6904	6845	6845
q19	1680	1553	1509	1509
q20	552	314	304	304
q21	4396	4064	4047	4047
q22	518	391	395	391
Total cold run time: 116326 ms
Total hot run time: 49670 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4307	4341	4278	4278
q2	315	222	224	222
q3	4169	4160	4134	4134
q4	2743	2722	2758	2722
q5	7286	7249	7184	7184
q6	235	122	120	120
q7	3215	2830	2869	2830
q8	4359	4446	4484	4446
q9	17109	16961	17045	16961
q10	4192	4233	4218	4218
q11	782	673	662	662
q12	1036	844	827	827
q13	6795	3719	3725	3719
q14	446	419	422	419
q15	502	458	462	458
q16	753	695	699	695
q17	3849	3875	3767	3767
q18	8847	8844	8753	8753
q19	1712	1709	1627	1627
q20	2354	2153	2123	2123
q21	8528	8472	8594	8472
q22	1046	959	976	959
Total cold run time: 84580 ms
Total hot run time: 79596 ms

doris-robot · 2024-04-09T11:01:52Z

TPC-DS: Total hot run time: 200871 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit db592740d338db6ec614309984f23c6e994b0e90, data reload: false

query1	913	387	381	381
query2	6522	2283	2024	2024
query3	6913	200	197	197
query4	20301	17939	18000	17939
query5	19702	6528	6483	6483
query6	274	213	225	213
query7	4146	304	299	299
query8	259	226	267	226
query9	3162	2708	2645	2645
query10	421	272	294	272
query11	11447	10750	10779	10750
query12	130	79	71	71
query13	5578	639	643	639
query14	17861	13386	13587	13386
query15	383	238	240	238
query16	6467	278	255	255
query17	1717	1462	868	868
query18	2310	402	398	398
query19	202	143	148	143
query20	81	75	75	75
query21	190	93	97	93
query22	5282	4823	5131	4823
query23	32567	31915	31721	31721
query24	6996	6487	6495	6487
query25	523	427	421	421
query26	523	159	158	158
query27	1882	296	298	296
query28	6092	2294	2243	2243
query29	2854	2799	2861	2799
query30	254	164	160	160
query31	905	739	730	730
query32	62	63	59	59
query33	393	253	252	252
query34	851	460	478	460
query35	1104	908	909	908
query36	1352	1235	1338	1235
query37	85	62	57	57
query38	3064	2948	2948	2948
query39	1370	1323	1304	1304
query40	200	96	94	94
query41	35	33	31	31
query42	80	77	78	77
query43	576	567	605	567
query44	1136	730	726	726
query45	245	231	224	224
query46	1228	984	985	984
query47	1799	1726	1875	1726
query48	986	687	670	670
query49	613	357	374	357
query50	859	602	602	602
query51	4746	4613	4626	4613
query52	87	85	75	75
query53	458	323	317	317
query54	2651	2436	2465	2436
query55	80	90	78	78
query56	211	203	197	197
query57	1227	1131	1098	1098
query58	215	200	191	191
query59	3579	3240	3146	3146
query60	198	185	198	185
query61	83	82	83	82
query62	832	509	493	493
query63	461	343	332	332
query64	2546	1505	1408	1408
query65	3644	3549	3544	3544
query66	760	378	372	372
query67	15893	15498	15381	15381
query68	9706	652	683	652
query69	571	354	342	342
query70	2121	1326	1386	1326
query71	411	303	300	300
query72	6376	3412	3396	3396
query73	726	333	327	327
query74	6351	5891	5791	5791
query75	5432	3636	3774	3636
query76	5994	1146	1168	1146
query77	1037	251	251	251
query78	12473	11570	11464	11464
query79	7473	650	661	650
query80	1087	385	390	385
query81	491	233	236	233
query82	1695	104	93	93
query83	170	144	132	132
query84	253	69	66	66
query85	839	280	277	277
query86	337	332	279	279
query87	3275	3024	2950	2950
query88	4882	2381	2372	2372
query89	371	274	282	274
query90	1955	209	203	203
query91	147	117	113	113
query92	53	52	51	51
query93	5992	583	571	571
query94	699	198	198	198
query95	1094	1059	1068	1059
query96	645	331	332	331
query97	6473	6340	6365	6340
query98	189	179	162	162
query99	2949	924	839	839
Total cold run time: 312833 ms
Total hot run time: 200871 ms

doris-robot · 2024-04-09T11:07:15Z

ClickBench: Total hot run time: 30.5 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit db592740d338db6ec614309984f23c6e994b0e90, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.05
query4	1.80	0.07	0.07
query5	0.54	0.54	0.53
query6	1.29	0.61	0.62
query7	0.02	0.02	0.01
query8	0.04	0.02	0.02
query9	0.52	0.47	0.49
query10	0.53	0.55	0.53
query11	0.12	0.08	0.08
query12	0.11	0.08	0.09
query13	0.62	0.62	0.63
query14	0.79	0.79	0.79
query15	0.79	0.76	0.75
query16	0.36	0.37	0.38
query17	1.00	0.98	1.02
query18	0.24	0.25	0.23
query19	1.96	1.82	1.88
query20	0.01	0.01	0.01
query21	15.47	0.55	0.53
query22	2.11	1.95	1.49
query23	16.97	1.03	0.89
query24	5.23	1.23	0.94
query25	0.36	0.06	0.09
query26	0.60	0.16	0.15
query27	0.05	0.04	0.04
query28	7.73	0.78	0.72
query29	12.71	2.32	2.30
query30	0.56	0.53	0.51
query31	2.82	0.38	0.38
query32	3.38	0.50	0.49
query33	3.13	3.08	3.11
query34	15.27	4.81	4.79
query35	4.84	4.82	4.84
query36	1.06	1.02	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.13
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.76 s
Total hot run time: 30.5 s

doris-robot · 2024-04-09T11:10:29Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit db592740d338db6ec614309984f23c6e994b0e90 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.5 seconds inserted 10000000 Rows, about 487K ops/s

CalvinKirs · 2024-04-10T07:50:01Z

run buildall

CalvinKirs · 2024-04-10T08:26:53Z

run buildall

CalvinKirs · 2024-04-11T02:00:31Z

run buildall

CalvinKirs · 2024-04-11T07:13:14Z

run buildall

doris-robot · 2024-04-11T07:52:27Z

TPC-H: Total hot run time: 49883 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a807ba5f71678c2ce98a8d1dda444a133de1a50, data reload: false

------ Round 1 ----------------------------------
q1	17828	4351	4322	4322
q2	2039	150	138	138
q3	10319	1904	1928	1904
q4	10362	1215	1309	1215
q5	8756	3953	4018	3953
q6	229	121	125	121
q7	2011	1585	1570	1570
q8	9250	2697	2702	2697
q9	11133	10586	10626	10586
q10	8677	3531	3531	3531
q11	421	253	248	248
q12	459	302	302	302
q13	18341	3960	4028	3960
q14	352	328	319	319
q15	494	454	451	451
q16	693	588	591	588
q17	1113	940	960	940
q18	7309	6898	6815	6815
q19	1694	1554	1518	1518
q20	527	310	294	294
q21	4515	4029	4070	4029
q22	509	382	400	382
Total cold run time: 117031 ms
Total hot run time: 49883 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4265	4269	4247	4247
q2	319	228	217	217
q3	4140	4154	4139	4139
q4	2739	2715	2744	2715
q5	7292	7188	7188	7188
q6	231	121	116	116
q7	3225	2831	2767	2767
q8	4301	4408	4434	4408
q9	17599	17205	17033	17033
q10	4191	4212	4226	4212
q11	766	676	661	661
q12	1024	870	851	851
q13	6789	3760	3763	3760
q14	457	423	421	421
q15	508	442	460	442
q16	754	699	697	697
q17	3811	3768	3835	3768
q18	8655	8785	8837	8785
q19	1685	1674	1638	1638
q20	2390	2178	2080	2080
q21	8520	8360	8432	8360
q22	1036	938	978	938
Total cold run time: 84697 ms
Total hot run time: 79443 ms

doris-robot · 2024-04-11T08:05:00Z

TPC-DS: Total hot run time: 202530 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a807ba5f71678c2ce98a8d1dda444a133de1a50, data reload: false

query1	914	387	383	383
query2	6539	2242	2280	2242
query3	6917	198	197	197
query4	20083	17874	17983	17874
query5	19707	6562	6539	6539
query6	297	212	229	212
query7	4145	304	296	296
query8	258	226	239	226
query9	3170	2724	2653	2653
query10	435	296	304	296
query11	11393	10714	10627	10627
query12	123	69	71	69
query13	5586	660	632	632
query14	18049	13485	13358	13358
query15	359	225	233	225
query16	6437	280	258	258
query17	1729	1457	874	874
query18	2309	404	405	404
query19	206	141	145	141
query20	79	78	72	72
query21	187	93	95	93
query22	5209	4792	5024	4792
query23	32374	31738	31901	31738
query24	6968	6454	6591	6454
query25	526	429	420	420
query26	525	167	157	157
query27	1851	304	298	298
query28	6123	2282	2231	2231
query29	3072	2627	2798	2627
query30	240	172	163	163
query31	924	775	736	736
query32	66	61	62	61
query33	384	250	251	250
query34	858	479	500	479
query35	1129	952	899	899
query36	1312	1420	1104	1104
query37	102	67	61	61
query38	3129	2957	2988	2957
query39	1381	1328	1309	1309
query40	204	95	92	92
query41	35	33	32	32
query42	83	92	86	86
query43	600	560	589	560
query44	1149	730	723	723
query45	233	223	227	223
query46	1237	971	978	971
query47	1830	1745	1773	1745
query48	1008	706	670	670
query49	626	365	381	365
query50	866	639	614	614
query51	4709	4678	4644	4644
query52	77	80	71	71
query53	443	313	317	313
query54	2684	2459	2417	2417
query55	82	75	83	75
query56	214	198	214	198
query57	1145	1219	1079	1079
query58	216	193	204	193
query59	3440	3391	3293	3293
query60	205	184	184	184
query61	85	82	87	82
query62	835	523	462	462
query63	471	332	334	332
query64	2529	1459	1326	1326
query65	3611	3590	3554	3554
query66	788	380	363	363
query67	17453	17235	16892	16892
query68	5809	655	663	655
query69	538	345	336	336
query70	1399	1376	1365	1365
query71	373	298	300	298
query72	6250	3446	3406	3406
query73	718	323	320	320
query74	6271	5860	5788	5788
query75	4453	3698	3695	3695
query76	3323	1147	1165	1147
query77	557	243	255	243
query78	12964	11648	12546	11648
query79	8693	652	647	647
query80	3461	400	388	388
query81	530	226	230	226
query82	812	95	100	95
query83	257	133	140	133
query84	260	71	68	68
query85	1441	290	276	276
query86	355	279	290	279
query87	3244	3023	3018	3018
query88	4963	2352	2363	2352
query89	362	278	277	277
query90	1751	212	215	212
query91	146	117	116	116
query92	56	53	53	53
query93	4655	568	566	566
query94	893	199	192	192
query95	1105	1066	1050	1050
query96	647	329	345	329
query97	6594	6364	6460	6364
query98	190	179	163	163
query99	3045	859	962	859
Total cold run time: 308068 ms
Total hot run time: 202530 ms

doris-robot · 2024-04-11T08:10:26Z

ClickBench: Total hot run time: 30.83 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a807ba5f71678c2ce98a8d1dda444a133de1a50, data reload: false

query1	0.02	0.03	0.02
query2	0.06	0.02	0.02
query3	0.24	0.04	0.05
query4	1.79	0.07	0.08
query5	0.53	0.53	0.52
query6	1.24	0.62	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.47	0.48
query10	0.53	0.54	0.53
query11	0.12	0.08	0.09
query12	0.11	0.08	0.09
query13	0.62	0.62	0.61
query14	0.79	0.80	0.78
query15	0.79	0.75	0.76
query16	0.36	0.39	0.37
query17	0.97	1.00	1.03
query18	0.22	0.24	0.25
query19	1.94	1.77	1.83
query20	0.02	0.02	0.01
query21	15.45	0.57	0.56
query22	1.92	2.73	1.62
query23	16.87	0.98	0.93
query24	6.08	3.78	1.11
query25	0.38	0.11	0.05
query26	0.92	0.15	0.16
query27	0.04	0.04	0.05
query28	4.52	0.76	0.72
query29	12.64	2.31	2.39
query30	0.56	0.49	0.51
query31	2.82	0.38	0.37
query32	3.42	0.49	0.48
query33	3.05	3.09	3.07
query34	15.24	4.82	4.78
query35	4.88	4.87	4.88
query36	1.07	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.03
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.01
Total cold run time: 101.14 s
Total hot run time: 30.83 s

doris-robot · 2024-04-11T08:13:38Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4a807ba5f71678c2ce98a8d1dda444a133de1a50 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.4 seconds inserted 10000000 Rows, about 490K ops/s

CalvinKirs · 2024-04-11T08:17:14Z

run buildall

doris-robot · 2024-04-12T07:17:21Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 052acdebcacf4e44c561954999ad9827d207cacd with default session variables
Stream load json:         21 seconds loaded 2358488459 Bytes, about 107 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.4 seconds inserted 10000000 Rows, about 490K ops/s

xiaokang · 2024-04-17T13:27:42Z

@CalvinKirs Do you submit this pr to wrong branch?

CalvinKirs · 2024-04-17T14:26:58Z

@CalvinKirs Do you submit this pr to wrong branch?↳

No, this is a 2.0 change, but it is still in WIP status.

CalvinKirs · 2024-04-19T13:50:20Z

run buildall

CalvinKirs · 2024-04-20T00:49:19Z

run buildall

CalvinKirs · 2024-04-20T11:08:06Z

run buildall

CalvinKirs · 2024-04-21T13:00:29Z

run buildall

CalvinKirs · 2024-04-22T02:26:29Z

run buildall

morningman

LGTM

github-actions · 2024-04-22T02:44:40Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-04-22T02:44:42Z

PR approved by anyone and no changes requested.

doris-robot · 2024-04-22T02:51:28Z

TPC-H: Total hot run time: 49988 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3ea63c41948fef8ba000318a9cab259098327926, data reload: false

------ Round 1 ----------------------------------
q1	18087	4431	4347	4347
q2	2032	153	143	143
q3	10272	1909	1956	1909
q4	10340	1280	1334	1280
q5	8402	4066	3963	3963
q6	227	120	126	120
q7	2077	1574	1584	1574
q8	9533	2760	2753	2753
q9	14234	10484	10410	10410
q10	8664	3544	3532	3532
q11	409	247	253	247
q12	461	306	306	306
q13	18333	3948	3998	3948
q14	365	320	333	320
q15	499	453	444	444
q16	690	607	591	591
q17	1135	974	936	936
q18	7383	6860	6890	6860
q19	1724	1592	1508	1508
q20	521	306	308	306
q21	4444	4139	4095	4095
q22	504	412	396	396
Total cold run time: 120336 ms
Total hot run time: 49988 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4306	4348	4317	4317
q2	319	222	223	222
q3	4123	4168	4181	4168
q4	2755	2750	2739	2739
q5	7316	7198	7188	7188
q6	237	122	121	121
q7	3239	2812	2853	2812
q8	4399	4519	4463	4463
q9	17096	17113	16961	16961
q10	4213	4259	4278	4259
q11	762	699	697	697
q12	1030	856	855	855
q13	7072	3730	3746	3730
q14	453	430	419	419
q15	492	462	473	462
q16	749	705	716	705
q17	3841	3856	3920	3856
q18	8819	8716	8717	8716
q19	1742	1747	1686	1686
q20	2407	2178	2120	2120
q21	8644	8497	8519	8497
q22	1056	932	974	932
Total cold run time: 85070 ms
Total hot run time: 79925 ms

doris-robot · 2024-04-22T03:04:17Z

TPC-DS: Total hot run time: 202975 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3ea63c41948fef8ba000318a9cab259098327926, data reload: false

query1	926	391	380	380
query2	6522	2976	2747	2747
query3	6919	196	203	196
query4	21529	17896	18051	17896
query5	19764	6538	6469	6469
query6	282	216	229	216
query7	4150	297	296	296
query8	262	271	236	236
query9	3147	2731	2639	2639
query10	411	288	307	288
query11	11398	10673	10684	10673
query12	129	77	70	70
query13	5575	706	676	676
query14	17629	13971	13420	13420
query15	360	229	236	229
query16	6453	269	256	256
query17	1726	1446	872	872
query18	2315	409	401	401
query19	206	144	141	141
query20	78	77	70	70
query21	181	89	94	89
query22	5010	4971	5034	4971
query23	32420	31934	31861	31861
query24	7764	6526	6510	6510
query25	517	447	432	432
query26	1246	166	158	158
query27	2231	299	291	291
query28	6145	2361	2313	2313
query29	2902	2763	2761	2761
query30	238	163	160	160
query31	914	724	737	724
query32	65	64	55	55
query33	427	249	248	248
query34	852	476	496	476
query35	1114	867	878	867
query36	1242	1103	1318	1103
query37	92	62	60	60
query38	3088	2932	2911	2911
query39	1369	1314	1319	1314
query40	303	96	95	95
query41	37	35	35	35
query42	78	87	79	79
query43	791	700	676	676
query44	1153	724	734	724
query45	241	229	228	228
query46	1245	979	974	974
query47	1808	1765	1668	1668
query48	1040	701	727	701
query49	644	363	369	363
query50	872	557	596	557
query51	4758	4743	4623	4623
query52	86	79	73	73
query53	443	314	310	310
query54	2651	2468	2471	2468
query55	83	73	79	73
query56	205	204	188	188
query57	1227	1083	1049	1049
query58	210	194	204	194
query59	4148	3988	4181	3988
query60	219	180	204	180
query61	85	80	80	80
query62	834	475	487	475
query63	477	326	325	325
query64	5163	1551	1447	1447
query65	3663	3588	3571	3571
query66	677	382	372	372
query67	16676	16346	15517	15517
query68	9114	676	670	670
query69	550	321	341	321
query70	1635	1436	1507	1436
query71	404	301	302	301
query72	6189	3424	3393	3393
query73	731	335	326	326
query74	6241	5827	5797	5797
query75	5366	3587	3705	3587
query76	5641	1133	1170	1133
query77	929	249	241	241
query78	12579	11592	12581	11592
query79	10571	643	662	643
query80	1147	401	381	381
query81	491	231	224	224
query82	1103	96	89	89
query83	168	126	126	126
query84	256	67	71	67
query85	855	291	294	291
query86	330	293	319	293
query87	3255	3014	3067	3014
query88	4770	2316	2309	2309
query89	426	284	294	284
query90	1941	208	204	204
query91	179	145	136	136
query92	59	50	48	48
query93	5613	607	596	596
query94	703	207	195	195
query95	1115	1060	1066	1060
query96	633	340	311	311
query97	6408	6503	6335	6335
query98	188	172	167	167
query99	2924	982	897	897
Total cold run time: 319929 ms
Total hot run time: 202975 ms

doris-robot · 2024-04-22T03:09:43Z

ClickBench: Total hot run time: 30.61 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3ea63c41948fef8ba000318a9cab259098327926, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.02	0.01
query3	0.24	0.05	0.05
query4	1.79	0.06	0.06
query5	0.53	0.53	0.52
query6	1.32	0.61	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.47
query10	0.53	0.53	0.55
query11	0.12	0.08	0.09
query12	0.11	0.08	0.09
query13	0.62	0.62	0.60
query14	0.79	0.80	0.78
query15	0.77	0.74	0.75
query16	0.35	0.38	0.37
query17	1.02	0.99	1.00
query18	0.25	0.24	0.25
query19	1.91	1.84	1.89
query20	0.02	0.01	0.00
query21	15.48	0.57	0.56
query22	2.38	2.68	1.73
query23	17.13	0.94	0.91
query24	4.84	0.96	2.84
query25	0.38	0.10	0.05
query26	0.68	0.16	0.16
query27	0.04	0.03	0.04
query28	6.52	0.73	0.70
query29	12.63	2.25	2.16
query30	0.58	0.53	0.53
query31	2.81	0.39	0.38
query32	3.38	0.50	0.50
query33	3.06	3.06	3.06
query34	15.24	4.80	4.77
query35	4.85	4.84	4.84
query36	1.04	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.13
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 102.44 s
Total hot run time: 30.61 s

doris-robot · 2024-04-22T03:12:55Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3ea63c41948fef8ba000318a9cab259098327926 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

Utilize the directly-dependent guava library instead of relying on transitively included libraries. Adapt paimon 0.6.0

[feat](dependency)hivecatalog slimming down

5effec7

CalvinKirs changed the title ~~[feat](dependency)hivecatalog slimming down~~ [WIP](dependency)hivecatalog slimming down Apr 9, 2024

CalvinKirs added the not-merge/2.0 do not merge into 2.0 branch label Apr 9, 2024

[feat](dependency)hivecatalog slimming down

190cf67

github-actions bot added area/nereids area/planner Issues or PRs related to the query planner labels Apr 9, 2024

[feat](dependency)hivecatalog slimming down

db59274

Just for test

f1beb05

Just for test

4a807ba

xiaokang marked this pull request as draft April 17, 2024 13:27

replace release version

0147981

CalvinKirs removed the not-merge/2.0 do not merge into 2.0 branch label Apr 19, 2024

CalvinKirs changed the title ~~[WIP](dependency)hivecatalog slimming down~~ [chore](dependencies)upgrade hive catalog to 2.0.0 Apr 19, 2024

CalvinKirs marked this pull request as ready for review April 19, 2024 12:14

Merge branch 'branch-2.0' into branch-2.0-catalog

1fa5cfa

CalvinKirs and others added 2 commits April 22, 2024 10:25

Adapt paimon 0.6.0

8f783d2

Merge branch 'branch-2.0' into branch-2.0-catalog

3ea63c4

morningman approved these changes Apr 22, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 22, 2024

github-actions bot added the reviewed label Apr 22, 2024

CalvinKirs mentioned this pull request Apr 22, 2024

[chore](dependencies)upgrade hive catalog to 2.0.0 #33943

Merged

CalvinKirs merged commit 54f5d13 into apache:branch-2.0 Apr 22, 2024

CalvinKirs deleted the branch-2.0-catalog branch April 22, 2024 03:30

morningman pushed a commit to morningman/doris that referenced this pull request Apr 24, 2024

[chore](dependencies)upgrade hive catalog to 2.0.0 (apache#33424)

5c8af11

Utilize the directly-dependent guava library instead of relying on transitively included libraries. Adapt paimon 0.6.0

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[chore](dependencies)upgrade hive catalog to 2.0.0 (apache#33424)

a418b85

Utilize the directly-dependent guava library instead of relying on transitively included libraries. Adapt paimon 0.6.0

[chore](dependencies)upgrade hive catalog to 2.0.0 #33424

[chore](dependencies)upgrade hive catalog to 2.0.0 #33424

Uh oh!

Conversation

CalvinKirs commented Apr 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

CalvinKirs commented Apr 9, 2024

Uh oh!

CalvinKirs commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

CalvinKirs commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

doris-robot commented Apr 9, 2024

Uh oh!

CalvinKirs commented Apr 10, 2024

Uh oh!

CalvinKirs commented Apr 10, 2024

Uh oh!

CalvinKirs commented Apr 11, 2024

Uh oh!

CalvinKirs commented Apr 11, 2024

Uh oh!

doris-robot commented Apr 11, 2024

Uh oh!

doris-robot commented Apr 11, 2024

Uh oh!

doris-robot commented Apr 11, 2024

Uh oh!

doris-robot commented Apr 11, 2024

Uh oh!

CalvinKirs commented Apr 11, 2024

Uh oh!

doris-robot commented Apr 12, 2024

Uh oh!

xiaokang commented Apr 17, 2024

Uh oh!

CalvinKirs commented Apr 17, 2024

Uh oh!

CalvinKirs commented Apr 19, 2024

Uh oh!

CalvinKirs commented Apr 20, 2024

Uh oh!

CalvinKirs commented Apr 20, 2024

Uh oh!

CalvinKirs commented Apr 21, 2024

Uh oh!

CalvinKirs commented Apr 22, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 22, 2024

Uh oh!

github-actions bot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

Uh oh!

doris-robot commented Apr 22, 2024

CalvinKirs commented Apr 9, 2024 •

edited

Loading