[improve](cloud) use compute group instead of cluster and remove cloud_instance_id #40767

dataroaring · 2024-09-12T13:06:57Z

use compute group instead of cloud cluster.
use cluster instead of cloud_instance.
simply be config by passing cloud_unique_id to be, we add a config item (enable_use_cloud_unique_id_from_fe) in be in order to handle cloud_unique_id is different in ms sometimes, it should not happen.
opt sync_vault_info by sync from meta when vault id is empty.
use show storage vaults instead of show storage vault.
only first fe can be start from empty, otherwise multi master may start.
cancel decommission is not supported in cloud mode.

doris-robot · 2024-09-12T13:07:02Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

dataroaring · 2024-09-12T18:15:17Z

run buildall

doris-robot · 2024-09-12T19:55:43Z

TeamCity be ut coverage result:
Function Coverage: 36.89% (9463/25649)
Line Coverage: 28.25% (77793/275387)
Region Coverage: 27.67% (40193/145263)
Branch Coverage: 24.28% (20426/84130)
Coverage Report: http://coverage.selectdb-in.cc/coverage/97920ea14ca7cd2e15e9049cf704e32339849ce4_97920ea14ca7cd2e15e9049cf704e32339849ce4/report/index.html

dataroaring · 2024-09-13T05:44:21Z

run buildall

doris-robot · 2024-09-13T07:30:43Z

TeamCity be ut coverage result:
Function Coverage: 36.89% (9462/25650)
Line Coverage: 28.25% (77799/275402)
Region Coverage: 27.66% (40179/145278)
Branch Coverage: 24.26% (20412/84144)
Coverage Report: http://coverage.selectdb-in.cc/coverage/01bfca78bd31713b72b8898136ce41af854d7f55_01bfca78bd31713b72b8898136ce41af854d7f55/report/index.html

dataroaring · 2024-09-13T15:19:52Z

run buildall

doris-robot · 2024-09-13T16:52:04Z

TeamCity be ut coverage result:
Function Coverage: 36.89% (9465/25654)
Line Coverage: 28.26% (77837/275424)
Region Coverage: 27.66% (40192/145293)
Branch Coverage: 24.27% (20421/84138)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d1b77bb1ffe34a4a50941f5ed5fb7f11bf59b4ce_d1b77bb1ffe34a4a50941f5ed5fb7f11bf59b4ce/report/index.html

dataroaring · 2024-09-14T03:04:56Z

run buildall

doris-robot · 2024-09-14T04:53:52Z

TeamCity be ut coverage result:
Function Coverage: 36.92% (9474/25663)
Line Coverage: 28.28% (77913/275484)
Region Coverage: 27.69% (40234/145318)
Branch Coverage: 24.28% (20435/84154)
Coverage Report: http://coverage.selectdb-in.cc/coverage/33d0592ab0e370aae9afb645bab3b55bafc8e04d_33d0592ab0e370aae9afb645bab3b55bafc8e04d/report/index.html

dataroaring · 2024-09-14T07:03:34Z

run buildall

doris-robot · 2024-09-14T08:33:12Z

TPC-H: Total hot run time: 43230 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d32d9746789d1c085f4b73f84de4b734585c6b6, data reload: false

------ Round 1 ----------------------------------
q1	17748	7400	7171	7171
q2	2044	197	185	185
q3	10515	1409	1438	1409
q4	10258	973	1048	973
q5	7747	3276	3190	3190
q6	243	154	153	153
q7	1050	653	615	615
q8	9468	2064	2067	2064
q9	6869	6340	6358	6340
q10	7033	2581	2574	2574
q11	443	257	254	254
q12	416	233	230	230
q13	17771	3020	3005	3005
q14	283	252	253	252
q15	570	541	525	525
q16	522	436	428	428
q17	1003	958	961	958
q18	7531	6837	6757	6757
q19	1374	1255	1237	1237
q20	610	338	328	328
q21	3955	3567	3606	3567
q22	1108	1051	1015	1015
Total cold run time: 108561 ms
Total hot run time: 43230 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7325	7153	7200	7153
q2	335	230	226	226
q3	2961	2895	2917	2895
q4	1982	1995	1991	1991
q5	5468	5462	5485	5462
q6	232	142	142	142
q7	2045	1693	1685	1685
q8	3277	3357	3352	3352
q9	8488	8522	8504	8504
q10	3430	3498	3473	3473
q11	586	466	482	466
q12	778	601	598	598
q13	7791	3037	3035	3035
q14	298	274	264	264
q15	570	527	539	527
q16	495	452	449	449
q17	1774	1712	1743	1712
q18	8122	7785	7584	7584
q19	1723	1714	1710	1710
q20	2056	1806	1812	1806
q21	5774	5509	5547	5509
q22	1130	1018	1043	1018
Total cold run time: 66640 ms
Total hot run time: 59561 ms

dataroaring · 2024-09-14T08:35:46Z

run buildall

doris-robot · 2024-09-14T08:57:45Z

TeamCity be ut coverage result:
Function Coverage: 36.91% (9473/25663)
Line Coverage: 28.28% (77906/275493)
Region Coverage: 27.69% (40240/145321)
Branch Coverage: 24.29% (20439/84154)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9b91463f3dc3771a7225fbfe404bf284bcc83ff7_9b91463f3dc3771a7225fbfe404bf284bcc83ff7/report/index.html

doris-robot · 2024-09-14T09:10:01Z

TPC-H: Total hot run time: 43126 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b91463f3dc3771a7225fbfe404bf284bcc83ff7, data reload: false

------ Round 1 ----------------------------------
q1	17657	7309	7170	7170
q2	2046	191	195	191
q3	10637	1360	1339	1339
q4	10455	984	1015	984
q5	7716	3251	3161	3161
q6	245	159	158	158
q7	1046	648	625	625
q8	9685	2064	2042	2042
q9	7615	6369	6335	6335
q10	7028	2555	2520	2520
q11	439	252	261	252
q12	406	226	235	226
q13	17761	3038	3054	3038
q14	284	247	250	247
q15	594	530	518	518
q16	521	423	414	414
q17	1009	960	945	945
q18	7406	6910	6872	6872
q19	1385	1251	1240	1240
q20	610	328	325	325
q21	3952	3563	3520	3520
q22	1080	1004	1022	1004
Total cold run time: 109577 ms
Total hot run time: 43126 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7267	7117	7147	7117
q2	343	233	234	233
q3	2927	2919	2898	2898
q4	2011	1974	1979	1974
q5	5489	5423	5445	5423
q6	238	143	145	143
q7	2041	1675	1697	1675
q8	3242	3347	3311	3311
q9	8432	8461	8415	8415
q10	3402	3440	3442	3440
q11	573	474	472	472
q12	790	568	568	568
q13	6623	3008	3059	3008
q14	317	283	270	270
q15	562	522	520	520
q16	494	460	452	452
q17	1776	1705	1714	1705
q18	8092	7493	7744	7493
q19	1733	1711	1706	1706
q20	2031	1796	1816	1796
q21	5750	5527	5473	5473
q22	1117	981	1027	981
Total cold run time: 65250 ms
Total hot run time: 59073 ms

doris-robot · 2024-09-14T09:21:53Z

TPC-DS: Total hot run time: 195483 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b91463f3dc3771a7225fbfe404bf284bcc83ff7, data reload: false

query1	926	380	393	380
query2	6480	1765	1753	1753
query3	6662	210	218	210
query4	25888	23898	23943	23898
query5	5308	558	539	539
query6	267	171	160	160
query7	4589	299	308	299
query8	289	212	223	212
query9	8505	2597	2589	2589
query10	460	285	288	285
query11	16241	15780	15558	15558
query12	154	102	103	102
query13	1709	416	386	386
query14	11716	7386	6854	6854
query15	215	168	174	168
query16	7621	478	481	478
query17	1542	592	589	589
query18	1986	293	295	293
query19	195	152	151	151
query20	119	117	116	116
query21	216	106	106	106
query22	4519	4362	4274	4274
query23	34612	33756	33839	33756
query24	9531	3115	3090	3090
query25	668	421	425	421
query26	1141	164	160	160
query27	2552	281	294	281
query28	6718	2094	2067	2067
query29	970	448	442	442
query30	294	159	155	155
query31	1027	798	797	797
query32	103	57	60	57
query33	732	320	324	320
query34	921	502	493	493
query35	891	756	722	722
query36	1024	909	928	909
query37	170	86	84	84
query38	3938	3914	3876	3876
query39	1453	1429	1390	1390
query40	222	118	124	118
query41	49	51	48	48
query42	116	100	100	100
query43	505	462	452	452
query44	1246	799	780	780
query45	200	168	173	168
query46	1112	837	837	837
query47	1885	1762	1782	1762
query48	369	306	300	300
query49	1138	463	463	463
query50	928	434	457	434
query51	6932	7018	6910	6910
query52	101	91	89	89
query53	272	202	190	190
query54	783	476	471	471
query55	83	79	76	76
query56	299	281	282	281
query57	1238	1074	1092	1074
query58	262	241	239	239
query59	2812	2643	2756	2643
query60	317	298	289	289
query61	129	122	224	122
query62	911	667	662	662
query63	213	206	187	187
query64	5304	683	665	665
query65	3252	3145	3156	3145
query66	1413	290	320	290
query67	15925	15562	15464	15464
query68	3193	593	589	589
query69	414	275	272	272
query70	1163	1120	1087	1087
query71	346	278	283	278
query72	5966	4073	4017	4017
query73	753	333	330	330
query74	9265	9061	9212	9061
query75	3382	2729	2729	2729
query76	1910	1346	1318	1318
query77	458	335	321	321
query78	10077	9436	9368	9368
query79	1074	889	895	889
query80	1009	840	826	826
query81	599	268	271	268
query82	817	260	256	256
query83	216	191	192	191
query84	240	105	106	105
query85	741	420	402	402
query86	440	325	309	309
query87	4464	4291	4377	4291
query88	4175	4105	4081	4081
query89	381	365	374	365
query90	1758	321	321	321
query91	123	127	123	123
query92	82	78	76	76
query93	1120	1081	1055	1055
query94	836	357	368	357
query95	555	426	439	426
query96	471	474	471	471
query97	3174	3146	3125	3125
query98	245	253	221	221
query99	1596	1309	1276	1276
Total cold run time: 281411 ms
Total hot run time: 195483 ms

doris-robot · 2024-09-14T09:27:24Z

ClickBench: Total hot run time: 31.39 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9b91463f3dc3771a7225fbfe404bf284bcc83ff7, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.03
query3	0.22	0.04	0.04
query4	1.68	0.06	0.07
query5	0.51	0.50	0.51
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.54	0.51
query10	0.58	0.59	0.56
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.62	0.61	0.61
query14	1.48	1.48	1.47
query15	0.89	0.88	0.87
query16	0.37	0.37	0.37
query17	1.01	1.01	1.07
query18	0.16	0.16	0.16
query19	1.90	1.84	1.81
query20	0.01	0.02	0.01
query21	15.38	0.65	0.66
query22	3.99	6.91	1.56
query23	17.99	1.41	1.37
query24	2.28	0.23	0.22
query25	0.18	0.08	0.07
query26	0.28	0.18	0.18
query27	0.08	0.07	0.08
query28	13.18	1.13	1.10
query29	12.55	3.38	3.36
query30	0.25	0.05	0.06
query31	2.87	0.43	0.42
query32	3.22	0.49	0.50
query33	3.06	3.02	3.04
query34	15.45	4.32	4.33
query35	4.33	4.33	4.34
query36	0.68	0.49	0.48
query37	0.20	0.17	0.17
query38	0.16	0.16	0.16
query39	0.04	0.04	0.05
query40	0.16	0.14	0.13
query41	0.10	0.05	0.05
query42	0.07	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.17 s
Total hot run time: 31.39 s

dataroaring · 2024-09-14T09:27:58Z

run buildall

fe/fe-core/src/main/java/org/apache/doris/analysis/ShowCloudWarmUpStmt.java

fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudEnv.java

fe/fe-core/src/main/java/org/apache/doris/cloud/system/CloudSystemInfoService.java

regression-test/suites/cloud_p0/auth/test_grant_revoke_cluster_to_user.groovy

doris-robot · 2024-09-14T11:14:01Z

TeamCity be ut coverage result:
Function Coverage: 37.32% (9577/25663)
Line Coverage: 28.71% (79095/275501)
Region Coverage: 28.19% (40972/145332)
Branch Coverage: 24.82% (20884/84154)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7a59dc60043a6f8b9eb862e117c8f9976feac13e_7a59dc60043a6f8b9eb862e117c8f9976feac13e/report/index.html

doris-robot · 2024-09-14T11:18:02Z

TPC-H: Total hot run time: 42105 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7a59dc60043a6f8b9eb862e117c8f9976feac13e, data reload: false

------ Round 1 ----------------------------------
q1	17611	7496	7235	7235
q2	2042	161	154	154
q3	10884	1164	1278	1164
q4	10336	831	961	831
q5	7789	3127	3146	3127
q6	234	148	144	144
q7	1022	616	600	600
q8	9436	2000	2090	2000
q9	6869	6458	6421	6421
q10	7061	2339	2340	2339
q11	444	252	246	246
q12	399	208	211	208
q13	17804	2983	2991	2983
q14	246	218	215	215
q15	582	512	518	512
q16	505	427	418	418
q17	991	941	925	925
q18	7408	6765	6693	6693
q19	1397	1251	1255	1251
q20	577	276	289	276
q21	3942	3569	3376	3376
q22	1106	1003	987	987
Total cold run time: 108685 ms
Total hot run time: 42105 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7181	7113	7160	7113
q2	331	233	234	233
q3	3059	3138	3064	3064
q4	2082	2021	1950	1950
q5	5631	5591	5708	5591
q6	243	147	143	143
q7	2187	1752	1823	1752
q8	3304	3436	3426	3426
q9	8857	8804	8795	8795
q10	3485	3538	3557	3538
q11	583	475	481	475
q12	828	647	662	647
q13	11702	3166	3164	3164
q14	303	270	266	266
q15	565	511	513	511
q16	514	482	468	468
q17	1818	1730	1732	1730
q18	8375	8170	7997	7997
q19	1855	1766	1759	1759
q20	2142	1900	1878	1878
q21	5325	5253	5360	5253
q22	1143	1028	1023	1023
Total cold run time: 71513 ms
Total hot run time: 60776 ms

doris-robot · 2024-09-14T11:30:09Z

TPC-DS: Total hot run time: 197283 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7a59dc60043a6f8b9eb862e117c8f9976feac13e, data reload: false

query1	1294	880	933	880
query2	6418	1748	1729	1729
query3	10766	4037	3684	3684
query4	54728	26816	23830	23830
query5	5218	511	506	506
query6	426	174	181	174
query7	5766	306	302	302
query8	285	233	229	229
query9	7048	2580	2559	2559
query10	456	289	273	273
query11	16699	15580	15475	15475
query12	155	101	96	96
query13	1495	408	421	408
query14	10196	7312	7043	7043
query15	211	186	191	186
query16	6990	486	522	486
query17	1176	622	601	601
query18	1762	331	318	318
query19	198	150	149	149
query20	122	111	113	111
query21	225	108	111	108
query22	4585	4554	4436	4436
query23	34586	34145	33475	33475
query24	6107	3061	3095	3061
query25	504	394	401	394
query26	616	161	162	161
query27	1597	288	291	288
query28	2833	2106	2071	2071
query29	725	401	400	400
query30	220	148	151	148
query31	983	762	789	762
query32	71	54	53	53
query33	473	296	308	296
query34	892	483	478	478
query35	866	731	713	713
query36	1027	895	894	894
query37	133	74	75	74
query38	3934	3888	3960	3888
query39	1466	1357	1395	1357
query40	206	92	92	92
query41	47	45	45	45
query42	113	96	100	96
query43	478	424	424	424
query44	1164	790	766	766
query45	197	164	167	164
query46	1089	815	809	809
query47	1842	1756	1780	1756
query48	429	351	344	344
query49	683	391	379	379
query50	919	441	442	441
query51	7136	6877	6922	6877
query52	95	83	85	83
query53	245	202	179	179
query54	556	455	444	444
query55	81	76	75	75
query56	266	254	259	254
query57	1187	1044	1099	1044
query58	216	238	275	238
query59	2745	2777	2437	2437
query60	288	259	263	259
query61	98	97	102	97
query62	800	663	663	663
query63	217	186	185	185
query64	1447	629	646	629
query65	3229	3164	3188	3164
query66	680	289	299	289
query67	15963	15479	15470	15470
query68	2022	847	842	842
query69	439	329	339	329
query70	1184	1118	1179	1118
query71	345	325	327	325
query72	5000	3321	3249	3249
query73	587	574	568	568
query74	9168	9069	8960	8960
query75	3071	2975	3031	2975
query76	913	831	815	815
query77	473	345	350	345
query78	9506	9206	9229	9206
query79	898	870	846	846
query80	585	559	571	559
query81	454	232	236	232
query82	188	189	193	189
query83	161	151	157	151
query84	211	96	103	96
query85	640	376	350	350
query86	302	312	313	312
query87	4338	4269	4409	4269
query88	4076	4003	4042	4003
query89	357	349	349	349
query90	1473	308	290	290
query91	184	158	160	158
query92	78	74	74	74
query93	993	999	1001	999
query94	562	373	390	373
query95	432	406	414	406
query96	469	462	462	462
query97	3102	3116	3138	3116
query98	234	237	220	220
query99	1645	1298	1298	1298
Total cold run time: 289758 ms
Total hot run time: 197283 ms

doris-robot · 2024-09-14T11:35:37Z

ClickBench: Total hot run time: 31.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7a59dc60043a6f8b9eb862e117c8f9976feac13e, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.06	0.07
query5	0.49	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.56	0.52	0.53
query10	0.55	0.58	0.58
query11	0.16	0.11	0.12
query12	0.15	0.14	0.13
query13	0.63	0.61	0.61
query14	1.41	1.45	1.48
query15	0.91	0.92	0.89
query16	0.36	0.37	0.38
query17	1.05	1.03	1.02
query18	0.17	0.17	0.16
query19	1.92	1.82	1.84
query20	0.02	0.01	0.01
query21	15.42	0.66	0.66
query22	3.69	6.88	1.61
query23	17.73	1.42	1.35
query24	2.31	0.23	0.22
query25	0.18	0.08	0.08
query26	0.30	0.19	0.18
query27	0.08	0.08	0.07
query28	13.19	1.14	1.11
query29	12.56	3.35	3.30
query30	0.24	0.06	0.05
query31	2.88	0.42	0.42
query32	3.22	0.50	0.49
query33	3.05	3.06	3.01
query34	15.43	4.32	4.28
query35	4.34	4.33	4.33
query36	0.68	0.49	0.50
query37	0.18	0.17	0.16
query38	0.17	0.14	0.16
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.68 s
Total hot run time: 31.41 s

doris-robot · 2024-09-18T09:15:27Z

ClickBench: Total hot run time: 33.5 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99d9827e0701f7cca253408ca0985e51f1173cee, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.02
query3	0.22	0.06	0.06
query4	1.65	0.10	0.10
query5	0.50	0.48	0.49
query6	1.13	0.73	0.72
query7	0.01	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.54	0.54	0.57
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	2.98	2.98	2.96
query15	0.89	0.83	0.81
query16	0.39	0.39	0.40
query17	1.06	1.05	1.05
query18	0.21	0.21	0.21
query19	1.92	1.85	1.91
query20	0.01	0.01	0.00
query21	15.36	0.60	0.59
query22	2.46	3.15	2.05
query23	17.52	0.93	0.77
query24	2.95	1.58	1.80
query25	0.36	0.11	0.05
query26	0.49	0.14	0.14
query27	0.03	0.04	0.04
query28	9.70	1.11	1.07
query29	12.63	3.21	3.20
query30	0.25	0.07	0.06
query31	2.89	0.40	0.37
query32	3.26	0.46	0.47
query33	2.99	3.01	3.02
query34	16.85	4.41	4.35
query35	4.40	4.47	4.42
query36	0.68	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.38 s
Total hot run time: 33.5 s

dataroaring · 2024-09-18T12:51:11Z

run buildall

doris-robot · 2024-09-18T14:10:05Z

TeamCity be ut coverage result:
Function Coverage: 37.31% (9579/25671)
Line Coverage: 28.70% (79179/275857)
Region Coverage: 28.18% (41004/145503)
Branch Coverage: 24.80% (20894/84244)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9a7974a5efe0676a866507903ecf449341fbb381_9a7974a5efe0676a866507903ecf449341fbb381/report/index.html

doris-robot · 2024-09-18T14:13:17Z

TPC-H: Total hot run time: 41715 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a7974a5efe0676a866507903ecf449341fbb381, data reload: false

------ Round 1 ----------------------------------
q1	17942	7479	7363	7363
q2	2044	174	161	161
q3	10629	1170	1216	1170
q4	10246	726	848	726
q5	7758	3215	3081	3081
q6	238	151	149	149
q7	1025	606	602	602
q8	9437	2085	2086	2085
q9	6953	6428	6453	6428
q10	7034	2337	2351	2337
q11	447	247	247	247
q12	415	218	214	214
q13	17776	3000	2993	2993
q14	237	222	224	222
q15	577	537	528	528
q16	673	594	643	594
q17	990	827	814	814
q18	7657	6642	6762	6642
q19	1395	926	1029	926
q20	588	303	282	282
q21	4000	3156	3326	3156
q22	1118	1023	995	995
Total cold run time: 109179 ms
Total hot run time: 41715 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7283	7257	7217	7217
q2	321	231	231	231
q3	2910	2807	2826	2807
q4	1913	1737	1654	1654
q5	5413	5422	5455	5422
q6	228	143	142	142
q7	2087	1711	1702	1702
q8	3204	3352	3402	3352
q9	8411	8449	8455	8449
q10	3412	3401	3379	3379
q11	574	469	467	467
q12	769	568	600	568
q13	9065	2991	2979	2979
q14	290	257	269	257
q15	574	522	512	512
q16	714	683	665	665
q17	1795	1581	1551	1551
q18	7708	7322	7541	7322
q19	1697	1585	1461	1461
q20	2060	1812	1815	1812
q21	5582	5290	5203	5203
q22	1101	1044	1006	1006
Total cold run time: 67111 ms
Total hot run time: 58158 ms

doris-robot · 2024-09-18T14:25:18Z

TPC-DS: Total hot run time: 193841 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a7974a5efe0676a866507903ecf449341fbb381, data reload: false

query1	970	381	394	381
query2	6541	2101	2060	2060
query3	6696	215	220	215
query4	34425	23440	23329	23329
query5	4313	474	471	471
query6	263	163	161	161
query7	4614	292	298	292
query8	269	215	224	215
query9	9639	2617	2616	2616
query10	484	296	290	290
query11	18192	15179	15100	15100
query12	158	99	97	97
query13	1615	408	417	408
query14	10674	7553	6605	6605
query15	260	175	187	175
query16	7852	439	481	439
query17	1686	592	566	566
query18	1443	307	308	307
query19	371	158	147	147
query20	121	109	113	109
query21	220	104	105	104
query22	4658	4292	4302	4292
query23	34688	34068	33851	33851
query24	11193	2916	2939	2916
query25	623	409	407	407
query26	1174	157	167	157
query27	2366	286	291	286
query28	8008	2478	2428	2428
query29	794	426	421	421
query30	328	165	148	148
query31	1030	783	814	783
query32	103	56	60	56
query33	788	308	305	305
query34	932	497	499	497
query35	888	752	735	735
query36	1100	940	910	910
query37	151	92	88	88
query38	3987	3868	3959	3868
query39	1455	1410	1539	1410
query40	202	100	98	98
query41	52	49	49	49
query42	121	100	97	97
query43	531	485	495	485
query44	1215	833	796	796
query45	194	166	173	166
query46	1154	755	751	751
query47	1899	1827	1847	1827
query48	468	369	353	353
query49	1134	419	401	401
query50	832	394	399	394
query51	7115	6984	7008	6984
query52	103	94	84	84
query53	250	186	186	186
query54	1220	465	477	465
query55	79	81	76	76
query56	281	281	273	273
query57	1192	1058	1092	1058
query58	253	253	234	234
query59	3194	3016	3139	3016
query60	298	277	277	277
query61	127	147	96	96
query62	855	645	652	645
query63	220	185	185	185
query64	4134	655	643	643
query65	3288	3193	3168	3168
query66	771	297	298	297
query67	16037	15653	15553	15553
query68	3082	562	570	562
query69	454	282	286	282
query70	1189	1123	1101	1101
query71	323	268	270	268
query72	5662	4073	4011	4011
query73	761	339	334	334
query74	9492	8941	8985	8941
query75	3346	2644	2651	2644
query76	1910	882	953	882
query77	428	281	281	281
query78	10002	9525	9318	9318
query79	2056	868	851	851
query80	980	574	567	567
query81	581	256	251	251
query82	702	232	228	228
query83	204	166	155	155
query84	249	104	106	104
query85	750	388	360	360
query86	440	312	315	312
query87	4322	4324	4347	4324
query88	4685	4114	4084	4084
query89	380	361	358	358
query90	1811	321	315	315
query91	157	170	168	168
query92	90	74	73	73
query93	965	917	905	905
query94	780	365	374	365
query95	462	404	411	404
query96	494	489	491	489
query97	3164	3125	3145	3125
query98	230	247	222	222
query99	1447	1315	1255	1255
Total cold run time: 293889 ms
Total hot run time: 193841 ms

doris-robot · 2024-09-18T14:30:49Z

ClickBench: Total hot run time: 32.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a7974a5efe0676a866507903ecf449341fbb381, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.02
query3	0.24	0.07	0.06
query4	1.66	0.10	0.10
query5	0.53	0.50	0.52
query6	1.12	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.55	0.55	0.55
query11	0.14	0.10	0.11
query12	0.14	0.10	0.10
query13	0.60	0.59	0.60
query14	3.05	2.97	2.98
query15	0.88	0.82	0.83
query16	0.39	0.39	0.39
query17	1.06	1.06	1.05
query18	0.19	0.19	0.20
query19	2.00	2.01	1.85
query20	0.02	0.01	0.01
query21	15.35	0.61	0.62
query22	2.39	1.99	2.34
query23	17.43	0.80	0.76
query24	2.93	1.55	1.00
query25	0.24	0.11	0.09
query26	0.56	0.13	0.14
query27	0.04	0.05	0.04
query28	10.28	1.09	1.07
query29	12.57	3.24	3.25
query30	0.24	0.05	0.06
query31	2.87	0.39	0.38
query32	3.27	0.46	0.46
query33	3.00	3.02	3.05
query34	16.79	4.42	4.38
query35	4.46	4.39	4.48
query36	0.65	0.50	0.49
query37	0.09	0.06	0.05
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 106.84 s
Total hot run time: 32.97 s

dataroaring · 2024-09-19T01:13:16Z

run buildall

doris-robot · 2024-09-19T02:36:22Z

TPC-H: Total hot run time: 41807 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 196f98637236a9ce36c7826928e439c06d426d99, data reload: false

------ Round 1 ----------------------------------
q1	18191	7609	7405	7405
q2	2542	161	166	161
q3	11524	1195	1163	1163
q4	10550	737	766	737
q5	8116	3175	3117	3117
q6	241	152	147	147
q7	1025	633	612	612
q8	9489	2089	2076	2076
q9	6863	6446	6414	6414
q10	7015	2260	2289	2260
q11	446	246	247	246
q12	408	215	214	214
q13	17766	2951	2984	2951
q14	247	216	214	214
q15	584	526	504	504
q16	652	607	605	605
q17	976	842	822	822
q18	7359	6688	6691	6688
q19	1409	987	1096	987
q20	603	286	283	283
q21	3988	3203	3190	3190
q22	1085	1011	1043	1011
Total cold run time: 111079 ms
Total hot run time: 41807 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7278	7206	7191	7191
q2	332	229	235	229
q3	2926	2814	2783	2783
q4	1951	1697	1767	1697
q5	5366	5404	5370	5370
q6	226	138	144	138
q7	2080	1684	1678	1678
q8	3192	3321	3347	3321
q9	8437	8429	8460	8429
q10	3381	3362	3354	3354
q11	575	461	466	461
q12	750	556	580	556
q13	6318	3003	3016	3003
q14	290	268	265	265
q15	575	503	515	503
q16	704	657	659	657
q17	1776	1573	1563	1563
q18	7776	7275	7375	7275
q19	1674	1497	1545	1497
q20	2040	1801	1798	1798
q21	5422	5259	5058	5058
q22	1123	1031	1006	1006
Total cold run time: 64192 ms
Total hot run time: 57832 ms

doris-robot · 2024-09-19T02:48:22Z

TPC-DS: Total hot run time: 194324 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 196f98637236a9ce36c7826928e439c06d426d99, data reload: false

query1	984	370	386	370
query2	6520	2114	2013	2013
query3	6712	213	220	213
query4	34281	23300	23213	23213
query5	4306	465	469	465
query6	257	179	155	155
query7	4614	297	290	290
query8	273	217	214	214
query9	9620	2658	2652	2652
query10	458	284	287	284
query11	18137	15264	15122	15122
query12	159	95	94	94
query13	1630	424	399	399
query14	10545	7366	7467	7366
query15	316	176	178	176
query16	7939	450	456	450
query17	1791	571	543	543
query18	2038	304	303	303
query19	367	150	147	147
query20	115	111	111	111
query21	218	106	106	106
query22	4579	4136	4213	4136
query23	34675	33919	33871	33871
query24	11207	2854	2850	2850
query25	621	401	407	401
query26	1139	162	159	159
query27	2374	283	290	283
query28	7746	2485	2451	2451
query29	831	417	427	417
query30	310	155	155	155
query31	1005	790	832	790
query32	98	55	59	55
query33	772	297	302	297
query34	962	494	475	475
query35	902	744	731	731
query36	1081	938	961	938
query37	162	94	88	88
query38	4017	3936	3917	3917
query39	1493	1416	1386	1386
query40	209	96	97	96
query41	50	48	48	48
query42	114	97	99	97
query43	520	480	473	473
query44	1292	815	804	804
query45	201	167	169	167
query46	1143	738	771	738
query47	1920	1823	1851	1823
query48	452	367	353	353
query49	1142	434	423	423
query50	830	403	408	403
query51	7031	6971	6840	6840
query52	97	90	86	86
query53	249	186	180	180
query54	1365	471	467	467
query55	82	80	76	76
query56	288	262	263	262
query57	1227	1099	1082	1082
query58	251	228	239	228
query59	3106	3041	3010	3010
query60	302	281	283	281
query61	129	145	98	98
query62	834	653	649	649
query63	210	182	176	176
query64	5269	644	678	644
query65	3450	3223	3184	3184
query66	1434	299	295	295
query67	16032	15490	15471	15471
query68	3118	860	861	860
query69	451	339	341	339
query70	1200	1174	1182	1174
query71	344	329	327	327
query72	5932	3389	3362	3362
query73	596	584	584	584
query74	9570	8951	9048	8951
query75	3083	2846	2836	2836
query76	1964	858	880	858
query77	397	362	357	357
query78	9385	9227	9464	9227
query79	918	886	866	866
query80	586	573	580	573
query81	450	255	248	248
query82	234	226	227	226
query83	167	157	167	157
query84	242	103	98	98
query85	672	360	352	352
query86	302	331	313	313
query87	4335	4335	4368	4335
query88	4608	4097	4105	4097
query89	386	358	358	358
query90	1477	327	323	323
query91	164	165	169	165
query92	78	73	70	70
query93	922	917	893	893
query94	569	365	376	365
query95	419	406	409	406
query96	491	492	489	489
query97	3099	3104	3101	3101
query98	227	229	215	215
query99	1422	1311	1259	1259
Total cold run time: 292291 ms
Total hot run time: 194324 ms

doris-robot · 2024-09-19T02:53:47Z

ClickBench: Total hot run time: 32.09 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 196f98637236a9ce36c7826928e439c06d426d99, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.02	0.03
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.51	0.52	0.51
query6	1.14	0.74	0.72
query7	0.02	0.02	0.02
query8	0.04	0.02	0.03
query9	0.56	0.51	0.48
query10	0.55	0.57	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	3.14	2.95	3.06
query15	0.90	0.82	0.81
query16	0.39	0.38	0.40
query17	1.01	1.06	1.05
query18	0.21	0.20	0.20
query19	1.95	1.90	1.94
query20	0.01	0.01	0.00
query21	15.35	0.59	0.59
query22	2.66	2.06	1.62
query23	17.53	0.88	0.83
query24	3.08	0.67	0.38
query25	0.21	0.22	0.12
query26	0.30	0.14	0.13
query27	0.05	0.04	0.03
query28	11.46	1.10	1.07
query29	12.53	3.27	3.25
query30	0.24	0.06	0.05
query31	2.89	0.39	0.38
query32	3.27	0.47	0.47
query33	3.00	3.01	3.06
query34	17.07	4.37	4.44
query35	4.39	4.39	4.39
query36	0.68	0.50	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 108.48 s
Total hot run time: 32.09 s

doris-robot · 2024-09-19T03:01:50Z

TeamCity be ut coverage result:
Function Coverage: 37.32% (9582/25673)
Line Coverage: 28.71% (79217/275883)
Region Coverage: 28.19% (41015/145512)
Branch Coverage: 24.81% (20903/84250)
Coverage Report: http://coverage.selectdb-in.cc/coverage/196f98637236a9ce36c7826928e439c06d426d99_196f98637236a9ce36c7826928e439c06d426d99/report/index.html

docker/runtime/doris-compose/cluster.py

yujun777

LGTM

gavinchou · 2024-09-19T08:17:04Z

fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/manager/ClusterAction.java

 */
 @RestController
-@RequestMapping("/rest/v2/manager/cluster")
+@RequestMapping(path = {"/rest/v2/manager/cluster", "/rest/v2/manager/compute_group"})


seems no need to change.
this "cluster" is not that "cluster"

gavinchou · 2024-09-19T08:17:05Z

fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/manager/ClusterAction.java

    //   ]
    // }
-    @RequestMapping(path = "/cluster_info/conn_info", method = RequestMethod.GET)
+    @RequestMapping(path = {"/cluster_info/conn_info", "/compute_group_info/conn_info"}, method = RequestMethod.GET)


gavinchou · 2024-09-19T08:21:05Z

regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/Suite.groovy

        return s3Url
    }

+    String getJdbcPassword() {


seems not used

github-actions · 2024-09-19T09:21:50Z

PR approved by at least one committer and no changes requested.

…d_instance_id (#40767) 1. use compute group instead of cloud cluster. 2. use cluster instead of cloud_instance. 3. simply be config by passing cloud_unique_id to be, we add a config item (enable_use_cloud_unique_id_from_fe) in be in order to handle cloud_unique_id is different in ms sometimes, it should not happen. 5. opt sync_vault_info by sync from meta when vault id is empty. 6. use show storage vaults instead of show storage vault. 7. only first fe can be start from empty, otherwise multi master may start. 8. cancel decommission is not supported in cloud mode.

dataroaring requested review from CalvinKirs and morningman as code owners September 12, 2024 13:06