[improvement](balance) partition rebalance chose disk by rr #36826

[yujun777]

cherry pick from #36826

[yujun777]

run buildall

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[doris-robot]

TPC-H: Total hot run time: 49834 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 930072d83f878f09bb30ff66e40b7c664fb39970, data reload: false

------ Round 1 ----------------------------------
q1	17703	4349	4348	4348
q2	2080	162	156	156
q3	10280	1922	1928	1922
q4	10352	1233	1303	1233
q5	8442	3894	3918	3894
q6	231	145	121	121
q7	2017	1603	1616	1603
q8	9261	2728	2706	2706
q9	10407	10148	10221	10148
q10	8660	3549	3507	3507
q11	414	234	258	234
q12	468	301	297	297
q13	18338	3964	3989	3964
q14	366	326	334	326
q15	507	473	461	461
q16	680	571	571	571
q17	1139	939	979	939
q18	7256	6869	6850	6850
q19	1789	1717	1659	1659
q20	546	297	303	297
q21	4506	4149	4173	4149
q22	531	475	449	449
Total cold run time: 115973 ms
Total hot run time: 49834 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4419	4342	4302	4302
q2	325	228	218	218
q3	4188	4152	4175	4152
q4	2770	2749	2766	2749
q5	7236	7199	7112	7112
q6	236	118	119	118
q7	3269	2824	2905	2824
q8	4422	4536	4529	4529
q9	17139	17014	16943	16943
q10	4384	4366	4297	4297
q11	739	692	668	668
q12	1045	875	875	875
q13	7694	3775	3767	3767
q14	479	431	458	431
q15	513	462	461	461
q16	744	670	665	665
q17	3812	3877	3852	3852
q18	8733	8722	8753	8722
q19	1732	1717	1661	1661
q20	2365	2119	2104	2104
q21	8638	8563	8413	8413
q22	1040	955	973	955
Total cold run time: 85922 ms
Total hot run time: 79818 ms

[doris-robot]

TPC-DS: Total hot run time: 203547 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 930072d83f878f09bb30ff66e40b7c664fb39970, data reload: false

query1	929	423	385	385
query2	6572	2804	2721	2721
query3	6922	209	205	205
query4	21045	18769	17970	17970
query5	19742	6482	6504	6482
query6	313	217	247	217
query7	4289	306	321	306
query8	398	435	414	414
query9	3097	2670	2618	2618
query10	425	303	302	302
query11	11331	10717	10657	10657
query12	124	81	80	80
query13	5603	677	694	677
query14	18524	13653	13495	13495
query15	357	240	243	240
query16	6452	286	274	274
query17	1528	1433	869	869
query18	2281	409	406	406
query19	205	151	157	151
query20	82	80	90	80
query21	194	93	91	91
query22	5208	4986	5030	4986
query23	32611	31847	31889	31847
query24	6797	6494	6464	6464
query25	523	447	436	436
query26	516	161	163	161
query27	1794	294	292	292
query28	6145	2371	2348	2348
query29	2846	2716	2641	2641
query30	244	165	167	165
query31	905	734	791	734
query32	71	69	62	62
query33	407	253	243	243
query34	838	476	462	462
query35	1112	943	892	892
query36	1623	1178	1237	1178
query37	92	60	61	60
query38	3071	2870	2908	2870
query39	1370	1353	1330	1330
query40	211	98	97	97
query41	50	44	44	44
query42	93	94	87	87
query43	724	687	683	683
query44	1140	727	741	727
query45	251	238	235	235
query46	1230	955	1007	955
query47	1807	1785	1921	1785
query48	1019	700	710	700
query49	616	375	371	371
query50	865	637	663	637
query51	4775	4700	4637	4637
query52	93	82	89	82
query53	455	316	327	316
query54	2627	2422	2492	2422
query55	102	78	80	78
query56	215	232	208	208
query57	1144	1150	1048	1048
query58	213	195	201	195
query59	4148	3939	3838	3838
query60	209	197	212	197
query61	98	93	92	92
query62	816	488	485	485
query63	493	335	334	334
query64	2539	1515	1487	1487
query65	3606	3544	3576	3544
query66	777	384	369	369
query67	15613	15881	15616	15616
query68	8424	661	670	661
query69	587	353	349	349
query70	1586	1504	1313	1313
query71	400	304	325	304
query72	6648	3463	3499	3463
query73	735	323	323	323
query74	6235	5873	5882	5873
query75	5006	3691	3742	3691
query76	5075	1158	1193	1158
query77	742	261	263	261
query78	12495	11825	11521	11521
query79	9009	645	625	625
query80	1751	404	407	404
query81	491	236	235	235
query82	1699	100	95	95
query83	164	134	131	131
query84	252	72	72	72
query85	1014	321	317	317
query86	339	300	292	292
query87	3236	2966	3041	2966
query88	5150	2315	2324	2315
query89	395	306	299	299
query90	1860	221	218	218
query91	176	141	141	141
query92	58	55	54	54
query93	5614	560	612	560
query94	720	214	213	213
query95	1118	1047	1069	1047
query96	640	340	332	332
query97	6505	6241	6332	6241
query98	187	180	169	169
query99	2936	824	939	824
Total cold run time: 313762 ms
Total hot run time: 203547 ms

[doris-robot]

ClickBench: Total hot run time: 30.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 930072d83f878f09bb30ff66e40b7c664fb39970, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.80	0.06	0.06
query5	0.53	0.53	0.51
query6	1.24	0.61	0.62
query7	0.01	0.00	0.01
query8	0.04	0.03	0.02
query9	0.54	0.50	0.49
query10	0.56	0.54	0.53
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.60	0.61
query14	0.76	0.78	0.78
query15	0.78	0.76	0.78
query16	0.36	0.36	0.37
query17	1.02	1.01	1.01
query18	0.23	0.25	0.25
query19	1.91	1.87	1.85
query20	0.01	0.01	0.01
query21	15.46	0.56	0.56
query22	2.19	2.03	1.91
query23	17.24	1.08	0.95
query24	6.81	0.78	0.76
query25	0.42	0.11	0.04
query26	0.63	0.15	0.16
query27	0.04	0.04	0.03
query28	6.86	0.77	0.71
query29	12.64	2.31	2.37
query30	0.55	0.54	0.53
query31	2.82	0.40	0.38
query32	3.38	0.49	0.49
query33	3.08	3.04	3.10
query34	15.26	4.80	4.79
query35	4.86	4.83	4.84
query36	1.05	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.13	0.13
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.65 s
Total hot run time: 30.84 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 930072d83f878f09bb30ff66e40b7c664fb39970 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s