[fix](table) Breaking up dynamic partition table scheduling consuming peak CPU consumption

[deardeng]

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Breaking up dynamic partition table scheduling, consuming peak CPU consumption
Add a config to control it, default not use it feature

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[Thearas]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[deardeng]

run buildall

[doris-robot]

TPC-H: Total hot run time: 31872 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bbd48f1447a35422ebd1c1e6ce6049e5dbf0277b, data reload: false

------ Round 1 ----------------------------------
q1	17608	5223	5106	5106
q2	2052	309	177	177
q3	10389	1335	732	732
q4	10217	1038	542	542
q5	7547	2386	2381	2381
q6	203	166	133	133
q7	899	755	643	643
q8	9314	1279	1080	1080
q9	4850	4633	4693	4633
q10	6814	2337	1923	1923
q11	460	285	258	258
q12	350	369	228	228
q13	17768	3714	3169	3169
q14	237	234	220	220
q15	498	464	451	451
q16	641	630	577	577
q17	589	920	359	359
q18	6726	6168	6186	6168
q19	1212	955	565	565
q20	329	342	203	203
q21	2853	2185	2016	2016
q22	370	345	308	308
Total cold run time: 101926 ms
Total hot run time: 31872 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5223	5111	5098	5098
q2	236	328	240	240
q3	2190	2717	2311	2311
q4	1479	1913	1415	1415
q5	4260	4158	4200	4158
q6	202	160	123	123
q7	1874	1809	1677	1677
q8	2668	2743	2633	2633
q9	7275	7102	7149	7102
q10	3015	3171	2740	2740
q11	560	511	506	506
q12	698	766	633	633
q13	3539	3932	3406	3406
q14	277	304	272	272
q15	518	474	461	461
q16	666	685	671	671
q17	1150	1594	1341	1341
q18	7550	7251	7402	7251
q19	839	882	1001	882
q20	1973	1996	1848	1848
q21	5482	4929	4991	4929
q22	658	597	553	553
Total cold run time: 52332 ms
Total hot run time: 50250 ms

[doris-robot]

TPC-DS: Total hot run time: 184174 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bbd48f1447a35422ebd1c1e6ce6049e5dbf0277b, data reload: false

query1	977	397	379	379
query2	6528	1854	1856	1854
query3	6797	216	225	216
query4	26809	23945	23313	23313
query5	4317	648	479	479
query6	301	206	184	184
query7	4603	518	297	297
query8	299	256	252	252
query9	8580	2570	2562	2562
query10	458	304	263	263
query11	15660	15342	15165	15165
query12	170	110	105	105
query13	1682	532	392	392
query14	9716	6688	6367	6367
query15	208	189	167	167
query16	7400	627	493	493
query17	1179	718	552	552
query18	1952	389	294	294
query19	188	187	149	149
query20	121	114	111	111
query21	209	127	104	104
query22	4024	4472	4306	4306
query23	34186	33284	32764	32764
query24	7784	2384	2395	2384
query25	528	461	403	403
query26	1223	280	156	156
query27	2490	482	325	325
query28	4236	2401	2380	2380
query29	707	542	418	418
query30	233	193	163	163
query31	968	836	769	769
query32	74	63	61	61
query33	563	356	303	303
query34	794	857	516	516
query35	784	831	730	730
query36	984	962	879	879
query37	111	112	74	74
query38	4186	4099	4039	4039
query39	1442	1370	1382	1370
query40	206	117	112	112
query41	55	51	52	51
query42	120	107	108	107
query43	523	512	494	494
query44	1326	804	793	793
query45	178	168	168	168
query46	889	1056	660	660
query47	1731	1812	1708	1708
query48	381	427	302	302
query49	777	504	446	446
query50	694	739	410	410
query51	4178	4220	4110	4110
query52	108	106	102	102
query53	236	263	184	184
query54	478	510	411	411
query55	84	87	81	81
query56	274	274	255	255
query57	1118	1120	1066	1066
query58	247	236	236	236
query59	2629	2782	2536	2536
query60	281	275	261	261
query61	122	123	120	120
query62	777	747	643	643
query63	231	192	191	191
query64	4189	1043	645	645
query65	3223	3131	3158	3131
query66	1087	396	294	294
query67	15789	15522	15356	15356
query68	7676	889	502	502
query69	468	299	270	270
query70	1224	1085	1122	1085
query71	404	302	268	268
query72	5580	3588	3826	3588
query73	737	731	359	359
query74	8853	9120	8887	8887
query75	3242	3211	2716	2716
query76	3258	1183	756	756
query77	484	371	281	281
query78	9984	10039	9236	9236
query79	1746	885	596	596
query80	647	524	463	463
query81	542	278	242	242
query82	318	133	97	97
query83	174	172	163	163
query84	244	97	79	79
query85	738	379	298	298
query86	372	320	268	268
query87	4340	4465	4346	4346
query88	2967	2226	2237	2226
query89	384	313	287	287
query90	1860	195	191	191
query91	139	134	108	108
query92	67	72	64	64
query93	1560	1077	567	567
query94	642	421	306	306
query95	351	268	258	258
query96	489	554	263	263
query97	3276	3407	3307	3307
query98	219	210	199	199
query99	1286	1384	1287	1287
Total cold run time: 269673 ms
Total hot run time: 184174 ms

[doris-robot]

ClickBench: Total hot run time: 31.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bbd48f1447a35422ebd1c1e6ce6049e5dbf0277b, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.59	0.53	0.55
query6	1.21	0.72	0.71
query7	0.03	0.01	0.01
query8	0.05	0.03	0.03
query9	0.59	0.53	0.54
query10	0.56	0.57	0.57
query11	0.15	0.11	0.10
query12	0.13	0.11	0.12
query13	0.61	0.60	0.60
query14	2.68	2.83	2.71
query15	0.95	0.86	0.84
query16	0.36	0.37	0.39
query17	1.02	1.01	1.03
query18	0.20	0.20	0.20
query19	1.91	1.85	1.97
query20	0.02	0.01	0.01
query21	15.36	0.91	0.56
query22	0.75	1.14	0.67
query23	14.98	1.42	0.64
query24	7.49	1.42	1.03
query25	0.50	0.35	0.09
query26	0.53	0.17	0.14
query27	0.05	0.05	0.05
query28	9.99	0.87	0.44
query29	12.52	3.91	3.26
query30	0.24	0.09	0.08
query31	2.81	0.60	0.39
query32	3.22	0.55	0.46
query33	3.00	3.04	3.02
query34	15.83	5.12	4.54
query35	4.55	4.61	4.61
query36	0.66	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 106 s
Total hot run time: 31.2 s

[deardeng]

run buildall

[yujun777]

LGTM

[github-actions]

PR approved by anyone and no changes requested.

[doris-robot]

TPC-H: Total hot run time: 32195 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fa89d60c2d29739e651c97242657dc6114542a5, data reload: false

------ Round 1 ----------------------------------
q1	17620	5274	5132	5132
q2	2056	330	170	170
q3	10522	1322	720	720
q4	10233	1047	539	539
q5	7663	2481	2470	2470
q6	189	166	139	139
q7	942	749	619	619
q8	9312	1304	1185	1185
q9	4993	4695	4740	4695
q10	6831	2317	1903	1903
q11	478	279	258	258
q12	362	360	227	227
q13	17781	3660	3158	3158
q14	230	219	215	215
q15	510	473	455	455
q16	634	640	589	589
q17	592	875	352	352
q18	6911	6336	6255	6255
q19	1411	959	569	569
q20	335	333	198	198
q21	2972	2388	2025	2025
q22	369	348	322	322
Total cold run time: 102946 ms
Total hot run time: 32195 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5167	5170	5205	5170
q2	238	345	244	244
q3	2148	2676	2279	2279
q4	1506	1875	1424	1424
q5	4276	4186	4199	4186
q6	214	167	126	126
q7	1886	1908	1810	1810
q8	2641	2796	2704	2704
q9	7327	7094	7186	7094
q10	3044	3239	2755	2755
q11	570	514	506	506
q12	682	727	634	634
q13	3554	3903	3240	3240
q14	290	302	257	257
q15	521	470	467	467
q16	636	710	652	652
q17	1161	1644	1325	1325
q18	7700	7245	7308	7245
q19	858	906	1003	906
q20	1964	2009	1863	1863
q21	5482	5007	4943	4943
q22	663	570	545	545
Total cold run time: 52528 ms
Total hot run time: 50375 ms

[doris-robot]

TPC-DS: Total hot run time: 184251 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fa89d60c2d29739e651c97242657dc6114542a5, data reload: false

query1	970	383	370	370
query2	6571	1864	1859	1859
query3	6793	209	215	209
query4	25886	23298	23016	23016
query5	4405	660	484	484
query6	301	188	181	181
query7	4601	500	288	288
query8	290	244	223	223
query9	8594	2529	2511	2511
query10	460	325	248	248
query11	15754	15158	15064	15064
query12	154	114	107	107
query13	1657	526	388	388
query14	9061	6404	6319	6319
query15	240	195	173	173
query16	7148	652	470	470
query17	933	719	599	599
query18	1967	394	317	317
query19	196	177	157	157
query20	123	114	113	113
query21	209	118	100	100
query22	4258	4380	4206	4206
query23	33959	32981	33297	32981
query24	8172	2373	2366	2366
query25	544	462	387	387
query26	1226	271	156	156
query27	2545	496	338	338
query28	4184	2442	2375	2375
query29	768	548	434	434
query30	233	185	154	154
query31	966	859	790	790
query32	75	71	66	66
query33	589	356	309	309
query34	781	859	512	512
query35	805	810	753	753
query36	968	972	893	893
query37	118	101	73	73
query38	4147	4109	4122	4109
query39	1465	1405	1399	1399
query40	213	124	105	105
query41	52	52	53	52
query42	120	111	104	104
query43	502	516	483	483
query44	1280	797	788	788
query45	182	170	167	167
query46	870	1032	633	633
query47	1757	1781	1686	1686
query48	384	407	296	296
query49	793	524	429	429
query50	690	732	419	419
query51	4170	4183	4170	4170
query52	110	106	93	93
query53	222	255	191	191
query54	483	494	418	418
query55	81	81	87	81
query56	258	265	250	250
query57	1131	1151	1062	1062
query58	253	242	230	230
query59	2642	2585	2370	2370
query60	276	278	261	261
query61	127	114	114	114
query62	807	743	674	674
query63	230	196	196	196
query64	4346	1000	712	712
query65	3236	3125	3183	3125
query66	1137	417	319	319
query67	15828	15430	15277	15277
query68	7000	870	515	515
query69	483	301	266	266
query70	1200	1138	1109	1109
query71	416	291	260	260
query72	5665	3670	3683	3670
query73	734	744	346	346
query74	9238	9110	9055	9055
query75	3184	3161	2695	2695
query76	3211	1188	740	740
query77	486	354	288	288
query78	9840	10111	9332	9332
query79	1435	829	602	602
query80	601	593	468	468
query81	516	277	245	245
query82	215	130	98	98
query83	171	178	162	162
query84	297	95	75	75
query85	750	376	357	357
query86	428	313	301	301
query87	4531	4353	4347	4347
query88	2913	2234	2217	2217
query89	378	314	298	298
query90	1965	202	197	197
query91	142	135	113	113
query92	80	63	56	56
query93	1667	1050	587	587
query94	671	412	314	314
query95	354	274	269	269
query96	495	561	262	262
query97	3298	3515	3284	3284
query98	226	216	207	207
query99	1355	1386	1314	1314
Total cold run time: 268241 ms
Total hot run time: 184251 ms

[doris-robot]

ClickBench: Total hot run time: 30.73 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fa89d60c2d29739e651c97242657dc6114542a5, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.03
query3	0.23	0.06	0.07
query4	1.61	0.11	0.11
query5	0.56	0.55	0.55
query6	1.19	0.71	0.72
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.51
query10	0.57	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.61	0.60	0.59
query14	2.78	2.69	2.74
query15	0.93	0.84	0.85
query16	0.38	0.37	0.38
query17	1.04	1.06	1.04
query18	0.21	0.19	0.19
query19	1.94	1.87	1.93
query20	0.02	0.01	0.02
query21	15.36	0.91	0.54
query22	0.74	1.34	0.71
query23	14.76	1.38	0.62
query24	6.96	2.27	0.68
query25	0.55	0.22	0.06
query26	0.54	0.16	0.13
query27	0.06	0.05	0.04
query28	9.79	0.83	0.44
query29	12.53	3.88	3.28
query30	0.25	0.09	0.07
query31	2.83	0.58	0.39
query32	3.22	0.56	0.47
query33	3.11	2.98	3.09
query34	15.78	5.16	4.51
query35	4.55	4.58	4.56
query36	0.66	0.52	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.33 s
Total hot run time: 30.73 s

[dataroaring]

LGTM

[github-actions]

PR approved by at least one committer and no changes requested.