branch-3.1: [fix](catalog) fix deadlock of catalog and database #53626

[morningman]

bp #53626

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[morningman]

run buildall

[doris-robot]

TPC-H: Total hot run time: 40218 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fb7fe5cffb33714284e53e9e74ae4c8d1988cf42, data reload: false

------ Round 1 ----------------------------------
q1	17679	6748	6625	6625
q2	2076	212	205	205
q3	10517	1171	1188	1171
q4	10268	802	754	754
q5	7765	2888	2863	2863
q6	233	144	144	144
q7	1014	657	661	657
q8	9632	1984	2057	1984
q9	8213	6418	6540	6418
q10	7066	2284	2310	2284
q11	485	284	288	284
q12	432	240	240	240
q13	17789	3005	2994	2994
q14	258	231	228	228
q15	562	500	522	500
q16	484	397	398	397
q17	1001	604	553	553
q18	7217	6735	6669	6669
q19	1343	996	1032	996
q20	488	213	215	213
q21	3886	3250	3016	3016
q22	1113	1023	1037	1023
Total cold run time: 109521 ms
Total hot run time: 40218 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6690	6604	6607	6604
q2	345	250	244	244
q3	3098	3109	2969	2969
q4	2066	1773	1848	1773
q5	5727	5737	5748	5737
q6	218	136	139	136
q7	2274	1808	1827	1808
q8	3415	3525	3529	3525
q9	8991	8764	8895	8764
q10	3606	3552	3521	3521
q11	609	508	494	494
q12	821	602	629	602
q13	9070	3180	3176	3176
q14	325	282	271	271
q15	548	511	493	493
q16	511	472	454	454
q17	1867	1614	1598	1598
q18	8240	7838	7798	7798
q19	1704	1596	1642	1596
q20	2160	1895	1896	1895
q21	5313	5039	4954	4954
q22	1111	1044	1029	1029
Total cold run time: 68709 ms
Total hot run time: 59441 ms

[doris-robot]

TPC-H: Total hot run time: 32270 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fb7fe5cffb33714284e53e9e74ae4c8d1988cf42, data reload: false

------ Round 1 ----------------------------------
q1	17587	5443	5317	5317
q2	2056	298	164	164
q3	10414	1248	751	751
q4	10216	863	456	456
q5	7653	2333	2088	2088
q6	183	163	141	141
q7	923	746	618	618
q8	9322	1391	1195	1195
q9	5177	4867	4931	4867
q10	6762	2260	1836	1836
q11	473	273	260	260
q12	338	345	216	216
q13	17800	3600	3010	3010
q14	232	229	205	205
q15	515	462	471	462
q16	412	430	383	383
q17	576	848	350	350
q18	6980	6294	6241	6241
q19	1220	952	543	543
q20	330	342	211	211
q21	2911	2254	1969	1969
q22	1050	1022	987	987
Total cold run time: 103130 ms
Total hot run time: 32270 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5550	5515	5505	5505
q2	230	332	234	234
q3	2239	2600	2256	2256
q4	1376	1810	1362	1362
q5	4430	4794	4825	4794
q6	170	164	132	132
q7	2071	1945	1811	1811
q8	2601	2798	2653	2653
q9	7237	7155	7188	7155
q10	3035	3300	2810	2810
q11	572	495	496	495
q12	649	694	566	566
q13	3410	3822	3156	3156
q14	292	298	261	261
q15	522	475	463	463
q16	436	508	453	453
q17	1192	1735	1264	1264
q18	7476	7462	7265	7265
q19	778	1108	1066	1066
q20	1985	2050	1900	1900
q21	5237	4941	4663	4663
q22	1121	1062	1001	1001
Total cold run time: 52609 ms
Total hot run time: 51265 ms

[doris-robot]

TPC-DS: Total hot run time: 198056 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fb7fe5cffb33714284e53e9e74ae4c8d1988cf42, data reload: false

query1	1292	905	900	900
query2	6222	2049	1992	1992
query3	10937	4556	4545	4545
query4	32807	24032	23409	23409
query5	3929	579	480	480
query6	285	191	185	185
query7	4002	489	326	326
query8	305	249	240	240
query9	9453	2583	2567	2567
query10	480	332	263	263
query11	18537	15534	15122	15122
query12	160	105	101	101
query13	1545	543	400	400
query14	9106	7589	7610	7589
query15	219	199	194	194
query16	8093	686	508	508
query17	1582	772	622	622
query18	2141	435	342	342
query19	203	197	173	173
query20	127	117	116	116
query21	207	130	111	111
query22	4432	4532	4406	4406
query23	35354	34246	33825	33825
query24	7299	2721	2769	2721
query25	532	486	430	430
query26	832	289	184	184
query27	2138	478	365	365
query28	5179	2236	2219	2219
query29	703	605	466	466
query30	249	194	160	160
query31	1005	915	837	837
query32	90	63	60	60
query33	491	371	327	327
query34	740	846	530	530
query35	787	825	729	729
query36	994	1037	974	974
query37	102	97	96	96
query38	3991	4071	3933	3933
query39	1514	1469	1468	1468
query40	223	125	105	105
query41	61	58	52	52
query42	118	103	102	102
query43	507	492	498	492
query44	1370	858	832	832
query45	192	181	173	173
query46	880	1058	686	686
query47	1936	1949	1991	1949
query48	399	418	355	355
query49	743	481	393	393
query50	688	712	434	434
query51	7318	7276	7321	7276
query52	103	105	90	90
query53	225	262	197	197
query54	557	547	479	479
query55	83	78	83	78
query56	280	264	271	264
query57	1259	1277	1208	1208
query58	233	216	214	214
query59	3204	3317	3191	3191
query60	295	275	284	275
query61	116	129	123	123
query62	780	745	717	717
query63	228	198	200	198
query64	3661	1040	650	650
query65	3356	3266	3270	3266
query66	836	416	316	316
query67	16069	15835	15729	15729
query68	8480	826	543	543
query69	498	305	268	268
query70	1164	1135	1140	1135
query71	383	299	270	270
query72	5784	3777	3791	3777
query73	622	735	352	352
query74	10650	9462	8985	8985
query75	3197	3157	2670	2670
query76	3071	1152	798	798
query77	472	372	295	295
query78	10480	10301	9637	9637
query79	3968	882	601	601
query80	740	519	441	441
query81	509	256	218	218
query82	657	114	86	86
query83	171	156	144	144
query84	293	99	79	79
query85	774	374	303	303
query86	395	286	303	286
query87	4365	4331	4234	4234
query88	4953	2414	2391	2391
query89	412	335	297	297
query90	1727	192	191	191
query91	137	135	109	109
query92	59	52	52	52
query93	2350	852	547	547
query94	700	407	305	305
query95	340	281	267	267
query96	484	599	285	285
query97	3200	3276	3220	3220
query98	220	210	197	197
query99	1538	1417	1314	1314
Total cold run time: 296577 ms
Total hot run time: 198056 ms

[doris-robot]

ClickBench: Total hot run time: 29.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fb7fe5cffb33714284e53e9e74ae4c8d1988cf42, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.23	0.07	0.07
query4	1.63	0.10	0.11
query5	0.53	0.51	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.55	0.51	0.51
query10	0.56	0.54	0.54
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.62	0.60	0.60
query14	0.80	0.79	0.80
query15	0.84	0.83	0.84
query16	0.39	0.39	0.38
query17	1.09	1.08	0.97
query18	0.24	0.23	0.21
query19	1.94	1.85	1.94
query20	0.01	0.01	0.01
query21	15.38	0.89	0.59
query22	0.75	0.78	0.53
query23	15.26	1.44	0.59
query24	3.62	1.71	1.16
query25	0.23	0.14	0.17
query26	0.26	0.15	0.14
query27	0.06	0.05	0.05
query28	13.95	1.00	0.43
query29	12.61	3.90	3.26
query30	0.24	0.08	0.06
query31	2.84	0.60	0.39
query32	3.21	0.55	0.45
query33	2.94	2.96	3.05
query34	16.72	5.17	4.61
query35	4.59	4.51	4.64
query36	0.65	0.49	0.51
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.14
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.77 s
Total hot run time: 29.08 s