Skip to content

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Nov 7, 2024

pick: #43376

mtmv deadlock when

- table1 is olapTable
- mv1 is MTMV create by table1
- mv2 is MTMV create by table1

reason:
- mtmv-task-thread-1: refresh mv1,  blocked write lock :mv1
- show partitions: hold read lock: mv1 ,block read lock: table1
- insert into table1: block write lock: table1
- mtmv-task-thread-2: refresh mv2, hold read lock: table1, blocked read
lock :mv1

Problem Summary:

show partitions: not hold readLock of olapTable and MTMV at same time
getAndCopyPartitionItems: set max wait time


Co-authored-by: zhangdong <zhangdong@selectdb.com>
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented Nov 7, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40641 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ee01f219e4247749c3904e0975db47a475fd5825, data reload: false

------ Round 1 ----------------------------------
q1	17564	7403	7314	7314
q2	2071	166	165	165
q3	10689	1080	1164	1080
q4	10552	741	780	741
q5	7732	2878	2775	2775
q6	237	149	141	141
q7	961	618	594	594
q8	9370	1927	2005	1927
q9	6593	6419	6398	6398
q10	7002	2300	2296	2296
q11	459	258	258	258
q12	406	216	209	209
q13	17774	2985	2976	2976
q14	248	206	224	206
q15	567	529	506	506
q16	669	607	634	607
q17	968	601	602	601
q18	7222	6535	6558	6535
q19	1383	1018	1040	1018
q20	467	197	202	197
q21	3953	3219	3123	3123
q22	1055	982	974	974
Total cold run time: 107942 ms
Total hot run time: 40641 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7285	7256	7216	7216
q2	412	318	301	301
q3	2884	2847	2908	2847
q4	2017	1779	1809	1779
q5	5643	5693	5705	5693
q6	218	139	138	138
q7	2159	1779	1791	1779
q8	3342	3590	3555	3555
q9	8822	8865	8792	8792
q10	3514	3522	3515	3515
q11	607	509	492	492
q12	806	602	590	590
q13	16470	3160	3109	3109
q14	293	283	273	273
q15	586	544	538	538
q16	716	666	679	666
q17	1864	1631	1605	1605
q18	8255	7832	7803	7803
q19	5980	1544	1459	1459
q20	2127	1848	1800	1800
q21	5517	5352	5370	5352
q22	1116	1018	1009	1009
Total cold run time: 80633 ms
Total hot run time: 60311 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193863 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ee01f219e4247749c3904e0975db47a475fd5825, data reload: false

query1	2491	2213	2244	2213
query2	6222	2163	2105	2105
query3	14470	10685	236	236
query4	34145	23620	23304	23304
query5	5575	443	431	431
query6	281	188	184	184
query7	3992	313	307	307
query8	297	249	229	229
query9	9415	2712	2705	2705
query10	518	270	259	259
query11	18194	15174	15238	15174
query12	152	98	101	98
query13	1546	437	413	413
query14	9238	6887	7187	6887
query15	240	180	172	172
query16	7819	505	467	467
query17	1500	563	560	560
query18	2407	605	637	605
query19	252	180	183	180
query20	122	112	110	110
query21	207	104	103	103
query22	4709	4366	4337	4337
query23	34570	33887	33769	33769
query24	8127	3339	3347	3339
query25	573	387	380	380
query26	739	192	185	185
query27	1966	293	297	293
query28	5784	2511	2498	2498
query29	754	413	419	413
query30	382	339	303	303
query31	1009	752	795	752
query32	81	53	51	51
query33	542	269	279	269
query34	896	494	505	494
query35	844	712	718	712
query36	1077	954	944	944
query37	116	68	73	68
query38	4141	3877	3862	3862
query39	1489	1435	1419	1419
query40	198	99	98	98
query41	51	51	51	51
query42	109	100	95	95
query43	522	515	495	495
query44	1150	792	797	792
query45	181	163	162	162
query46	1141	693	696	693
query47	1854	1793	1811	1793
query48	459	372	381	372
query49	888	403	391	391
query50	828	412	422	412
query51	7080	7161	7194	7161
query52	102	106	88	88
query53	260	187	182	182
query54	574	462	464	462
query55	80	77	76	76
query56	271	260	249	249
query57	1187	1079	1103	1079
query58	214	204	208	204
query59	3289	3020	2999	2999
query60	288	262	254	254
query61	131	140	126	126
query62	808	750	639	639
query63	215	190	186	186
query64	4003	726	698	698
query65	3281	3180	3185	3180
query66	768	310	313	310
query67	15953	15409	15329	15329
query68	4484	599	599	599
query69	434	257	262	257
query70	1127	1110	1189	1110
query71	360	263	257	257
query72	6352	3963	3910	3910
query73	755	347	336	336
query74	10142	8894	8968	8894
query75	3343	2625	2621	2621
query76	2486	1068	1043	1043
query77	365	264	266	264
query78	10690	9709	9704	9704
query79	6313	588	591	588
query80	2475	415	427	415
query81	589	254	246	246
query82	1332	116	110	110
query83	279	162	156	156
query84	288	81	77	77
query85	2198	298	283	283
query86	490	294	308	294
query87	4590	4217	4339	4217
query88	5079	2445	2452	2445
query89	567	296	299	296
query90	2162	195	184	184
query91	171	144	145	144
query92	67	46	48	46
query93	7160	540	537	537
query94	994	278	275	275
query95	342	249	245	245
query96	619	291	284	284
query97	3321	3147	3148	3147
query98	213	195	193	193
query99	1825	1280	1316	1280
Total cold run time: 314379 ms
Total hot run time: 193863 ms

@zddr
Copy link
Contributor Author

zddr commented Nov 7, 2024

run p0

@morrySnow morrySnow merged commit cab76c2 into apache:branch-3.0 Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants