Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Nov 7, 2024

Cherry-picked from #43376

mtmv deadlock when

- table1 is olapTable
- mv1 is MTMV create by table1
- mv2 is MTMV create by table1

reason:
- mtmv-task-thread-1: refresh mv1,  blocked write lock :mv1
- show partitions: hold read lock: mv1 ,block read lock: table1
- insert into table1: block write lock: table1
- mtmv-task-thread-2: refresh mv2, hold read lock: table1, blocked read
lock :mv1

Problem Summary:

show partitions: not hold readLock of olapTable and MTMV at same time
getAndCopyPartitionItems: set max wait time


Co-authored-by: zhangdong <zhangdong@selectdb.com>
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40598 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f0b41656d580a4b0ac334e36ebdf83c77bc5c2f8, data reload: false

------ Round 1 ----------------------------------
q1	17564	7482	7364	7364
q2	2065	174	170	170
q3	10833	1089	1175	1089
q4	10544	775	733	733
q5	7759	2856	2848	2848
q6	231	146	149	146
q7	981	602	606	602
q8	9608	1971	2032	1971
q9	8079	6423	6444	6423
q10	7008	2252	2348	2252
q11	458	266	259	259
q12	397	212	213	212
q13	17766	2986	3007	2986
q14	233	210	252	210
q15	570	510	510	510
q16	685	589	601	589
q17	974	577	521	521
q18	7242	6657	6582	6582
q19	1802	986	953	953
q20	469	193	199	193
q21	3921	3254	3028	3028
q22	1098	957	977	957
Total cold run time: 110287 ms
Total hot run time: 40598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7625	7260	7215	7215
q2	402	302	307	302
q3	2992	2896	2901	2896
q4	2028	1741	1721	1721
q5	5693	5727	5704	5704
q6	226	143	141	141
q7	2159	1789	1779	1779
q8	3402	3544	3417	3417
q9	8870	8937	8776	8776
q10	3546	3807	3544	3544
q11	601	497	496	496
q12	827	605	616	605
q13	16484	3179	3159	3159
q14	293	262	274	262
q15	603	545	546	545
q16	701	672	669	669
q17	1865	1609	1628	1609
q18	8174	7983	7403	7403
q19	2394	1631	1665	1631
q20	2074	1834	1906	1834
q21	5316	5239	5358	5239
q22	1112	1054	1010	1010
Total cold run time: 77387 ms
Total hot run time: 59957 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193426 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f0b41656d580a4b0ac334e36ebdf83c77bc5c2f8, data reload: false

query1	2493	2250	2247	2247
query2	6289	2170	2122	2122
query3	14377	10237	240	240
query4	33376	23627	23815	23627
query5	4039	440	427	427
query6	286	176	168	168
query7	3976	300	311	300
query8	280	239	218	218
query9	9345	2708	2694	2694
query10	496	265	260	260
query11	18135	15194	15170	15170
query12	165	101	95	95
query13	1548	423	410	410
query14	9306	7149	6767	6767
query15	218	184	181	181
query16	7735	479	491	479
query17	1535	597	567	567
query18	2291	625	592	592
query19	237	188	184	184
query20	127	115	114	114
query21	208	114	110	110
query22	4676	4395	4231	4231
query23	35159	33964	33859	33859
query24	9532	3395	3391	3391
query25	554	410	409	409
query26	732	183	185	183
query27	1982	295	301	295
query28	6318	2526	2513	2513
query29	676	450	441	441
query30	398	305	301	301
query31	1039	778	835	778
query32	95	55	57	55
query33	581	279	282	279
query34	891	498	493	493
query35	834	717	702	702
query36	1059	935	961	935
query37	118	72	70	70
query38	3941	3829	3891	3829
query39	1475	1430	1419	1419
query40	206	100	102	100
query41	52	52	49	49
query42	105	98	93	93
query43	534	490	500	490
query44	1140	781	783	781
query45	186	168	161	161
query46	1128	719	713	713
query47	1901	1821	1842	1821
query48	450	367	382	367
query49	892	417	396	396
query50	806	414	405	405
query51	7221	7173	7182	7173
query52	109	85	92	85
query53	259	188	191	188
query54	601	485	441	441
query55	74	74	73	73
query56	243	237	225	225
query57	1182	1096	1134	1096
query58	233	201	230	201
query59	3373	3194	2850	2850
query60	285	249	246	246
query61	132	106	106	106
query62	840	673	655	655
query63	213	200	193	193
query64	3676	635	607	607
query65	3252	3155	3185	3155
query66	811	303	300	300
query67	15918	15468	15202	15202
query68	4462	577	596	577
query69	426	254	255	254
query70	1153	1128	1179	1128
query71	375	265	256	256
query72	6239	3912	3915	3912
query73	754	340	339	339
query74	10043	8879	8930	8879
query75	3334	2626	2621	2621
query76	2355	1064	915	915
query77	388	261	260	260
query78	10894	9783	9565	9565
query79	8018	586	578	578
query80	2000	427	414	414
query81	572	240	244	240
query82	1668	113	112	112
query83	288	158	156	156
query84	299	77	78	77
query85	1539	320	290	290
query86	464	293	288	288
query87	4439	4235	4246	4235
query88	5898	2400	2461	2400
query89	557	290	290	290
query90	2107	181	182	181
query91	185	146	144	144
query92	68	46	49	46
query93	7567	542	531	531
query94	939	285	275	275
query95	358	247	254	247
query96	632	278	297	278
query97	3327	3135	3156	3135
query98	215	192	193	192
query99	1788	1294	1284	1284
Total cold run time: 315695 ms
Total hot run time: 193426 ms

@dataroaring dataroaring reopened this Nov 10, 2024
@morrySnow morrySnow closed this Nov 11, 2024
@morrySnow morrySnow deleted the auto-pick-43376-branch-3.0 branch November 11, 2024 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants