Skip to content

branch-3.0: [fix](bdb) reset interrupted flag before calling bdbje #47874 #47941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Feb 24, 2025
Merged

branch-3.0: [fix](bdb) reset interrupted flag before calling bdbje #47874 #47941

dataroaring merged 1 commit into from
Feb 24, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Feb 14, 2025

Cherry-picked from #47874

@morningman
[fix](bdb) reset interrupted flag before calling bdbje (#47874)
b904f9c 
### What problem does this PR solve?

This pull request includes changes to ensure that the interrupted flag
of the current thread is reset before performing certain operations to
prevent failures in acquiring locks. The changes primarily affect the
`BDBEnvironment` and `BDBJEJournal` classes.

```
Caused by: java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326) ~[?:1.8.0_352-352]
        at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871) ~[?:1.8.0_352-352]
        at com.sleepycat.je.latch.SharedLatchImpl.acquireShared(SharedLatchImpl.java:103) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.tree.Tree.getRootINInternal(Tree.java:447) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.tree.Tree.getRootIN(Tree.java:431) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.tree.Tree.search(Tree.java:2185) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.tree.Tree.getFirstNode(Tree.java:812) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.CursorImpl.positionFirstOrLast(CursorImpl.java:1776) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.CursorImpl.traverseDbWithCursor(CursorImpl.java:3953) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.DbTree.getDbNames(DbTree.java:1808) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.getDatabaseNames(Environment.java:2458) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        ... 21 more
```

This a self-defend logic. Because we found that some times other logic
may set the thread as interrupted and
does not handle it.
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 14, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 14, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Feb 14, 2025

TPC-H: Total hot run time: 40574 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b904f9c7beede1373720d608ac02d2f81405e77d, data reload: false

------ Round 1 ----------------------------------
q1	17604	7309	7209	7209
q2	2055	173	167	167
q3	10566	1089	1165	1089
q4	10578	786	747	747
q5	7723	2828	2746	2746
q6	234	152	142	142
q7	994	614	602	602
q8	9350	1910	2009	1910
q9	6511	6386	6370	6370
q10	6987	2318	2329	2318
q11	463	270	262	262
q12	405	212	207	207
q13	17801	2993	2975	2975
q14	228	209	209	209
q15	562	518	531	518
q16	662	578	565	565
q17	968	532	566	532
q18	7108	6625	6707	6625
q19	1403	1082	1082	1082
q20	469	202	196	196
q21	3922	3150	3134	3134
q22	1078	1012	969	969
Total cold run time: 107671 ms
Total hot run time: 40574 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7242	7169	7200	7169
q2	329	235	230	230
q3	2922	2890	2889	2889
q4	2054	1861	1877	1861
q5	5692	5679	5782	5679
q6	231	143	141	141
q7	2267	1797	1857	1797
q8	3359	3529	3516	3516
q9	8753	8826	8833	8826
q10	3622	3580	3546	3546
q11	612	483	490	483
q12	818	582	610	582
q13	9193	3135	3167	3135
q14	308	300	270	270
q15	573	531	532	531
q16	702	640	655	640
q17	1814	1601	1579	1579
q18	8223	7708	7557	7557
q19	1665	1580	1639	1580
q20	2131	1843	1874	1843
q21	5584	5330	5264	5264
q22	1154	1019	1043	1019
Total cold run time: 69248 ms
Total hot run time: 60137 ms

@doris-robot
Copy link

doris-robot commented Feb 14, 2025

TPC-DS: Total hot run time: 197764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b904f9c7beede1373720d608ac02d2f81405e77d, data reload: false

query1	1308	956	903	903
query2	6247	2023	2001	2001
query3	10802	4363	4355	4355
query4	65943	28264	23818	23818
query5	5023	464	460	460
query6	415	191	198	191
query7	5540	312	305	305
query8	311	215	215	215
query9	8484	2691	2673	2673
query10	457	277	252	252
query11	17480	15240	15759	15240
query12	164	99	103	99
query13	1432	432	431	431
query14	10497	6781	7175	6781
query15	209	186	178	178
query16	7150	479	468	468
query17	1107	576	578	576
query18	1957	321	325	321
query19	218	166	166	166
query20	120	113	111	111
query21	206	104	108	104
query22	4556	4381	4703	4381
query23	34380	34026	34051	34026
query24	6188	2944	2915	2915
query25	537	434	440	434
query26	651	173	174	173
query27	1908	361	360	360
query28	4331	2481	2481	2481
query29	687	438	423	423
query30	252	164	160	160
query31	1008	844	832	832
query32	70	55	57	55
query33	400	288	290	288
query34	923	507	514	507
query35	865	717	715	715
query36	1062	964	980	964
query37	123	70	76	70
query38	4050	4051	4129	4051
query39	1517	1470	1449	1449
query40	208	97	98	97
query41	48	51	47	47
query42	113	105	99	99
query43	520	488	470	470
query44	1204	836	836	836
query45	188	173	168	168
query46	1161	725	711	711
query47	2023	1907	1955	1907
query48	507	391	410	391
query49	725	406	394	394
query50	878	431	430	430
query51	7313	7202	7042	7042
query52	101	89	87	87
query53	253	176	177	176
query54	560	454	452	452
query55	77	72	77	72
query56	258	233	244	233
query57	1246	1125	1085	1085
query58	204	198	212	198
query59	3079	2853	2770	2770
query60	275	255	246	246
query61	106	103	107	103
query62	860	713	721	713
query63	212	187	192	187
query64	1434	677	660	660
query65	3245	3261	3215	3215
query66	724	298	302	298
query67	15825	15847	15626	15626
query68	4106	585	566	566
query69	440	264	265	264
query70	1137	1108	1111	1108
query71	361	264	258	258
query72	6376	4234	4261	4234
query73	750	353	352	352
query74	10175	9184	9218	9184
query75	3367	2658	2648	2648
query76	1855	1035	1056	1035
query77	505	264	262	262
query78	10594	9642	9572	9572
query79	1161	604	599	599
query80	843	429	427	427
query81	512	242	240	240
query82	1317	120	124	120
query83	256	142	146	142
query84	289	83	79	79
query85	881	314	285	285
query86	332	294	272	272
query87	4416	4352	4300	4300
query88	3520	2434	2375	2375
query89	417	287	289	287
query90	2011	187	183	183
query91	176	152	150	150
query92	66	50	49	49
query93	1317	544	546	544
query94	772	283	285	283
query95	352	251	251	251
query96	606	279	290	279
query97	3332	3162	3229	3162
query98	206	201	198	198
query99	1912	1457	1411	1411
Total cold run time: 316285 ms
Total hot run time: 197764 ms

@doris-robot
Copy link

doris-robot commented Feb 14, 2025

ClickBench: Total hot run time: 33.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b904f9c7beede1373720d608ac02d2f81405e77d, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.66	0.10	0.11
query5	0.53	0.52	0.50
query6	1.13	0.73	0.74
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.50	0.50
query10	0.55	0.57	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.62	0.61
query14	2.74	2.74	2.80
query15	0.85	0.84	0.82
query16	0.38	0.40	0.39
query17	0.99	1.03	1.01
query18	0.24	0.22	0.22
query19	1.95	2.04	1.87
query20	0.02	0.01	0.01
query21	15.37	0.59	0.61
query22	2.72	2.32	2.06
query23	18.04	1.42	0.82
query24	3.89	1.53	1.84
query25	0.29	0.17	0.11
query26	0.73	0.15	0.15
query27	0.05	0.04	0.06
query28	11.09	1.14	1.08
query29	12.71	3.21	3.23
query30	0.27	0.06	0.06
query31	2.93	0.39	0.39
query32	3.38	0.46	0.46
query33	3.01	3.02	3.01
query34	17.00	4.47	4.43
query35	4.49	4.46	4.48
query36	0.66	0.50	0.49
query37	0.10	0.06	0.07
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 110.06 s
Total hot run time: 33.64 s

dataroaring
dataroaring approved these changes Feb 24, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit fc05629 into branch-3.0 Feb 24, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-47874-branch-3.0 branch February 24, 2025 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @morningman