[pipeline](fix) Avoid to use a freed dependency when cancelled #34584

Gabriel39 · 2024-05-09T07:13:47Z

Proposed changes

*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1715228866 (unix time) try "date -d @1715228866" if you are using GNU date ***
*** Current BE git commitID: 6849cd5 ***
*** SIGSEGV address not mapped to object (@0x41) received by PID 9394 (TID 11207 OR 0x7f2db5012700) from PID 65; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
3# 0x00007F35F1187090 in /lib/x86_64-linux-gnu/libc.so.6
4# std::atomic::operator bool() const at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87
5# doris::pipeline::Dependency::set_ready() at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/dependency.cpp:56
6# doris::pipeline::PipelineTask::clear_blocking_state() at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.h:199
7# doris::pipeline::PipelineFragmentContext::cancel(doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_fragment_context.cpp:219
8# doris::QueryContext::cancel_pipeline_context(int, doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_master/doris/be/src/runtime/query_context.cpp:261
9# doris::FragmentMgr::cancel_fragment(doris::TUniqueId const&, int, doris::PPlanFragmentCancelReason const&, std::__cxx11::basic_string, std::allocator > const&) at /home/zcp/repo_center/doris_master/doris/be/src/runtime/fragment_mgr.cpp:1007
10# doris::PInternalService::cancel_plan_fragment(google::protobuf::RpcController*, doris::PCancelPlanFragmentRequest const*, doris::PCancelPlanFragmentResult*, google::protobuf::Closure*)::$_0::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/service/internal_service.cpp:582
11# void std::__invoke_impl(std::__invoke_other, doris::PInternalService::cancel_plan_fragment(google::protobuf::RpcController*, doris::PCancelPlanFragmentRequest const*, doris::PCancelPlanFragmentResult*, google::protobuf::Closure*)::$_0&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
12# std::enable_if, void>::type std::__invoke_r(doris::PInternalService::cancel_plan_fragment(google::protobuf::RpcController*, doris::PCancelPlanFragmentRequest const*, doris::PCancelPlanFragmentResult*, google::protobuf::Closure*)::$_0&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
13# std::_Function_handler::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
14# std::function::operator()() const at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
15# doris::WorkThreadPool::work_thread(int) at /home/zcp/repo_center/doris_master/doris/be/src/util/work_thread_pool.hpp:158
16# void std::__invoke_impl::* const&)(int), doris::WorkThreadPool*&, int&>(std::__invoke_memfun_deref, void (doris::WorkThreadPool::* const&)(int), doris::WorkThreadPool*&, int&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
17# std::__invoke_result::* const&)(int), doris::WorkThreadPool*&, int&>::type std::__invoke::* const&)(int), doris::WorkThreadPool*&, int&>(void (doris::WorkThreadPool::* const&)(int), doris::WorkThreadPool*&, int&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
18# decltype (std::__invoke((this)._M_pmf, std::forward&>({parm#1}), std::forward({parm#1}))) std::_Mem_fn_base::)(int), true>::operator()&, int&>(doris::WorkThreadPool*&, int&) const at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:131
19# void std::__invoke_impl::)(int)>&, doris::WorkThreadPool&, int&>(std::__invoke_other, std::_Mem_fn::)(int)>&, doris::WorkThreadPool&, int&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
20# std::enable_if::)(int)>&, doris::WorkThreadPool&, int&>, void>::type std::__invoke_r::)(int)>&, doris::WorkThreadPool&, int&>(std::_Mem_fn::)(int)>&, doris::WorkThreadPool&, int&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
21# void std::_Bind_result::)(int)> (doris::WorkThreadPool, int)>::__call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:570
22# void std::_Bind_result::)(int)> (doris::WorkThreadPool, int)>::operator()<>() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:629
23# void std::__invoke_impl::)(int)> (doris::WorkThreadPool, int)>>(std::__invoke_other, std::_Bind_result::)(int)> (doris::WorkThreadPool, int)>&&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
24# std::__invoke_result::)(int)> (doris::WorkThreadPool, int)>>::type std::__invoke::)(int)> (doris::WorkThreadPool, int)>>(std::_Bind_result::)(int)> (doris::WorkThreadPool, int)>&&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
25# void std::thread::_Invoker::)(int)> (doris::WorkThreadPool, int)> > >::_M_invoke<0ul>(std::_Index_tuple<0ul>) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253
26# std::thread::_Invoker::)(int)> (doris::WorkThreadPool, int)> > >::operator()() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260
27# std::thread::_State_impl::)(int)> (doris::WorkThreadPool, int)> > > >::_M_run() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211
28# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84
29# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
30# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-09T07:13:53Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Gabriel39 · 2024-05-09T07:13:55Z

run buildall

github-actions · 2024-05-09T07:19:23Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-09T07:38:01Z

TeamCity be ut coverage result:
Function Coverage: 35.69% (8983/25172)
Line Coverage: 27.34% (74217/271469)
Region Coverage: 26.57% (38364/144385)
Branch Coverage: 23.39% (19569/83664)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e04afdb11be86d99a139192609f2eed954ea2c2b_e04afdb11be86d99a139192609f2eed954ea2c2b/report/index.html

doris-robot · 2024-05-09T07:46:29Z

TPC-H: Total hot run time: 41746 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e04afdb11be86d99a139192609f2eed954ea2c2b, data reload: false

------ Round 1 ----------------------------------
q1	17602	4298	4204	4204
q2	2020	184	186	184
q3	10475	1228	1186	1186
q4	10199	822	763	763
q5	7475	2730	2725	2725
q6	216	132	135	132
q7	1025	596	591	591
q8	9242	2177	2105	2105
q9	9846	6726	6712	6712
q10	9227	3963	3820	3820
q11	449	251	254	251
q12	455	237	225	225
q13	17353	3167	3314	3167
q14	263	221	211	211
q15	499	468	471	468
q16	491	412	425	412
q17	976	720	744	720
q18	8299	7857	7697	7697
q19	7289	1518	1556	1518
q20	649	346	321	321
q21	5209	4074	4054	4054
q22	355	283	280	280
Total cold run time: 119614 ms
Total hot run time: 41746 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4505	4422	4417	4417
q2	376	282	280	280
q3	3224	2978	2881	2881
q4	1838	1548	1556	1548
q5	5495	5493	5510	5493
q6	217	122	124	122
q7	2321	1953	1972	1953
q8	3245	3384	3392	3384
q9	8645	8695	8579	8579
q10	3901	3869	3805	3805
q11	605	490	486	486
q12	820	598	624	598
q13	16289	3151	3174	3151
q14	287	269	266	266
q15	529	474	472	472
q16	467	412	420	412
q17	1770	1483	1464	1464
q18	7778	7674	7491	7491
q19	1664	1573	1576	1573
q20	1953	1766	1742	1742
q21	9888	4796	4821	4796
q22	542	483	475	475
Total cold run time: 76359 ms
Total hot run time: 55388 ms

doris-robot · 2024-05-09T07:57:46Z

TPC-DS: Total hot run time: 186250 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e04afdb11be86d99a139192609f2eed954ea2c2b, data reload: false

query1	912	367	345	345
query2	6746	2465	2274	2274
query3	6640	208	219	208
query4	23133	21319	21254	21254
query5	4203	410	418	410
query6	267	191	170	170
query7	4579	293	281	281
query8	236	185	184	184
query9	8546	2409	2398	2398
query10	440	259	264	259
query11	14755	14153	14165	14153
query12	132	93	88	88
query13	1655	373	362	362
query14	8670	8498	6971	6971
query15	218	172	170	170
query16	7746	255	266	255
query17	1729	544	540	540
query18	1943	279	266	266
query19	197	147	149	147
query20	93	82	88	82
query21	192	131	128	128
query22	4994	4823	4827	4823
query23	34305	33606	33585	33585
query24	6301	2810	2831	2810
query25	452	366	370	366
query26	691	155	148	148
query27	1837	309	326	309
query28	3688	2053	2052	2052
query29	823	593	591	591
query30	235	149	152	149
query31	962	764	728	728
query32	58	52	50	50
query33	498	242	241	241
query34	888	478	474	474
query35	759	689	657	657
query36	1048	945	960	945
query37	106	62	64	62
query38	2882	2783	2748	2748
query39	1643	1557	1561	1557
query40	201	124	121	121
query41	44	38	38	38
query42	102	93	97	93
query43	563	567	522	522
query44	1086	721	735	721
query45	257	252	225	225
query46	1056	725	721	721
query47	1952	1887	1857	1857
query48	374	299	301	299
query49	773	393	401	393
query50	769	387	385	385
query51	6860	6845	6750	6750
query52	101	85	90	85
query53	349	280	277	277
query54	535	428	453	428
query55	73	72	70	70
query56	245	217	222	217
query57	1230	1139	1166	1139
query58	215	200	199	199
query59	3436	3261	3228	3228
query60	254	228	235	228
query61	88	85	89	85
query62	597	463	477	463
query63	314	279	279	279
query64	7638	7422	7335	7335
query65	3138	3092	3089	3089
query66	782	339	327	327
query67	15323	14912	15016	14912
query68	4617	533	525	525
query69	493	301	305	301
query70	1153	1166	1143	1143
query71	374	267	266	266
query72	7908	2571	2318	2318
query73	696	323	318	318
query74	6547	6066	6187	6066
query75	3337	2624	2638	2624
query76	2302	946	961	946
query77	558	263	264	263
query78	10624	10178	10182	10178
query79	1823	509	506	506
query80	859	425	423	423
query81	501	248	221	221
query82	960	92	88	88
query83	239	165	161	161
query84	254	88	85	85
query85	893	260	255	255
query86	423	325	280	280
query87	3307	3081	3108	3081
query88	3504	2458	2472	2458
query89	496	393	389	389
query90	2109	189	187	187
query91	124	95	99	95
query92	60	50	50	50
query93	1695	536	499	499
query94	1140	180	182	180
query95	392	307	295	295
query96	587	275	265	265
query97	3144	2922	2960	2922
query98	240	221	220	220
query99	1163	916	935	916
Total cold run time: 266079 ms
Total hot run time: 186250 ms

HappenLee

LGTM

Gabriel39 · 2024-05-10T04:03:59Z

run buildall

github-actions · 2024-05-10T05:02:37Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-10T05:02:39Z

PR approved by anyone and no changes requested.

doris-robot · 2024-05-10T05:26:12Z

TeamCity be ut coverage result:
Function Coverage: 35.68% (8982/25171)
Line Coverage: 27.33% (74204/271493)
Region Coverage: 26.57% (38354/144369)
Branch Coverage: 23.39% (19561/83638)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6d292429c7cdcbc4590c7d235e72bdfa56f9bd6b_6d292429c7cdcbc4590c7d235e72bdfa56f9bd6b/report/index.html

Gabriel39 · 2024-05-10T06:30:56Z

run buildall

doris-robot · 2024-05-10T07:15:19Z

TeamCity be ut coverage result:
Function Coverage: 35.68% (8982/25171)
Line Coverage: 27.33% (74201/271505)
Region Coverage: 26.57% (38362/144376)
Branch Coverage: 23.39% (19563/83638)
Coverage Report: http://coverage.selectdb-in.cc/coverage/80223933661e8be6788c60a5bd9058b076ea07d6_80223933661e8be6788c60a5bd9058b076ea07d6/report/index.html

Gabriel39 · 2024-05-10T13:15:29Z

run buildall

doris-robot · 2024-05-10T13:48:07Z

TeamCity be ut coverage result:
Function Coverage: 35.67% (8978/25172)
Line Coverage: 27.34% (74228/271548)
Region Coverage: 26.58% (38380/144408)
Branch Coverage: 23.39% (19565/83650)
Coverage Report: http://coverage.selectdb-in.cc/coverage/80223933661e8be6788c60a5bd9058b076ea07d6_80223933661e8be6788c60a5bd9058b076ea07d6/report/index.html

github-actions · 2024-05-11T03:26:05Z

PR approved by at least one committer and no changes requested.

HappenLee

LGTM

…e#34584)

… (#38046) ## Proposed changes pick #34584

HappenLee previously approved these changes May 10, 2024

View reviewed changes

update

bf8477b

Gabriel39 dismissed HappenLee’s stale review via bf8477b May 10, 2024 04:02

Gabriel39 force-pushed the fix_0509 branch from e04afdb to bf8477b Compare May 10, 2024 04:02

update

6d29242

yiguolei previously approved these changes May 10, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 10, 2024

github-actions bot added the reviewed label May 10, 2024

update

8022393

Gabriel39 dismissed yiguolei’s stale review via 8022393 May 10, 2024 06:29

github-actions bot removed the approved Indicates a PR has been approved by one committer. label May 10, 2024

yiguolei approved these changes May 11, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 11, 2024

HappenLee approved these changes May 11, 2024

View reviewed changes

Gabriel39 merged commit b891501 into apache:master May 11, 2024

ByteYue pushed a commit to ByteYue/doris that referenced this pull request May 15, 2024

[pipeline](fix) Avoid to use a freed dependency when cancelled (apach…

0a7f293

…e#34584)

Gabriel39 added a commit to Gabriel39/incubator-doris that referenced this pull request Jul 18, 2024

[pipeline](fix) Avoid to use a freed dependency when cancelled (apach…

0ac279f

…e#34584)

Gabriel39 mentioned this pull request Jul 18, 2024

[pipeline](fix) Avoid to use a freed dependency when cancelled (#34584) #38046

Merged

yiguolei pushed a commit that referenced this pull request Jul 18, 2024

[pipeline](fix) Avoid to use a freed dependency when cancelled (#34584)…

88d771d

… (#38046) ## Proposed changes pick #34584

yiguolei added the dev/2.1.5-merged label Jul 18, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[pipeline](fix) Avoid to use a freed dependency when cancelled #34584

[pipeline](fix) Avoid to use a freed dependency when cancelled #34584

Uh oh!

Conversation

Gabriel39 commented May 9, 2024

Proposed changes

Further comments

Uh oh!

doris-robot commented May 9, 2024

Uh oh!

Gabriel39 commented May 9, 2024

Uh oh!

github-actions bot commented May 9, 2024

Uh oh!

doris-robot commented May 9, 2024

Uh oh!

doris-robot commented May 9, 2024

Uh oh!

doris-robot commented May 9, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

Gabriel39 commented May 10, 2024

Uh oh!

github-actions bot commented May 10, 2024

Uh oh!

github-actions bot commented May 10, 2024

Uh oh!

doris-robot commented May 10, 2024

Uh oh!

Gabriel39 commented May 10, 2024

Uh oh!

doris-robot commented May 10, 2024

Uh oh!

Gabriel39 commented May 10, 2024

Uh oh!

doris-robot commented May 10, 2024

Uh oh!

github-actions bot commented May 11, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants