[Bug](runtime-filter) fix brpc ctrl use after free and remove apply filter v1 #35186

BiteTheDDDDt · 2024-05-22T03:52:19Z

Proposed changes

do not pick this pr to 2.1 coz will broken rolling upgrade 2.0 to 2.1 compatibility

==3466609==ERROR: AddressSanitizer: heap-use-after-free on address 0x618002362928 at pc 0x55862351a934 bp 0x7fc25fc84290 sp 0x7fc25fc84288
READ of size 8 at 0x618002362928 thread T1997
pure virtual method called
terminate called without an active exception
*** Query id: 99cf19b643a34f40-836df91f53120e33 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1716277295 (unix time) try "date -d @1716277295" if you are using GNU date ***
*** Current BE git commitID: 6fe533eede ***
*** SIGABRT unknown detail explain (@0x34e571) received by PID 3466609 (TID 3470467 OR 0x7fc17f0c3700) from PID 3466609; stack trace: ***
pure virtual method called
terminate called recursively
pure virtual method called
terminate called recursively
    #0 0x55862351a933 in std::__cxx11::basic_string, std::allocator>::size() const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:907:16
    #1 0x558623683fe3 in std::__cxx11::basic_string, std::allocator>::resize(unsigned long, char) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:375:38
    #2 0x5586591e228d  (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c54828d) (BuildId: 96775b8c13e02111)
    #3 0x5586591e254c in butil::string_appendf(std::__cxx11::basic_string, std::allocator>*, char const*, ...) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c54854c) (BuildId: 96775b8c13e02111)
    #4 0x558659213f85 in brpc::Controller::SetFailed(int, char const*, ...) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c579f85) (BuildId: 96775b8c13e02111)
    #5 0x558659218b35 in brpc::Controller::HandleSocketFailed(bthread_id_t, void*, int, std::__cxx11::basic_string, std::allocator> const&) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c57eb35) (BuildId: 96775b8c13e02111)
    #6 0x5586592109c9 in bthread_id_error_verbose (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c5769c9) (BuildId: 96775b8c13e02111)
    #7 0x5586591d0a51 in bthread::TimerThread::Task::run_and_delete() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c536a51) (BuildId: 96775b8c13e02111)
    #8 0x5586591d141a in bthread::TimerThread::run() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c53741a) (BuildId: 96775b8c13e02111)
    #9 0x5586591d1e78 in bthread::TimerThread::run_this(void*) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x5c537e78) (BuildId: 96775b8c13e02111)
    #10 0x7fca8abd7608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    #11 0x7fca8ae84132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x618002362928 is located 168 bytes inside of 792-byte region [0x618002362880,0x618002362b98)
freed by thread T1917 (brpc_light) here:
    #0 0x558623500d9d in operator delete(void*) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x26866d9d) (BuildId: 96775b8c13e02111)
    #1 0x558627179365 in std::default_delete>::operator()(doris::AsyncRPCContext*) const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
    #2 0x55862717924b in std::unique_ptr, std::default_delete>>::~unique_ptr() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
    #3 0x558627179186 in void std::destroy_at, std::default_delete>>>(std::unique_ptr, std::default_delete>>*) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
    #4 0x558627179156 in void std::_Destroy, std::default_delete>>>(std::unique_ptr, std::default_delete>>*) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
    #5 0x55862717910a in void std::_Destroy_aux::__destroy, std::default_delete>>*>(std::unique_ptr, std::default_delete>>*, std::unique_ptr, std::default_delete>>*) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
    #6 0x5586271790ae in void std::_Destroy, std::default_delete>>*>(std::unique_ptr, std::default_delete>>*, std::unique_ptr, std::default_delete>>*) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
    #7 0x558627178f72 in void std::_Destroy, std::default_delete>>*, std::unique_ptr, std::default_delete>>>(std::unique_ptr, std::default_delete>>*, std::unique_ptr, std::default_delete>>*, std::allocator, std::default_delete>>>&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
    #8 0x5586271534d0 in std::vector, std::default_delete>>, std::allocator, std::default_delete>>>>::~vector() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
    #9 0x55862713e124 in doris::RuntimeFilterMergeControllerEntity::merge(doris::PMergeFilterRequest const*, butil::IOBufAsZeroCopyInputStream*, bool) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/runtime_filter_mgr.cpp:507:9
    #10 0x558626cdacea in doris::FragmentMgr::merge_filter(doris::PMergeFilterRequest const*, butil::IOBufAsZeroCopyInputStream*) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/fragment_mgr.cpp:1480:44
    #11 0x558627564e40 in doris::PInternalServiceImpl::merge_filter(google::protobuf::RpcController*, doris::PMergeFilterRequest const*, doris::PMergeFilterResponse*, google::protobuf::Closure*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-2.1/doris/be/src/service/internal_service.cpp:1299:48
    #12 0x558627564b96 in void std::__invoke_impl(std::__invoke_other, doris::PInternalServiceImpl::merge_filter(google::protobuf::RpcController*, doris::PMergeFilterRequest const*, doris::PMergeFilterResponse*, google::protobuf::Closure*)::$_0&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #13 0x558627564b08 in std::enable_if, void>::type std::__invoke_r(doris::PInternalServiceImpl::merge_filter(google::protobuf::RpcController*, doris::PMergeFilterRequest const*, doris::PMergeFilterResponse*, google::protobuf::Closure*)::$_0&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
    #14 0x5586275648de in std::_Function_handler::_M_invoke(std::_Any_data const&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
    #15 0x55862375c4e6 in std::function::operator()() const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
    #16 0x5586275a5fae in doris::WorkThreadPool::work_thread(int) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/work_thread_pool.hpp:158:17
    #17 0x5586275a75c4 in void std::__invoke_impl::* const&)(int), doris::WorkThreadPool*&, int&>(std::__invoke_memfun_deref, void (doris::WorkThreadPool::* const&)(int), doris::WorkThreadPool*&, int&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14
    #18 0x5586275a7486 in std::__invoke_result::* const&)(int), doris::WorkThreadPool*&, int&>::type std::__invoke::* const&)(int), doris::WorkThreadPool*&, int&>(void (doris::WorkThreadPool::* const&)(int), doris::WorkThreadPool*&, int&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
    #19 0x5586275a7446 in decltype(std::__invoke((*this)._M_pmf, std::forward*&>(fp), std::forward(fp))) std::_Mem_fn_base::*)(int), true>::operator()*&, int&>(doris::WorkThreadPool*&, int&) const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:131:11
    #20 0x5586275a7406 in void std::__invoke_impl::*)(int)>&, doris::WorkThreadPool*&, int&>(std::__invoke_other, std::_Mem_fn::*)(int)>&, doris::WorkThreadPool*&, int&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #21 0x5586275a7338 in std::enable_if::*)(int)>&, doris::WorkThreadPool*&, int&>, void>::type std::__invoke_r::*)(int)>&, doris::WorkThreadPool*&, int&>(std::_Mem_fn::*)(int)>&, doris::WorkThreadPool*&, int&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
    #22 0x5586275a7285 in void std::_Bind_result::*)(int)> (doris::WorkThreadPool*, int)>::__call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:570:11
    #23 0x5586275a70df in void std::_Bind_result::*)(int)> (doris::WorkThreadPool*, int)>::operator()<>() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:629:17
    #24 0x5586275a6fe6 in void std::__invoke_impl::*)(int)> (doris::WorkThreadPool*, int)>>(std::__invoke_other, std::_Bind_result::*)(int)> (doris::WorkThreadPool*, int)>&&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #25 0x5586275a6f86 in std::__invoke_result::*)(int)> (doris::WorkThreadPool*, int)>>::type std::__invoke::*)(int)> (doris::WorkThreadPool*, int)>>(std::_Bind_result::*)(int)> (doris::WorkThreadPool*, int)>&&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
    #26 0x5586275a6f4e in void std::thread::_Invoker::*)(int)> (doris::WorkThreadPool*, int)>>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13
    #27 0x5586275a6f16 in std::thread::_Invoker::*)(int)> (doris::WorkThreadPool*, int)>>>::operator()() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11
    #28 0x5586275a6e5a in std::thread::_State_impl::*)(int)> (doris::WorkThreadPool*, int)>>>>::_M_run() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13
    #29 0x55865a9237ef in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-22T03:52:25Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

BiteTheDDDDt · 2024-05-22T03:52:26Z

run buildall

github-actions · 2024-05-22T03:58:11Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-22T04:49:10Z

TPC-H: Total hot run time: 42195 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 68e1a4a731337dc63c5660f1e10d6631450956eb, data reload: false

------ Round 1 ----------------------------------
q1	17597	4742	4237	4237
q2	2020	186	189	186
q3	10482	1238	1162	1162
q4	10187	877	862	862
q5	7467	2786	2729	2729
q6	220	130	134	130
q7	1035	615	598	598
q8	9219	2182	2147	2147
q9	9297	6744	6730	6730
q10	9387	4029	3902	3902
q11	438	241	241	241
q12	461	224	235	224
q13	18124	3188	3277	3188
q14	257	217	229	217
q15	510	477	480	477
q16	512	394	386	386
q17	1006	731	687	687
q18	8531	7868	7744	7744
q19	3435	1565	1582	1565
q20	666	338	322	322
q21	5334	4193	4186	4186
q22	364	282	275	275
Total cold run time: 116549 ms
Total hot run time: 42195 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4512	4396	4405	4396
q2	376	266	280	266
q3	3138	2988	2746	2746
q4	1898	1576	1637	1576
q5	5556	5517	5517	5517
q6	216	119	121	119
q7	2348	1979	1987	1979
q8	3276	3417	3431	3417
q9	8749	8712	8561	8561
q10	4007	3873	3843	3843
q11	615	487	514	487
q12	811	627	603	603
q13	15834	3148	3160	3148
q14	293	267	274	267
q15	540	479	480	479
q16	483	423	447	423
q17	1776	1479	1446	1446
q18	7738	8001	7543	7543
q19	1703	1529	1580	1529
q20	1956	1816	1758	1758
q21	8100	5046	4829	4829
q22	561	493	486	486
Total cold run time: 74486 ms
Total hot run time: 55418 ms

doris-robot · 2024-05-22T05:00:32Z

TPC-DS: Total hot run time: 180751 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 68e1a4a731337dc63c5660f1e10d6631450956eb, data reload: false

query1	921	387	374	374
query2	6438	2448	2277	2277
query3	6652	210	214	210
query4	22800	21221	21348	21221
query5	4105	414	426	414
query6	257	174	175	174
query7	4575	293	285	285
query8	239	185	188	185
query9	8594	2393	2386	2386
query10	436	252	290	252
query11	14637	14264	14172	14172
query12	144	91	96	91
query13	1651	369	369	369
query14	9846	8242	6869	6869
query15	197	164	169	164
query16	7757	260	252	252
query17	1750	567	534	534
query18	1952	274	265	265
query19	208	145	148	145
query20	92	89	91	89
query21	200	127	127	127
query22	5074	4805	4921	4805
query23	34192	33471	33520	33471
query24	11789	2808	2907	2808
query25	630	360	357	357
query26	1707	154	149	149
query27	3037	326	340	326
query28	7613	2058	2057	2057
query29	1051	609	593	593
query30	274	171	178	171
query31	944	755	735	735
query32	88	52	52	52
query33	735	246	239	239
query34	1067	486	488	486
query35	828	681	663	663
query36	1086	915	924	915
query37	268	73	72	72
query38	2883	2757	2745	2745
query39	1590	1544	1563	1544
query40	246	122	125	122
query41	45	44	43	43
query42	125	91	100	91
query43	575	531	553	531
query44	1241	723	732	723
query45	275	259	250	250
query46	1081	715	716	715
query47	1982	1896	1946	1896
query48	362	290	293	290
query49	956	388	389	388
query50	767	378	374	374
query51	6935	6864	6832	6832
query52	108	87	90	87
query53	347	281	281	281
query54	855	425	432	425
query55	74	70	73	70
query56	236	223	218	218
query57	1238	1139	1158	1139
query58	217	202	230	202
query59	3299	3076	3100	3076
query60	249	221	229	221
query61	93	85	86	85
query62	648	476	467	467
query63	301	279	286	279
query64	9568	2211	1747	1747
query65	3191	3128	3142	3128
query66	1327	356	336	336
query67	15391	15059	15322	15059
query68	4421	519	533	519
query69	467	295	295	295
query70	1194	1073	1120	1073
query71	406	262	265	262
query72	7001	2565	2381	2381
query73	710	316	313	313
query74	6648	6264	6302	6264
query75	3277	2599	2597	2597
query76	2468	986	945	945
query77	368	261	266	261
query78	10711	10124	10104	10104
query79	1616	514	509	509
query80	1047	423	426	423
query81	528	243	241	241
query82	1153	98	94	94
query83	240	162	162	162
query84	240	85	83	83
query85	1326	281	323	281
query86	436	301	308	301
query87	3324	3103	3191	3103
query88	3288	2356	2345	2345
query89	478	380	381	380
query90	1934	188	184	184
query91	132	173	96	96
query92	62	49	48	48
query93	1415	502	498	498
query94	1232	180	178	178
query95	384	295	295	295
query96	578	262	264	262
query97	3255	3031	3017	3017
query98	232	225	214	214
query99	1114	905	931	905
Total cold run time: 281724 ms
Total hot run time: 180751 ms

doris-robot · 2024-05-22T05:06:02Z

ClickBench: Total hot run time: 30.54 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 68e1a4a731337dc63c5660f1e10d6631450956eb, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.06	0.07
query5	0.49	0.50	0.50
query6	1.18	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.52	0.49	0.50
query10	0.55	0.55	0.54
query11	0.15	0.12	0.11
query12	0.14	0.11	0.11
query13	0.59	0.58	0.60
query14	0.78	0.77	0.77
query15	0.83	0.82	0.80
query16	0.35	0.37	0.37
query17	1.02	0.98	0.96
query18	0.22	0.25	0.21
query19	1.89	1.78	1.70
query20	0.02	0.01	0.01
query21	15.56	0.68	0.66
query22	4.96	6.77	1.98
query23	18.27	1.40	1.23
query24	1.54	0.28	0.23
query25	0.14	0.09	0.08
query26	0.26	0.17	0.16
query27	0.07	0.08	0.07
query28	13.45	1.01	1.00
query29	13.86	3.24	3.19
query30	0.24	0.06	0.05
query31	2.87	0.38	0.37
query32	3.29	0.47	0.47
query33	2.90	2.94	2.90
query34	17.24	4.44	4.48
query35	4.52	4.56	4.67
query36	0.64	0.46	0.46
query37	0.18	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.15
query41	0.08	0.04	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.36 s
Total hot run time: 30.54 s

doris-robot · 2024-05-22T05:26:25Z

TeamCity be ut coverage result:
Function Coverage: 35.68% (9017/25269)
Line Coverage: 27.33% (74576/272884)
Region Coverage: 26.55% (38566/145267)
Branch Coverage: 23.39% (19665/84066)
Coverage Report: http://coverage.selectdb-in.cc/coverage/68e1a4a731337dc63c5660f1e10d6631450956eb_68e1a4a731337dc63c5660f1e10d6631450956eb/report/index.html

BiteTheDDDDt · 2024-05-22T07:43:38Z

run buildall

github-actions · 2024-05-22T07:49:29Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-05-22T08:20:01Z

TPC-H: Total hot run time: 42205 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d7cf8abbbaad2241c5a4688a4e61a3b526d8b12, data reload: false

------ Round 1 ----------------------------------
q1	17612	4402	4282	4282
q2	2025	183	190	183
q3	10492	1206	1260	1206
q4	10202	882	805	805
q5	7488	2756	2767	2756
q6	233	134	134	134
q7	964	616	609	609
q8	9236	2189	2132	2132
q9	9435	6798	6886	6798
q10	9275	3952	3943	3943
q11	471	236	248	236
q12	482	231	217	217
q13	17537	3233	3148	3148
q14	249	231	221	221
q15	507	482	489	482
q16	524	420	388	388
q17	1003	636	707	636
q18	8504	8014	7859	7859
q19	1685	1624	1575	1575
q20	669	309	313	309
q21	5226	4034	4004	4004
q22	350	282	286	282
Total cold run time: 114169 ms
Total hot run time: 42205 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4580	4424	4416	4416
q2	392	288	276	276
q3	3197	2946	2764	2764
q4	1902	1635	1663	1635
q5	5475	5509	5528	5509
q6	208	121	120	120
q7	2205	1786	1784	1784
q8	3281	3442	3373	3373
q9	8662	8646	8646	8646
q10	3903	3852	3832	3832
q11	589	505	496	496
q12	775	620	609	609
q13	16075	3188	3180	3180
q14	304	271	271	271
q15	525	472	488	472
q16	501	429	427	427
q17	1791	1485	1469	1469
q18	7693	7652	7621	7621
q19	1705	1601	1598	1598
q20	2004	1757	1820	1757
q21	10233	4763	4747	4747
q22	572	479	499	479
Total cold run time: 76572 ms
Total hot run time: 55481 ms

doris-robot · 2024-05-22T08:30:47Z

TPC-DS: Total hot run time: 168472 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d7cf8abbbaad2241c5a4688a4e61a3b526d8b12, data reload: false

query1	895	382	368	368
query2	6427	2397	2350	2350
query3	6657	204	203	203
query4	20809	17300	17410	17300
query5	4161	417	418	417
query6	240	149	154	149
query7	4595	291	282	282
query8	245	188	182	182
query9	8612	2405	2384	2384
query10	455	297	258	258
query11	10590	10191	10254	10191
query12	133	89	87	87
query13	1638	357	354	354
query14	10084	7530	6263	6263
query15	214	172	172	172
query16	7717	265	251	251
query17	1742	532	515	515
query18	1958	275	272	272
query19	208	171	156	156
query20	93	86	87	86
query21	199	129	128	128
query22	4415	4206	3976	3976
query23	33644	33091	32919	32919
query24	11070	2899	2779	2779
query25	620	345	353	345
query26	1542	158	154	154
query27	2986	320	322	320
query28	7469	2068	2052	2052
query29	975	611	597	597
query30	268	172	183	172
query31	986	779	727	727
query32	91	51	53	51
query33	761	304	256	256
query34	954	467	470	467
query35	723	615	582	582
query36	1101	897	919	897
query37	126	68	68	68
query38	2910	2823	2862	2823
query39	837	788	799	788
query40	248	125	122	122
query41	46	44	44	44
query42	103	98	98	98
query43	603	566	546	546
query44	1191	731	745	731
query45	178	163	160	160
query46	1077	700	717	700
query47	1849	1814	1791	1791
query48	371	300	311	300
query49	933	376	399	376
query50	798	394	369	369
query51	6733	6607	6660	6607
query52	104	90	85	85
query53	351	282	280	280
query54	838	423	414	414
query55	78	75	73	73
query56	258	235	240	235
query57	1118	1047	1057	1047
query58	227	208	211	208
query59	3319	3013	3177	3013
query60	270	254	254	254
query61	89	90	125	90
query62	610	470	453	453
query63	311	276	279	276
query64	9648	2230	1717	1717
query65	3140	3108	3123	3108
query66	1326	339	331	331
query67	15152	14857	14976	14857
query68	4577	539	539	539
query69	427	264	266	264
query70	1109	1069	1139	1069
query71	399	275	261	261
query72	7571	2789	2524	2524
query73	714	314	316	314
query74	6038	5692	5638	5638
query75	3330	2605	2637	2605
query76	2214	920	990	920
query77	387	263	260	260
query78	10216	9876	9852	9852
query79	1744	508	510	508
query80	1018	434	420	420
query81	561	249	241	241
query82	995	96	91	91
query83	256	175	168	168
query84	251	88	90	88
query85	1343	313	355	313
query86	431	302	306	302
query87	3379	3166	3135	3135
query88	2972	2316	2313	2313
query89	479	375	374	374
query90	1945	180	184	180
query91	125	94	96	94
query92	65	51	49	49
query93	1239	501	503	501
query94	1289	181	183	181
query95	410	306	303	303
query96	575	269	262	262
query97	3220	3018	3041	3018
query98	239	225	215	215
query99	1087	859	844	844
Total cold run time: 270724 ms
Total hot run time: 168472 ms

doris-robot · 2024-05-22T08:31:43Z

TeamCity be ut coverage result:
Function Coverage: 35.66% (9014/25277)
Line Coverage: 27.31% (74511/272866)
Region Coverage: 26.54% (38556/145293)
Branch Coverage: 23.39% (19669/84076)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3d7cf8abbbaad2241c5a4688a4e61a3b526d8b12_3d7cf8abbbaad2241c5a4688a4e61a3b526d8b12/report/index.html

doris-robot · 2024-05-22T08:36:19Z

ClickBench: Total hot run time: 30.16 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d7cf8abbbaad2241c5a4688a4e61a3b526d8b12, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.65	0.10	0.09
query5	0.50	0.50	0.52
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.49
query10	0.55	0.55	0.53
query11	0.16	0.11	0.12
query12	0.16	0.12	0.12
query13	0.59	0.58	0.59
query14	0.76	0.78	0.77
query15	0.83	0.82	0.81
query16	0.36	0.35	0.37
query17	1.03	1.03	1.01
query18	0.22	0.22	0.26
query19	1.88	1.73	1.73
query20	0.02	0.00	0.01
query21	15.48	0.68	0.66
query22	4.64	7.60	1.52
query23	18.27	1.39	1.21
query24	2.02	0.22	0.20
query25	0.13	0.09	0.07
query26	0.27	0.16	0.15
query27	0.08	0.08	0.08
query28	13.30	1.01	0.99
query29	13.28	3.28	3.27
query30	0.25	0.06	0.05
query31	2.88	0.39	0.38
query32	3.28	0.47	0.46
query33	2.88	3.00	2.88
query34	17.11	4.43	4.45
query35	4.50	4.54	4.58
query36	0.65	0.46	0.46
query37	0.19	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.17	0.15	0.17
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.54 s
Total hot run time: 30.16 s

yiguolei

lgtm

github-actions · 2024-05-22T12:32:27Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-22T12:32:29Z

PR approved by anyone and no changes requested.

…ilter v1 (#35186) fix brpc ctrl use after free and remove apply filter v1

…ilter v1 (apache#35186) fix brpc ctrl use after free and remove apply filter v1

part of #35186

update [Chore](runtime-filter) enlarge sync filter size rpc timeout limit (apache#37103) enlarge sync filter size rpc timeout limit rf will failed when rpc timeout, so we need enlarge limit ``` sync filter size meet error, filter: RuntimeFilter: (id = 3, type = in_or_bloomfilter, need_local_merge: false, is_broadcast: false, build_bf_cardinality: true ``` update update update format [Bug](runtime-filter) fix brpc ctrl use after free and remove apply filter v1 (apache#35186) fix brpc ctrl use after free and remove apply filter v1 reset update

fix brpc ctrl use after free and remove apply filter v1

68e1a4a

github-actions bot added the meta-change label May 22, 2024

update

3d7cf8a

yiguolei approved these changes May 22, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 22, 2024

github-actions bot added the reviewed label May 22, 2024

zclllyybb approved these changes May 23, 2024

View reviewed changes

BiteTheDDDDt merged commit ab0742a into apache:master May 23, 2024

dataroaring pushed a commit that referenced this pull request May 26, 2024

[Bug](runtime-filter) fix brpc ctrl use after free and remove apply f…

0d616be

…ilter v1 (#35186) fix brpc ctrl use after free and remove apply filter v1

BiteTheDDDDt added a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Jul 3, 2024

[Bug](runtime-filter) fix brpc ctrl use after free and remove apply f…

ed48340

…ilter v1 (apache#35186) fix brpc ctrl use after free and remove apply filter v1

BiteTheDDDDt mentioned this pull request Jul 3, 2024

[Bug](runtime-filter) fix brpc ctrl use after free #37223

Merged

BiteTheDDDDt added a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Jul 3, 2024

[Bug](runtime-filter) fix brpc ctrl use after free and remove apply f…

5583d87

…ilter v1 (apache#35186) fix brpc ctrl use after free and remove apply filter v1

morningman pushed a commit that referenced this pull request Jul 3, 2024

[Bug](runtime-filter) fix brpc ctrl use after free (#37223)

ffc57c9

part of #35186

morningman added the dev/2.1.5-merged label Jul 3, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

[Bug](runtime-filter) fix brpc ctrl use after free and remove apply filter v1 #35186

[Bug](runtime-filter) fix brpc ctrl use after free and remove apply filter v1 #35186

Uh oh!

Conversation

BiteTheDDDDt commented May 22, 2024

Proposed changes

Further comments

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

BiteTheDDDDt commented May 22, 2024

Uh oh!

github-actions bot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

BiteTheDDDDt commented May 22, 2024

Uh oh!

github-actions bot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

doris-robot commented May 22, 2024

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 22, 2024

Uh oh!

github-actions bot commented May 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants