Skip to content

[fix](memory) Add cache manager check for exist #35590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
xinyiZzz merged 3 commits into from
Jun 5, 2024
Merged

[fix](memory) Add cache manager check for exist #35590

xinyiZzz merged 3 commits into from
Jun 5, 2024

Conversation

@xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented May 29, 2024 •
edited
Loading

Proposed changes

Debug CacheManager::_caches unordered_map multithreaded visibility.

#0  0x0000557d141fede4 in std::__shared_ptr<doris::MemTracker::MemCounter, (__gnu_cxx::_Lock_policy)2>::get (this=0x28) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291
1291          { return _M_ptr; }
[Current thread is 1 (Thread 0x7fcd139bd700 (LWP 3807666))]
(gdb)
(gdb) bt
#0  0x0000557d141fede4 in std::__shared_ptr<doris::MemTracker::MemCounter, (__gnu_cxx::_Lock_policy)2>::get (this=0x28) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291
#1  0x0000557d141fed97 in std::__shared_ptr_access<doris::MemTracker::MemCounter, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get (this=0x28) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990
#2  0x0000557d141fe607 in std::__shared_ptr_access<doris::MemTracker::MemCounter, (__gnu_cxx::_Lock_policy)2, false, false>::operator-> (this=0x28) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984
#3  0x0000557d141fd857 in doris::MemTracker::consume (this=0x0, bytes=106) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/memory/mem_tracker.h:141
#4  0x0000557d146a4331 in doris::LRUCachePolicy::insert (this=0x60c00036a600, key=..., value=0x60300151a890, charge=1, tracking_bytes=4, priority=doris::CachePriority::NORMAL) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/memory/lru_cache_policy.h:100
#5  0x0000557d1466387d in doris::CreateTabletIdxCache::set_index (this=0x60c00036a600, key=..., next_idx=3) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:1500
#6  0x0000557d14662c8d in doris::StorageEngine::_get_and_set_next_disk_index (this=0x61d000102280, partition_id=10112, storage_medium=doris::TStorageMedium::HDD) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:460
#7  0x0000557d14664da8 in doris::StorageEngine::get_stores_for_create_tablet (this=0x61d000102280, partition_id=10112, storage_medium=doris::TStorageMedium::HDD) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:529
#8  0x0000557d1467246f in doris::StorageEngine::create_tablet (this=0x61d000102280, request=..., profile=0x7fcd119a6020) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:1151
#9  0x0000557d16edfbbc in doris::create_tablet_callback (engine=..., req=...) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/task_worker_pool.cpp:1361
#10 0x0000557d181b72f4 in doris::AgentServer::start_workers(doris::ExecEnv*)::$_10::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const (this=0x606000dd4638, task=...)
    at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/agent_server.cpp:123
#11 0x0000557d181b728f in std::__invoke_impl<void, doris::AgentServer::start_workers(doris::ExecEnv*)::$_10&, doris::TAgentTaskRequest const&>(std::__invoke_other, doris::AgentServer::start_workers(doris::ExecEnv*)::$_10&, doris::TAgentTaskRequest const&) (__f=...,
    __args=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#12 0x0000557d181b7201 in std::__invoke_r<void, doris::AgentServer::start_workers(doris::ExecEnv*)::$_10&, doris::TAgentTaskRequest const&>(doris::AgentServer::start_workers(doris::ExecEnv*)::$_10&, doris::TAgentTaskRequest const&) (__fn=..., __args=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#13 0x0000557d181b7037 in std::_Function_handler<void (doris::TAgentTaskRequest const&), doris::AgentServer::start_workers(doris::ExecEnv*)::$_10>::_M_invoke(std::_Any_data const&, doris::TAgentTaskRequest const&) (__functor=..., __args=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#14 0x0000557d16f02cff in std::function<void (doris::TAgentTaskRequest const&)>::operator()(doris::TAgentTaskRequest const&) const (this=0x606000dd4638, __args=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#15 0x0000557d16ef17b4 in doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}::operator()() const (this=0x62000001a080)
    at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/task_worker_pool.cpp:437
#16 0x0000557d16ef1757 in std::__invoke_impl<void, doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}&>(std::__invoke_other, doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}&) (__f=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#17 0x0000557d16ef16c9 in std::__invoke_r<void, doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}&>(doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}&) (__fn=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#18 0x0000557d16ef13ef in std::_Function_handler<void (), doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (
    __functor=...) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#19 0x0000557d1442dfb7 in std::function<void ()>::operator()() const (this=0x606000ebb6f8) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#20 0x0000557d187c61db in doris::FunctionRunnable::run (this=0x606000ebb6f0) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:48
#21 0x0000557d187af5ff in doris::ThreadPool::dispatch_thread (this=0x6150006cc300) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:543
#22 0x0000557d187db496 in std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&> (__f=@0x6030015178c0: (void (doris::ThreadPool::*)(doris::ThreadPool * const)) 0x557d187ad620 <doris::ThreadPool::dispatch_thread()>,
    __t=@0x6030015178d0: 0x6150006cc300) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
#23 0x0000557d187db33f in std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&> (__fn=@0x6030015178c0: (void (doris::ThreadPool::*)(doris::ThreadPool * const)) 0x557d187ad620 <doris::ThreadPool::dispatch_thread()>, __args=@0x6030015178d0: 0x6150006cc300)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
#24 0x0000557d187db2b7 in std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (this=0x6030015178c0, __args=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420
#25 0x0000557d187db150 in std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() (this=0x6030015178c0) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503
#26 0x0000557d187db057 in std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) (__f=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#27 0x0000557d187dafc9 in std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) (__fn=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#28 0x0000557d187dab8f in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) (__functor=...)
    at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#29 0x0000557d1442dfb7 in std::function<void ()>::operator()() const (this=0x6110014c6ed8) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#30 0x0000557d18773ea4 in doris::Thread::supervise_thread (arg=0x6110014c6ec0) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thread.cpp:498
#31 0x00007fd3a3c32609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x00007fd3a3edf133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) f 5
#5  0x0000557d1466387d in doris::CreateTabletIdxCache::set_index (this=0x60c00036a600, key=..., next_idx=3) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:1500
1500    /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp: No such file or directory.
(gdb) p value
$1 = (doris::CreateTabletIdxCache::CacheValue *) 0x60300151a890
(gdb) p (LRUCacheValueBase*)value
$2 = (doris::LRUCacheValueBase *) 0x60300151a890
(gdb) p *(LRUCacheValueBase*)value
$3 = {_vptr$LRUCacheValueBase = 0x557d4a8eb440 <vtable for doris::CreateTabletIdxCache::CacheValue+16>, _tracking_bytes = 0, _mem_tracker = 0x0}
#5  0x0000557d1466387d in doris::CreateTabletIdxCache::set_index (this=0x60c00036a600, key=..., next_idx=3) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp:1500
1500	in /home/zcp/repo_center/doris_branch-2.1/doris/be/src/olap/storage_engine.cpp
(gdb) p *this
$4 = {<doris::LRUCachePolicy> = {<doris::CachePolicy> = {_vptr$CachePolicy = 0x557d4a8eb310 <vtable for doris::CreateTabletIdxCache+16>, _type = doris::CachePolicy::CacheType::CREATE_TABLET_RR_IDX_CACHE, _mem_tracker = {
        _M_t = {<std::__uniq_ptr_impl<doris::MemTracker, std::default_delete<doris::MemTracker> >> = {
            _M_t = {<std::_Tuple_impl<0ul, doris::MemTracker*, std::default_delete<doris::MemTracker> >> = {<std::_Tuple_impl<1ul, std::default_delete<doris::MemTracker> >> = {<std::_Head_base<1ul, std::default_delete<doris::MemTracker>, true>> = {
                    _M_head_impl = {<No data fields>}}, <No data fields>}, <std::_Head_base<0ul, doris::MemTracker*, false>> = {_M_head_impl = 0x60c0003697c0}, <No data fields>}, <No data fields>}}, <No data fields>}},
      _mem_tracker_by_allocator = {<std::__shared_ptr<doris::MemTrackerLimiter, (__gnu_cxx::_Lock_policy)2>> = {<std::__shared_ptr_access<doris::MemTrackerLimiter, (__gnu_cxx::_Lock_policy)2, false, false>> = {<No data fields>}, _M_ptr = 0x0, _M_refcount = {
            _M_pi = 0x0}}, <No data fields>}, _profile = {_M_t = {<std::__uniq_ptr_impl<doris::RuntimeProfile, std::default_delete<doris::RuntimeProfile> >> = {
            _M_t = {<std::_Tuple_impl<0ul, doris::RuntimeProfile*, std::default_delete<doris::RuntimeProfile> >> = {<std::_Tuple_impl<1ul, std::default_delete<doris::RuntimeProfile> >> = {<std::_Head_base<1ul, std::default_delete<doris::RuntimeProfile>, true>> = {
                    _M_head_impl = {<No data fields>}}, <No data fields>}, <std::_Head_base<0ul, doris::RuntimeProfile*, false>> = {_M_head_impl = 0x617000127900}, <No data fields>}, <No data fields>}}, <No data fields>}}, _prune_stale_number_counter = 0x6030008fdc70,
      _prune_all_number_counter = 0x6030008fdd00, _freed_memory_counter = 0x6030008fdd60, _freed_entrys_counter = 0x6030008fdd90, _cost_timer = 0x6030008fddc0, _stale_sweep_time_s = 1800, _enable_prune = true},
    _cache = {<std::__shared_ptr<doris::Cache, (__gnu_cxx::_Lock_policy)2>> = {<std::__shared_ptr_access<doris::Cache, (__gnu_cxx::_Lock_policy)2, false, false>> = {<No data fields>}, _M_ptr = 0x60f000078220, _M_refcount = {_M_pi = 0x6030008fe150}}, <No data fields>},
    _lru_cache_type = doris::NUMBER}, <No data fields>}

@doris-robot
Copy link

doris-robot commented May 29, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented May 29, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented May 29, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-H: Total hot run time: 41505 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f75e4e3dc98302e7ba835bd8de8ba6f637c67b53, data reload: false

------ Round 1 ----------------------------------
q1	17627	4420	4311	4311
q2	2038	201	190	190
q3	10433	1312	1228	1228
q4	10186	840	762	762
q5	7480	2784	2658	2658
q6	225	134	136	134
q7	966	615	625	615
q8	9224	2162	2121	2121
q9	9493	6756	6756	6756
q10	9174	3910	3923	3910
q11	446	239	241	239
q12	534	233	234	233
q13	17292	3255	3247	3247
q14	289	227	225	225
q15	531	486	479	479
q16	520	400	399	399
q17	1014	774	792	774
q18	8378	7982	7707	7707
q19	6892	1585	1548	1548
q20	640	338	337	337
q21	5136	3349	4085	3349
q22	359	283	283	283
Total cold run time: 118877 ms
Total hot run time: 41505 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4532	4495	4499	4495
q2	377	270	280	270
q3	3205	2952	2868	2868
q4	1985	1662	1660	1660
q5	5318	5537	5523	5523
q6	220	127	125	125
q7	2246	1835	1835	1835
q8	3231	3448	3415	3415
q9	8657	8717	8726	8717
q10	4100	3755	3715	3715
q11	602	513	492	492
q12	786	645	639	639
q13	15837	3151	3148	3148
q14	304	272	282	272
q15	529	462	480	462
q16	494	433	428	428
q17	1814	1546	1498	1498
q18	8003	7587	7425	7425
q19	1690	1588	1582	1582
q20	1994	1833	1776	1776
q21	9217	4742	4796	4742
q22	563	474	493	474
Total cold run time: 75704 ms
Total hot run time: 55561 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-DS: Total hot run time: 169884 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f75e4e3dc98302e7ba835bd8de8ba6f637c67b53, data reload: false

query1	931	403	370	370
query2	6451	2388	2428	2388
query3	6652	210	214	210
query4	19906	17301	17371	17301
query5	4162	442	426	426
query6	258	163	154	154
query7	4589	298	293	293
query8	251	192	181	181
query9	8414	2378	2366	2366
query10	460	285	273	273
query11	10453	10082	10066	10066
query12	135	92	91	91
query13	1644	397	364	364
query14	10103	6868	7108	6868
query15	208	170	170	170
query16	7717	263	264	263
query17	1726	517	517	517
query18	1946	268	274	268
query19	202	150	151	150
query20	92	82	85	82
query21	195	128	133	128
query22	4221	4051	3976	3976
query23	33694	33276	33094	33094
query24	9681	2833	2792	2792
query25	559	359	362	359
query26	703	161	156	156
query27	2186	321	322	321
query28	5753	2071	2055	2055
query29	860	615	611	611
query30	219	147	151	147
query31	953	742	750	742
query32	89	58	53	53
query33	653	271	276	271
query34	848	491	478	478
query35	709	627	603	603
query36	1054	887	903	887
query37	104	65	68	65
query38	2891	2783	2728	2728
query39	850	795	791	791
query40	200	125	127	125
query41	48	46	49	46
query42	104	100	97	97
query43	577	546	551	546
query44	1076	775	756	756
query45	171	168	161	161
query46	1090	731	717	717
query47	1848	1743	1761	1743
query48	367	306	304	304
query49	831	380	391	380
query50	764	392	387	387
query51	6777	6724	6703	6703
query52	102	89	100	89
query53	355	286	287	286
query54	832	448	450	448
query55	76	73	73	73
query56	269	251	246	246
query57	1099	1031	1020	1020
query58	235	209	217	209
query59	3394	3120	3134	3120
query60	265	270	259	259
query61	90	91	83	83
query62	585	456	477	456
query63	318	282	288	282
query64	8446	2273	1803	1803
query65	3158	3091	3107	3091
query66	773	331	315	315
query67	15312	15110	14798	14798
query68	4500	552	565	552
query69	444	269	273	269
query70	1149	1150	1152	1150
query71	372	276	271	271
query72	7252	5635	2742	2742
query73	720	324	324	324
query74	6026	5643	5657	5643
query75	3324	2663	2639	2639
query76	2192	992	970	970
query77	380	284	267	267
query78	10267	9763	10047	9763
query79	1634	509	521	509
query80	1098	438	436	436
query81	555	225	220	220
query82	779	88	92	88
query83	247	168	172	168
query84	246	87	91	87
query85	1235	266	257	257
query86	449	300	320	300
query87	3285	3109	3192	3109
query88	3293	2413	2415	2413
query89	472	394	377	377
query90	2050	193	191	191
query91	122	103	152	103
query92	64	48	50	48
query93	1704	528	515	515
query94	1299	190	190	190
query95	405	318	312	312
query96	583	271	271	271
query97	3198	3032	3035	3032
query98	238	226	216	216
query99	1179	851	880	851
Total cold run time: 262445 ms
Total hot run time: 169884 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

ClickBench: Total hot run time: 30.59 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f75e4e3dc98302e7ba835bd8de8ba6f637c67b53, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.50	0.49
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.54	0.49	0.50
query10	0.53	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.11
query13	0.60	0.59	0.60
query14	0.77	0.79	0.77
query15	0.84	0.81	0.81
query16	0.38	0.35	0.36
query17	0.95	0.98	1.00
query18	0.21	0.25	0.25
query19	1.74	1.69	1.70
query20	0.01	0.02	0.01
query21	15.73	0.66	0.66
query22	4.05	7.33	2.13
query23	18.23	1.32	1.24
query24	1.83	0.21	0.22
query25	0.14	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.07	0.08
query28	13.34	1.03	1.00
query29	13.75	3.31	3.27
query30	0.24	0.08	0.05
query31	2.86	0.38	0.39
query32	3.30	0.48	0.46
query33	2.89	2.85	2.87
query34	17.16	4.36	4.38
query35	4.54	4.48	4.64
query36	0.64	0.46	0.47
query37	0.17	0.16	0.14
query38	0.15	0.14	0.14
query39	0.05	0.03	0.03
query40	0.16	0.13	0.14
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.03
Total cold run time: 110.36 s
Total hot run time: 30.59 s

@doris-robot
Copy link

doris-robot commented May 29, 2024

TeamCity be ut coverage result:
Function Coverage: 35.89% (9123/25417)
Line Coverage: 27.37% (74960/273909)
Region Coverage: 26.61% (38817/145875)
Branch Coverage: 23.45% (19753/84240)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f75e4e3dc98302e7ba835bd8de8ba6f637c67b53_f75e4e3dc98302e7ba835bd8de8ba6f637c67b53/report/index.html

@xinyiZzz xinyiZzz force-pushed the 20240527_add_fix_cache_manager branch from f75e4e3 to 5ef418d Compare May 29, 2024 12:53
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented May 29, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented May 29, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-H: Total hot run time: 40436 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5ef418d1997cec9fa708a803667af445fc13bd54, data reload: false

------ Round 1 ----------------------------------
q1	17666	4423	4301	4301
q2	2044	199	197	197
q3	10511	1179	1234	1179
q4	10170	763	735	735
q5	7481	2702	2768	2702
q6	225	135	134	134
q7	955	585	585	585
q8	9210	2083	2110	2083
q9	9038	6530	6520	6520
q10	8942	3788	3710	3710
q11	476	248	242	242
q12	488	224	227	224
q13	17772	3005	2975	2975
q14	277	214	211	211
q15	520	483	472	472
q16	519	389	382	382
q17	982	753	747	747
q18	8279	7666	7579	7579
q19	7669	1651	1581	1581
q20	663	318	300	300
q21	5018	3252	3862	3252
q22	390	325	329	325
Total cold run time: 119295 ms
Total hot run time: 40436 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4418	4271	4241	4241
q2	365	266	268	266
q3	3032	2772	2746	2746
q4	1884	1607	1587	1587
q5	5323	5319	5311	5311
q6	218	125	126	125
q7	2121	1698	1758	1698
q8	3224	3395	3381	3381
q9	8400	8394	8379	8379
q10	3891	3680	3692	3680
q11	585	487	491	487
q12	748	603	592	592
q13	17260	3016	3034	3016
q14	284	267	260	260
q15	533	476	475	475
q16	470	417	419	417
q17	1815	1513	1491	1491
q18	7687	7640	7283	7283
q19	1676	1626	1592	1592
q20	1993	1769	1815	1769
q21	4809	4888	4723	4723
q22	610	578	535	535
Total cold run time: 71346 ms
Total hot run time: 54054 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

TPC-DS: Total hot run time: 168062 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5ef418d1997cec9fa708a803667af445fc13bd54, data reload: false

query1	932	386	370	370
query2	6815	2286	2295	2286
query3	6651	215	219	215
query4	20272	17459	17308	17308
query5	4135	417	441	417
query6	254	153	155	153
query7	4576	302	289	289
query8	324	294	273	273
query9	8441	2420	2398	2398
query10	470	282	264	264
query11	10375	10060	10222	10060
query12	139	92	92	92
query13	1682	365	353	353
query14	9347	6810	5877	5877
query15	226	192	187	187
query16	7831	263	267	263
query17	1364	508	503	503
query18	1956	274	263	263
query19	207	153	148	148
query20	91	86	90	86
query21	199	126	130	126
query22	4023	3988	3884	3884
query23	33626	33090	32986	32986
query24	10860	2880	2876	2876
query25	577	350	364	350
query26	709	154	159	154
query27	2258	319	322	319
query28	5850	2058	2076	2058
query29	888	603	609	603
query30	291	149	151	149
query31	965	738	756	738
query32	89	58	55	55
query33	656	261	267	261
query34	872	472	485	472
query35	737	605	586	586
query36	1069	930	908	908
query37	110	66	68	66
query38	2876	2781	2753	2753
query39	881	826	797	797
query40	198	123	122	122
query41	53	52	50	50
query42	104	101	98	98
query43	561	529	550	529
query44	1065	728	740	728
query45	193	177	166	166
query46	1072	688	681	681
query47	1847	1771	1791	1771
query48	360	307	297	297
query49	1033	385	385	385
query50	772	379	389	379
query51	6792	6596	6665	6596
query52	110	87	99	87
query53	356	294	287	287
query54	913	437	435	435
query55	76	74	74	74
query56	277	235	266	235
query57	1101	1053	1101	1053
query58	236	202	219	202
query59	3287	3061	3201	3061
query60	271	252	261	252
query61	93	87	85	85
query62	634	453	458	453
query63	316	289	286	286
query64	8612	2256	1804	1804
query65	3206	3116	3116	3116
query66	813	334	333	333
query67	15143	14858	14804	14804
query68	4799	536	544	536
query69	511	330	267	267
query70	1165	1019	1117	1019
query71	413	269	275	269
query72	8030	2772	2508	2508
query73	730	323	321	321
query74	6012	5613	5646	5613
query75	3419	2626	2638	2626
query76	3113	1019	1042	1019
query77	628	265	264	264
query78	10240	9746	9770	9746
query79	2351	512	515	512
query80	1274	486	428	428
query81	536	216	222	216
query82	1095	91	91	91
query83	271	171	177	171
query84	235	92	83	83
query85	1421	268	275	268
query86	473	279	332	279
query87	3269	3123	3131	3123
query88	4017	2361	2341	2341
query89	475	378	377	377
query90	1908	184	185	184
query91	123	94	95	94
query92	56	48	48	48
query93	2265	513	497	497
query94	1192	185	183	183
query95	391	306	316	306
query96	599	311	266	266
query97	3211	3017	3078	3017
query98	244	227	220	220
query99	1149	855	869	855
Total cold run time: 268733 ms
Total hot run time: 168062 ms

@doris-robot
Copy link

doris-robot commented May 29, 2024

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5ef418d1997cec9fa708a803667af445fc13bd54, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.24	0.05	0.05
query4	1.66	0.09	0.09
query5	0.51	0.48	0.50
query6	1.13	0.71	0.72
query7	0.03	0.02	0.01
query8	0.05	0.05	0.04
query9	0.53	0.48	0.50
query10	0.54	0.56	0.55
query11	0.17	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.60	0.60
query14	0.76	0.78	0.78
query15	0.83	0.82	0.83
query16	0.38	0.36	0.36
query17	0.98	0.95	0.93
query18	0.20	0.25	0.21
query19	1.81	1.70	1.72
query20	0.02	0.01	0.02
query21	15.52	0.69	0.68
query22	5.40	6.40	1.73
query23	18.32	1.37	1.25
query24	1.74	0.27	0.24
query25	0.14	0.08	0.08
query26	0.28	0.16	0.17
query27	0.08	0.08	0.08
query28	13.31	1.01	0.99
query29	12.97	3.32	3.23
query30	0.25	0.07	0.05
query31	2.84	0.38	0.40
query32	3.28	0.46	0.47
query33	2.89	2.93	2.95
query34	17.21	4.44	4.45
query35	4.49	4.56	4.59
query36	0.69	0.49	0.45
query37	0.18	0.16	0.16
query38	0.15	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.15	0.14
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.03	0.03
Total cold run time: 110.86 s
Total hot run time: 30.42 s

@doris-robot
Copy link

doris-robot commented May 29, 2024

TeamCity be ut coverage result:
Function Coverage: 35.98% (9146/25419)
Line Coverage: 27.43% (75122/273911)
Region Coverage: 26.63% (38876/145961)
Branch Coverage: 23.48% (19792/84310)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5ef418d1997cec9fa708a803667af445fc13bd54_5ef418d1997cec9fa708a803667af445fc13bd54/report/index.html

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jun 3, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TeamCity be ut coverage result:
Function Coverage: 36.26% (9233/25460)
Line Coverage: 27.60% (75691/274207)
Region Coverage: 26.82% (39199/146154)
Branch Coverage: 23.54% (19858/84348)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1b0c8bbb32c76e05d89d26afc43e649ecf6b37bd_1b0c8bbb32c76e05d89d26afc43e649ecf6b37bd/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 41395 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b0c8bbb32c76e05d89d26afc43e649ecf6b37bd, data reload: false

------ Round 1 ----------------------------------
q1	17617	4488	4337	4337
q2	2024	191	196	191
q3	10466	1217	1221	1217
q4	10192	927	802	802
q5	7491	2776	2658	2658
q6	230	136	135	135
q7	974	624	628	624
q8	9218	2207	2153	2153
q9	9267	6723	6701	6701
q10	9464	3893	3985	3893
q11	470	244	259	244
q12	457	223	235	223
q13	17258	3284	3204	3204
q14	264	225	225	225
q15	537	488	475	475
q16	518	409	398	398
q17	1009	701	750	701
q18	8449	8115	7868	7868
q19	7307	1357	1567	1357
q20	672	324	326	324
q21	5142	4078	3324	3324
q22	399	348	341	341
Total cold run time: 119425 ms
Total hot run time: 41395 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4599	4477	4514	4477
q2	376	260	273	260
q3	3164	2943	2896	2896
q4	2048	1713	1647	1647
q5	5367	5524	5536	5524
q6	224	127	127	127
q7	2206	1789	1821	1789
q8	3242	3426	3411	3411
q9	8728	8604	8690	8604
q10	4130	3897	3884	3884
q11	586	504	485	485
q12	771	606	610	606
q13	16819	3106	3186	3106
q14	304	269	280	269
q15	526	486	480	480
q16	484	455	435	435
q17	1857	1534	1500	1500
q18	8092	7672	7319	7319
q19	1801	1650	1675	1650
q20	2723	1815	1800	1800
q21	6873	4731	4745	4731
q22	788	538	550	538
Total cold run time: 75708 ms
Total hot run time: 55538 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 172755 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b0c8bbb32c76e05d89d26afc43e649ecf6b37bd, data reload: false

query1	908	373	372	372
query2	6469	2490	2374	2374
query3	6651	202	203	202
query4	19377	17325	17417	17325
query5	4116	456	453	453
query6	253	157	150	150
query7	4584	304	306	304
query8	312	293	277	277
query9	8574	2377	2361	2361
query10	449	318	266	266
query11	10399	10018	9879	9879
query12	137	87	84	84
query13	1631	371	384	371
query14	9453	7391	7539	7391
query15	237	187	184	184
query16	7825	271	269	269
query17	1705	539	512	512
query18	1960	272	264	264
query19	205	148	150	148
query20	92	81	81	81
query21	217	135	130	130
query22	4241	4276	4008	4008
query23	33879	33104	32965	32965
query24	11060	2791	2799	2791
query25	641	377	384	377
query26	1450	161	161	161
query27	2938	367	321	321
query28	7520	2048	2050	2048
query29	960	601	594	594
query30	279	151	151	151
query31	953	748	732	732
query32	93	51	54	51
query33	763	279	276	276
query34	959	475	468	468
query35	736	611	593	593
query36	1100	910	881	881
query37	148	68	70	68
query38	2938	2800	2749	2749
query39	864	801	783	783
query40	216	126	122	122
query41	52	51	52	51
query42	119	98	96	96
query43	575	565	541	541
query44	1260	745	761	745
query45	204	164	168	164
query46	1084	746	682	682
query47	1849	1787	1800	1787
query48	387	306	302	302
query49	999	439	421	421
query50	782	391	390	390
query51	6840	6632	6722	6632
query52	101	90	89	89
query53	383	288	284	284
query54	851	472	456	456
query55	74	71	71	71
query56	294	257	260	257
query57	1128	1112	1075	1075
query58	272	280	260	260
query59	3470	3370	3160	3160
query60	294	274	259	259
query61	93	93	93	93
query62	639	482	451	451
query63	317	293	294	293
query64	8957	2273	1771	1771
query65	3199	3102	3123	3102
query66	1285	331	326	326
query67	15276	14986	14936	14936
query68	5438	553	541	541
query69	638	434	412	412
query70	1148	1133	1054	1054
query71	451	285	281	281
query72	7868	5208	5554	5208
query73	789	330	320	320
query74	5849	5592	5499	5499
query75	3772	2636	2628	2628
query76	3165	1046	943	943
query77	656	343	296	296
query78	10261	9799	9656	9656
query79	1715	503	518	503
query80	1287	474	456	456
query81	577	227	231	227
query82	720	105	105	105
query83	193	168	171	168
query84	267	93	83	83
query85	1311	270	266	266
query86	406	299	326	299
query87	3298	3082	3074	3074
query88	3834	2450	2436	2436
query89	469	389	380	380
query90	1878	186	187	186
query91	122	98	98	98
query92	56	48	49	48
query93	2783	524	514	514
query94	1179	197	195	195
query95	422	322	326	322
query96	610	336	277	277
query97	3233	3017	3047	3017
query98	238	209	215	209
query99	1367	839	824	824
Total cold run time: 274076 ms
Total hot run time: 172755 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1b0c8bbb32c76e05d89d26afc43e649ecf6b37bd, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.08
query5	0.52	0.49	0.54
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.53	0.49	0.49
query10	0.54	0.54	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.59
query14	0.76	0.80	0.78
query15	0.82	0.82	0.81
query16	0.36	0.36	0.36
query17	1.00	0.94	0.99
query18	0.24	0.24	0.24
query19	1.81	1.70	1.70
query20	0.02	0.01	0.01
query21	15.51	0.69	0.67
query22	4.62	7.65	1.54
query23	18.34	1.36	1.32
query24	2.06	0.19	0.22
query25	0.15	0.08	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.26	1.01	1.00
query29	12.88	3.39	3.35
query30	0.24	0.06	0.05
query31	2.87	0.39	0.38
query32	3.28	0.48	0.47
query33	2.89	2.89	2.88
query34	17.06	4.41	4.40
query35	4.52	4.46	4.46
query36	0.65	0.50	0.49
query37	0.18	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.15
query41	0.10	0.05	0.04
query42	0.06	0.04	0.05
query43	0.04	0.03	0.03
Total cold run time: 110.14 s
Total hot run time: 30.24 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jun 3, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Jun 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-H: Total hot run time: 40627 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e0fcc3b7336c38b2dbd4d8afc4e489cfffee76ee, data reload: false

------ Round 1 ----------------------------------
q1	17671	4532	4328	4328
q2	2035	197	199	197
q3	10590	1201	1230	1201
q4	11197	832	846	832
q5	7673	2738	2682	2682
q6	225	143	141	141
q7	972	615	627	615
q8	9568	2145	2131	2131
q9	9237	6654	6651	6651
q10	9290	3790	3740	3740
q11	451	237	249	237
q12	452	231	232	231
q13	17763	2986	2989	2986
q14	254	221	226	221
q15	521	471	478	471
q16	512	386	380	380
q17	989	709	733	709
q18	8155	7501	7507	7501
q19	2062	1457	1589	1457
q20	654	310	320	310
q21	5010	3266	3885	3266
q22	398	340	350	340
Total cold run time: 115679 ms
Total hot run time: 40627 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4302	4272	4272
q2	373	276	270	270
q3	2962	2729	2777	2729
q4	1884	1641	1658	1641
q5	5267	5276	5391	5276
q6	214	128	129	128
q7	2127	1774	1731	1731
q8	3192	3338	3340	3338
q9	8435	8383	8353	8353
q10	3915	3673	3698	3673
q11	567	490	504	490
q12	780	608	615	608
q13	16256	2985	2989	2985
q14	290	274	270	270
q15	524	473	469	469
q16	473	408	438	408
q17	1791	1479	1456	1456
q18	7576	7554	7469	7469
q19	2480	1693	1497	1497
q20	2008	1796	1770	1770
q21	4900	4778	4714	4714
q22	627	551	534	534
Total cold run time: 71038 ms
Total hot run time: 54081 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TeamCity be ut coverage result:
Function Coverage: 36.29% (9240/25460)
Line Coverage: 27.66% (75854/274274)
Region Coverage: 26.87% (39281/146166)
Branch Coverage: 23.60% (19912/84356)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e0fcc3b7336c38b2dbd4d8afc4e489cfffee76ee_e0fcc3b7336c38b2dbd4d8afc4e489cfffee76ee/report/index.html

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

TPC-DS: Total hot run time: 169750 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e0fcc3b7336c38b2dbd4d8afc4e489cfffee76ee, data reload: false

query1	925	383	381	381
query2	6452	2426	2359	2359
query3	6645	208	207	207
query4	19374	17539	17302	17302
query5	4175	469	451	451
query6	246	158	152	152
query7	4591	300	294	294
query8	319	291	300	291
query9	8708	2418	2392	2392
query10	472	287	293	287
query11	10473	10030	10061	10030
query12	132	94	90	90
query13	1625	361	368	361
query14	10158	6731	7706	6731
query15	236	184	184	184
query16	7866	271	271	271
query17	1836	531	516	516
query18	1966	268	286	268
query19	202	152	151	151
query20	95	88	93	88
query21	209	135	133	133
query22	4483	4059	4092	4059
query23	33635	32988	33119	32988
query24	12034	2886	2776	2776
query25	677	350	358	350
query26	1807	153	155	153
query27	2972	331	330	330
query28	7270	2103	2144	2103
query29	1174	605	609	605
query30	292	149	152	149
query31	991	736	723	723
query32	93	51	54	51
query33	766	282	293	282
query34	973	473	479	473
query35	746	608	611	608
query36	1112	957	941	941
query37	292	68	70	68
query38	2854	2722	2732	2722
query39	882	779	781	779
query40	286	122	130	122
query41	54	51	50	50
query42	116	99	96	96
query43	588	529	557	529
query44	1276	721	753	721
query45	200	163	174	163
query46	1085	709	713	709
query47	1873	1766	1775	1766
query48	381	338	310	310
query49	1188	403	403	403
query50	784	389	400	389
query51	6994	6768	6760	6760
query52	105	92	96	92
query53	357	284	288	284
query54	957	447	444	444
query55	74	74	75	74
query56	281	256	259	256
query57	1129	1055	1062	1055
query58	265	253	225	225
query59	3359	3294	3233	3233
query60	308	273	271	271
query61	91	122	92	92
query62	641	432	438	432
query63	324	285	290	285
query64	9863	2254	1709	1709
query65	3172	3112	3110	3110
query66	1381	325	318	318
query67	15414	14945	14946	14945
query68	4623	572	547	547
query69	471	308	311	308
query70	1105	1148	1016	1016
query71	485	286	274	274
query72	6878	5587	2767	2767
query73	761	334	323	323
query74	5946	5515	5461	5461
query75	3472	2668	2618	2618
query76	2864	926	980	926
query77	460	296	284	284
query78	10366	9819	9699	9699
query79	2287	519	518	518
query80	1337	471	458	458
query81	525	233	221	221
query82	718	103	100	100
query83	202	174	175	174
query84	271	88	86	86
query85	1431	277	284	277
query86	421	290	310	290
query87	3338	3069	3094	3069
query88	3979	2427	2337	2337
query89	471	386	374	374
query90	1891	188	188	188
query91	126	98	98	98
query92	61	49	50	49
query93	3059	521	507	507
query94	1189	196	192	192
query95	407	315	309	309
query96	604	270	273	270
query97	3214	3055	3023	3023
query98	238	216	216	216
query99	1240	853	829	829
Total cold run time: 276744 ms
Total hot run time: 169750 ms

@doris-robot
Copy link

doris-robot commented Jun 3, 2024

ClickBench: Total hot run time: 30.58 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e0fcc3b7336c38b2dbd4d8afc4e489cfffee76ee, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.68	0.07	0.07
query5	0.51	0.48	0.49
query6	1.13	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.54	0.54
query11	0.16	0.11	0.10
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.80	0.78	0.78
query15	0.83	0.82	0.83
query16	0.36	0.38	0.37
query17	0.97	0.99	1.00
query18	0.20	0.25	0.23
query19	1.76	1.67	1.67
query20	0.02	0.01	0.01
query21	15.46	0.68	0.66
query22	4.69	7.20	1.85
query23	18.32	1.43	1.32
query24	1.66	0.37	0.20
query25	0.14	0.09	0.08
query26	0.26	0.17	0.18
query27	0.08	0.09	0.08
query28	13.23	1.01	0.99
query29	13.19	3.40	3.38
query30	0.25	0.06	0.05
query31	2.87	0.38	0.38
query32	3.26	0.48	0.48
query33	2.92	2.88	2.85
query34	17.12	4.43	4.42
query35	4.50	4.44	4.49
query36	0.66	0.49	0.46
query37	0.18	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.19	0.14	0.14
query41	0.09	0.05	0.06
query42	0.06	0.06	0.06
query43	0.04	0.04	0.04
Total cold run time: 110.03 s
Total hot run time: 30.58 s

@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 5, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2024

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Jun 5, 2024
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xinyiZzz xinyiZzz merged commit b7b77ad into apache:master Jun 5, 2024
dataroaring pushed a commit that referenced this pull request Jun 7, 2024
@xinyiZzz @dataroaring
[fix](memory) Add cache manager check for exist (#35590)
96a7a0a 
Debug CacheManager::_caches unordered_map multithreaded visibility.
@xinyiZzz xinyiZzz mentioned this pull request Jun 13, 2024
Merged
xinyiZzz added a commit that referenced this pull request Jun 24, 2024
@xinyiZzz
[fix](memory) Refactor LRU cache policy memory tracking (#36235)
1a3a5b6 
Fix #35590, CacheManager get CachePolicy should be used lock.

Add LRUCachePolicyTrackingAllocator and LRUCachePolicyTrackingManual.
tracking memory in LRUCachePolicy::insert, LRUCacheValueBase not need init mem_tracker, so not need to get CachePolicy under lock.
dataroaring pushed a commit that referenced this pull request Jun 26, 2024
@xinyiZzz @dataroaring
[fix](memory) Refactor LRU cache policy memory tracking (#36235)
73eec55 
Fix #35590, CacheManager get CachePolicy should be used lock.

Add LRUCachePolicyTrackingAllocator and LRUCachePolicyTrackingManual.
tracking memory in LRUCachePolicy::insert, LRUCacheValueBase not need init mem_tracker, so not need to get CachePolicy under lock.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 11, 2024
@xinyiZzz
[fix](memory) Add cache manager check for exist (apache#35590)
1a2036b 
Debug CacheManager::_caches unordered_map multithreaded visibility.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 11, 2024
@xinyiZzz
[fix](memory) Refactor LRU cache policy memory tracking (apache#36235)
cbe6f0c 
Fix apache#35590, CacheManager get CachePolicy should be used lock.

Add LRUCachePolicyTrackingAllocator and LRUCachePolicyTrackingManual.
tracking memory in LRUCachePolicy::insert, LRUCacheValueBase not need init mem_tracker, so not need to get CachePolicy under lock.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 11, 2024
@xinyiZzz
[fix](memory) Add cache manager check for exist (apache#35590)
9dff269 
Debug CacheManager::_caches unordered_map multithreaded visibility.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 11, 2024
@xinyiZzz
[fix](memory) Refactor LRU cache policy memory tracking (apache#36235)
220f667 
Fix apache#35590, CacheManager get CachePolicy should be used lock.

Add LRUCachePolicyTrackingAllocator and LRUCachePolicyTrackingManual.
tracking memory in LRUCachePolicy::insert, LRUCacheValueBase not need init mem_tracker, so not need to get CachePolicy under lock.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@wangbo wangbo wangbo approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xinyiZzz @doris-robot @yiguolei @wangbo @dataroaring