branch-3.0: [Opt](mow) Move `DeleteBitmapAggCache` to `ExecEnv` (#52649) #52737

bobhan1 · 2025-07-03T11:48:46Z

pick #52649

hello-stephen · 2025-07-03T11:48:52Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-07-03T11:49:21Z

run buildall

doris-robot · 2025-07-03T12:59:44Z

TPC-H: Total hot run time: 39995 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7dbd8926631a7598a5d97f7a67c41febbf210711, data reload: false

------ Round 1 ----------------------------------
q1	17730	6884	6672	6672
q2	2070	168	172	168
q3	10745	1076	1220	1076
q4	10471	766	807	766
q5	7772	2930	2833	2833
q6	218	138	130	130
q7	983	610	628	610
q8	9348	1948	2073	1948
q9	6685	6437	6441	6437
q10	6986	2313	2291	2291
q11	469	258	261	258
q12	403	212	212	212
q13	17772	3005	3043	3005
q14	230	201	207	201
q15	500	451	466	451
q16	480	396	387	387
q17	989	627	522	522
q18	7332	6692	6724	6692
q19	1391	1049	1090	1049
q20	479	202	210	202
q21	3924	3099	3181	3099
q22	1123	1005	986	986
Total cold run time: 108100 ms
Total hot run time: 39995 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6596	6682	6593	6593
q2	328	232	230	230
q3	2923	2786	2950	2786
q4	2015	1845	1832	1832
q5	5839	5756	5755	5755
q6	211	128	129	128
q7	2236	1817	1781	1781
q8	3409	3602	3515	3515
q9	8874	8854	8919	8854
q10	3587	3553	3567	3553
q11	591	479	478	478
q12	827	585	624	585
q13	8081	3116	3222	3116
q14	298	265	272	265
q15	509	456	463	456
q16	507	449	440	440
q17	1888	1608	1632	1608
q18	8227	7825	7810	7810
q19	1688	1531	1478	1478
q20	2118	1820	1810	1810
q21	5209	4915	5027	4915
q22	1088	1036	1020	1020
Total cold run time: 67049 ms
Total hot run time: 59008 ms

doris-robot · 2025-07-03T13:12:17Z

TPC-DS: Total hot run time: 196186 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7dbd8926631a7598a5d97f7a67c41febbf210711, data reload: false

query1	1299	912	913	912
query2	6301	1930	1859	1859
query3	10815	4433	4360	4360
query4	60791	28901	23431	23431
query5	5206	460	430	430
query6	393	178	179	178
query7	5446	318	317	317
query8	318	225	221	221
query9	8487	2589	2571	2571
query10	431	274	251	251
query11	17800	15223	15561	15223
query12	156	112	103	103
query13	1449	439	436	436
query14	10454	6764	6734	6734
query15	201	183	177	177
query16	7082	482	484	482
query17	1200	586	586	586
query18	1883	326	319	319
query19	216	159	159	159
query20	118	111	108	108
query21	209	107	104	104
query22	4710	4542	4885	4542
query23	34927	33952	34456	33952
query24	6174	2961	2882	2882
query25	545	426	431	426
query26	678	168	165	165
query27	1906	355	371	355
query28	4096	2158	2124	2124
query29	711	475	465	465
query30	245	157	159	157
query31	989	807	831	807
query32	66	55	57	55
query33	410	309	304	304
query34	917	516	516	516
query35	855	759	713	713
query36	1064	944	964	944
query37	118	61	72	61
query38	3971	3947	3930	3930
query39	1512	1476	1446	1446
query40	210	98	94	94
query41	47	47	48	47
query42	110	102	97	97
query43	519	484	481	481
query44	1179	843	848	843
query45	178	167	167	167
query46	1136	708	717	708
query47	2033	1963	1942	1942
query48	488	379	389	379
query49	723	411	394	394
query50	828	406	409	406
query51	7485	7295	7248	7248
query52	103	87	87	87
query53	264	185	182	182
query54	570	450	465	450
query55	81	75	76	75
query56	266	277	252	252
query57	1311	1208	1227	1208
query58	231	204	215	204
query59	3161	2963	2906	2906
query60	296	253	254	253
query61	146	112	113	112
query62	808	679	697	679
query63	216	193	190	190
query64	2411	646	634	634
query65	3366	3208	3203	3203
query66	814	287	293	287
query67	15758	15654	15621	15621
query68	4152	577	571	571
query69	420	257	258	257
query70	1169	1075	1124	1075
query71	325	261	248	248
query72	6391	4082	3899	3899
query73	740	347	349	347
query74	10487	8969	9130	8969
query75	3362	2653	2618	2618
query76	2197	1022	1148	1022
query77	493	275	263	263
query78	10697	9697	9672	9672
query79	1537	607	581	581
query80	881	416	427	416
query81	494	220	219	219
query82	1250	87	85	85
query83	245	142	141	141
query84	283	75	81	75
query85	890	301	289	289
query86	323	301	295	295
query87	4480	4173	4213	4173
query88	3843	2365	2347	2347
query89	416	283	285	283
query90	1968	179	181	179
query91	186	145	148	145
query92	68	48	48	48
query93	1944	554	547	547
query94	764	293	300	293
query95	363	256	255	255
query96	620	277	274	274
query97	3348	3149	3178	3149
query98	215	200	200	200
query99	1576	1297	1323	1297
Total cold run time: 315014 ms
Total hot run time: 196186 ms

doris-robot · 2025-07-03T13:17:36Z

ClickBench: Total hot run time: 29.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7dbd8926631a7598a5d97f7a67c41febbf210711, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.11	0.11
query5	0.52	0.52	0.53
query6	1.15	0.72	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.55	0.50	0.48
query10	0.54	0.54	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.59	0.59
query14	0.77	0.79	0.80
query15	0.86	0.81	0.82
query16	0.38	0.35	0.38
query17	0.99	0.99	1.02
query18	0.24	0.20	0.22
query19	1.89	1.86	1.79
query20	0.02	0.01	0.01
query21	15.41	0.60	0.59
query22	2.12	1.59	2.51
query23	16.97	1.01	0.72
query24	2.69	1.09	1.14
query25	0.24	0.10	0.09
query26	0.47	0.14	0.13
query27	0.04	0.04	0.06
query28	10.60	0.47	0.44
query29	12.63	3.18	3.23
query30	0.26	0.06	0.06
query31	2.86	0.39	0.38
query32	3.25	0.47	0.45
query33	2.98	3.00	3.02
query34	17.08	4.47	4.47
query35	4.51	4.51	4.45
query36	0.68	0.47	0.48
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 104.09 s
Total hot run time: 29.71 s

doris-robot · 2025-07-03T13:18:27Z

BE UT Coverage Report

Increment line coverage 83.33% (25/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	41.24% (10944/26539)
Line Coverage	32.01% (93668/292621)
Region Coverage	31.12% (48324/155289)
Branch Coverage	27.57% (24748/89774)

bobhan1 · 2025-07-04T01:59:53Z

run p0

bobhan1 · 2025-07-04T06:11:02Z

p0 failed due to memory leaks, which will be fixed in #52718

dataroaring

LGTM

github-actions · 2025-07-07T01:56:20Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-07-07T01:56:22Z

PR approved by anyone and no changes requested.

@0x10

fix apache#52596, DeleteBitmapAggCache should be destructed before CacheManager ``` SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /root/doris/be/src/runtime/memory/cache_policy.cpp:35:31 *** Query id: 0-0 *** *** is nereids: 0 *** *** tablet id: 0 *** *** Aborted at 1751405577 (unix time) try "date -d @1751405577" if you are using GNU date *** *** Current BE git commitID: cd35cf0eb5 *** *** SIGSEGV address not mapped to object (@0x10) received by PID 15013 (TID 15013 OR 0x7fced202a4c0) from PID 16; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420 1# PosixSignals::chained_handler(int, siginfo_t*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007FCED2404090 in /lib/x86_64-linux-gnu/libc.so.6 4# __pthread_mutex_lock at ../nptl/pthread_mutex_lock.c:67 5# doris::CacheManager::unregister_cache(doris::CachePolicy::CacheType) at /root/doris/be/src/runtime/memory/cache_manager.h:56 6# doris::CachePolicy::~CachePolicy() at /root/doris/be/src/runtime/memory/cache_policy.cpp:36 7# doris::DeleteBitmap::AggCachePolicy::~AggCachePolicy() at /root/doris/be/src/olap/tablet_meta.h:568 8# __run_exit_handlers at /build/glibc-SzIz7B/glibc-2.31/stdlib/exit.c:109 9# on_exit at /build/glibc-SzIz7B/glibc-2.31/stdlib/on_exit.c:26 10# __libc_start_main at ../csu/libc-start.c:261 11# _start in /home/work/unlimit_teamcity/TeamCity/Agents/20250702042258agent_172.16.0.200_1/work/60183217f6ee2a9c/output/be/lib/doris_be

bobhan1 · 2025-07-07T07:36:15Z

run buildall

doris-robot · 2025-07-07T09:02:29Z

BE UT Coverage Report

Increment line coverage 83.33% (25/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	41.21% (10947/26563)
Line Coverage	31.97% (93689/293081)
Region Coverage	31.10% (48330/155426)
Branch Coverage	27.54% (24747/89858)

dataroaring

LGTM

github-actions · 2025-07-07T13:04:53Z

PR approved by at least one committer and no changes requested.

bobhan1 requested a review from dataroaring as a code owner July 3, 2025 11:48

dataroaring previously approved these changes Jul 7, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 7, 2025

github-actions bot added the reviewed label Jul 7, 2025

bobhan1 dismissed dataroaring’s stale review via 0c408a8 July 7, 2025 07:35

bobhan1 force-pushed the branch-3.0-pick-52649 branch from 7dbd892 to 0c408a8 Compare July 7, 2025 07:35

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 7, 2025

dataroaring approved these changes Jul 7, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 7, 2025

dataroaring merged commit 63b07dd into apache:branch-3.0 Jul 8, 2025
21 of 25 checks passed

branch-3.0: [Opt](mow) Move DeleteBitmapAggCache to ExecEnv (#52649) #52737

branch-3.0: [Opt](mow) Move DeleteBitmapAggCache to ExecEnv (#52649) #52737

Uh oh!

Conversation

bobhan1 commented Jul 3, 2025

Uh oh!

hello-stephen commented Jul 3, 2025

Uh oh!

bobhan1 commented Jul 3, 2025

Uh oh!

doris-robot commented Jul 3, 2025

Uh oh!

doris-robot commented Jul 3, 2025

Uh oh!

doris-robot commented Jul 3, 2025

Uh oh!

doris-robot commented Jul 3, 2025

BE UT Coverage Report

Uh oh!

bobhan1 commented Jul 4, 2025

Uh oh!

bobhan1 commented Jul 4, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

bobhan1 commented Jul 7, 2025

Uh oh!

doris-robot commented Jul 7, 2025

BE UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

branch-3.0: [Opt](mow) Move `DeleteBitmapAggCache` to `ExecEnv` (#52649) #52737

branch-3.0: [Opt](mow) Move `DeleteBitmapAggCache` to `ExecEnv` (#52649) #52737