Skip to content

[fix](memory) Fix OwnedSlice free memory #40043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Aug 28, 2024
Merged

[fix](memory) Fix OwnedSlice free memory #40043

yiguolei merged 1 commit into from
Aug 28, 2024

Conversation

@xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Aug 28, 2024 

I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone

I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon

@doris-robot
Copy link

doris-robot commented Aug 28, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Aug 28, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2024

clang-tidy review says "All clean, LGTM! 👍"

yiguolei
yiguolei approved these changes Aug 28, 2024
View reviewed changes
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 28, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2024

PR approved by anyone and no changes requested.

@yiguolei
Copy link
Contributor

yiguolei commented Aug 28, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Aug 28, 2024

TPC-H: Total hot run time: 38414 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 28dd4dd7bb4dd1532161b5cb33baf43a783533d3, data reload: false

------ Round 1 ----------------------------------
q1	17803	4520	4307	4307
q2	2018	187	185	185
q3	10509	1241	1080	1080
q4	10173	653	763	653
q5	7759	3057	2841	2841
q6	236	141	146	141
q7	972	618	624	618
q8	9334	2086	2105	2086
q9	7201	6535	6521	6521
q10	7019	2295	2234	2234
q11	453	246	240	240
q12	409	238	232	232
q13	17807	3054	3149	3054
q14	292	244	231	231
q15	526	487	486	486
q16	571	514	520	514
q17	989	661	645	645
q18	7485	6938	6945	6938
q19	1402	1013	1097	1013
q20	680	333	324	324
q21	4101	3049	3105	3049
q22	1142	1025	1022	1022
Total cold run time: 108881 ms
Total hot run time: 38414 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4356	4238	4319	4238
q2	394	288	281	281
q3	2904	2721	2714	2714
q4	1973	1662	1691	1662
q5	5451	5425	5450	5425
q6	221	134	135	134
q7	2110	1761	1750	1750
q8	3209	3370	3330	3330
q9	8551	8408	8414	8408
q10	3469	3234	3237	3234
q11	595	522	491	491
q12	796	630	617	617
q13	13085	3062	3033	3033
q14	320	282	280	280
q15	522	474	485	474
q16	605	543	555	543
q17	1818	1515	1474	1474
q18	7851	7596	7487	7487
q19	1656	1498	1576	1498
q20	2059	1829	1808	1808
q21	5513	5240	5258	5240
q22	1130	1098	1030	1030
Total cold run time: 68588 ms
Total hot run time: 55151 ms

@doris-robot
Copy link

doris-robot commented Aug 28, 2024

TPC-DS: Total hot run time: 188207 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 28dd4dd7bb4dd1532161b5cb33baf43a783533d3, data reload: false

query1	929	382	378	378
query2	6460	1989	1901	1901
query3	6640	216	224	216
query4	30565	23107	23157	23107
query5	4161	523	501	501
query6	265	168	165	165
query7	4596	297	293	293
query8	252	214	223	214
query9	8623	2499	2492	2492
query10	428	273	272	272
query11	16264	15301	15348	15301
query12	158	99	100	99
query13	1640	384	374	374
query14	10072	6931	6905	6905
query15	263	168	181	168
query16	8103	459	484	459
query17	1572	567	538	538
query18	2108	282	285	282
query19	268	151	144	144
query20	115	109	109	109
query21	211	103	102	102
query22	4370	4161	4127	4127
query23	34183	33449	33324	33324
query24	11138	2927	2936	2927
query25	652	420	405	405
query26	1200	166	177	166
query27	2358	289	287	287
query28	7397	2132	2108	2108
query29	849	436	477	436
query30	294	157	150	150
query31	977	747	791	747
query32	100	59	64	59
query33	750	284	291	284
query34	992	520	514	514
query35	885	752	717	717
query36	1058	937	965	937
query37	167	101	97	97
query38	4022	3852	3841	3841
query39	1437	1436	1395	1395
query40	205	117	120	117
query41	50	47	46	46
query42	113	97	101	97
query43	501	467	450	450
query44	1188	764	746	746
query45	202	169	168	168
query46	1100	773	763	763
query47	1851	1823	1766	1766
query48	372	291	295	291
query49	1093	427	429	427
query50	807	422	429	422
query51	7224	7080	7066	7066
query52	99	94	91	91
query53	269	190	184	184
query54	909	466	457	457
query55	79	80	78	78
query56	305	261	254	254
query57	1229	1073	1085	1073
query58	247	239	236	236
query59	2909	2669	2635	2635
query60	313	275	274	274
query61	100	97	101	97
query62	847	631	674	631
query63	224	190	188	188
query64	5265	707	637	637
query65	3227	3190	3175	3175
query66	1420	335	345	335
query67	15604	15378	15235	15235
query68	4090	572	565	565
query69	409	278	280	278
query70	1178	1105	1133	1105
query71	347	286	273	273
query72	6409	4067	3982	3982
query73	741	343	351	343
query74	9038	8892	8839	8839
query75	3404	2720	2625	2625
query76	2294	999	1039	999
query77	496	332	333	332
query78	9584	9010	9085	9010
query79	1071	568	563	563
query80	682	514	531	514
query81	454	243	234	234
query82	243	158	153	153
query83	176	153	153	153
query84	222	76	74	74
query85	673	286	281	281
query86	313	291	272	272
query87	4383	4258	4263	4258
query88	3207	2353	2341	2341
query89	387	289	287	287
query90	1823	206	201	201
query91	127	108	101	101
query92	60	53	51	51
query93	1042	544	547	544
query94	714	351	311	311
query95	363	262	261	261
query96	588	271	277	271
query97	3205	3096	3046	3046
query98	214	228	204	204
query99	1439	1256	1277	1256
Total cold run time: 283650 ms
Total hot run time: 188207 ms

@doris-robot
Copy link

doris-robot commented Aug 28, 2024

ClickBench: Total hot run time: 31.97 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 28dd4dd7bb4dd1532161b5cb33baf43a783533d3, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.68	0.07	0.06
query5	0.50	0.49	0.50
query6	1.12	0.74	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.56	0.48	0.48
query10	0.55	0.53	0.54
query11	0.16	0.11	0.12
query12	0.15	0.12	0.11
query13	0.61	0.59	0.58
query14	2.09	2.05	2.03
query15	0.91	0.83	0.81
query16	0.37	0.37	0.38
query17	1.06	1.04	1.00
query18	0.21	0.20	0.22
query19	1.81	1.75	1.76
query20	0.01	0.01	0.01
query21	15.41	0.69	0.68
query22	4.41	7.49	1.82
query23	18.24	1.43	1.29
query24	2.03	0.26	0.23
query25	0.16	0.09	0.07
query26	0.26	0.17	0.18
query27	0.08	0.07	0.07
query28	13.23	1.02	1.01
query29	12.59	3.31	3.32
query30	0.25	0.05	0.05
query31	2.89	0.41	0.39
query32	3.24	0.48	0.48
query33	2.95	3.02	3.01
query34	17.12	4.54	4.44
query35	4.50	4.45	4.46
query36	0.65	0.50	0.47
query37	0.20	0.15	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.13
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 110.99 s
Total hot run time: 31.97 s

@yiguolei yiguolei merged commit d09a39b into apache:master Aug 28, 2024
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 9, 2024
@xinyiZzz
[fix](memory) Fix OwnedSlice free memory (apache#40043)
20a56ac 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
@xinyiZzz xinyiZzz mentioned this pull request Sep 9, 2024
Closed
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 9, 2024
@xinyiZzz
[fix](memory) Fix OwnedSlice free memory (apache#40043)
03309a3 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 10, 2024
@xinyiZzz
[fix](memory) Fix OwnedSlice free memory (apache#40043)
988cf89 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
yiguolei pushed a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 10, 2024
@xinyiZzz @yiguolei
[fix](memory) Fix OwnedSlice free memory (apache#40043)
ae46da6 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
yiguolei pushed a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 11, 2024
@xinyiZzz @yiguolei
[fix](memory) Fix OwnedSlice free memory (apache#40043)
c765804 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Sep 14, 2024
@xinyiZzz
[fix](memory) Fix OwnedSlice free memory (apache#40043)
8e705e5 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
@xinyiZzz xinyiZzz mentioned this pull request Sep 14, 2024
Merged
yiguolei added a commit that referenced this pull request Sep 15, 2024
@xinyiZzz @sollhui @yiguolei
[branch-2.1](memory) When Load ends, check memory tracker value retur…
b52b572 
…ns is equal to 0 (#40850)

pick
#38960
#39908
#40043
#40092
#40016
#40439

---------

Co-authored-by: hui lai <1353307710@qq.com>
Co-authored-by: yiguolei <676222867@qq.com>
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@xinyiZzz @dataroaring
[fix](memory) Fix OwnedSlice free memory (#40043)
a020c02 
```
I20240828 11:41:58.529152 20190 mem_tracker_limiter.cpp:193] [Address Sanitizer] memory buf not exist, mem tracker label: Load#Id=ac42f13dd1430d2c-bd60f5a4829d0792, consumption: 13054587, peak consumption: 13054587, buf: 0, size: 0, stack_trace:
        0#  doris::OwnedSlice::~OwnedSlice()
        1#  doris::segment_v2::ScalarColumnWriter::finish_current_page()
        2#  doris::segment_v2::ScalarColumnWriter::finish()
        3#  doris::segment_v2::VerticalSegmentWriter::write_batch()
        4#  doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long)
        5#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
        6#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
        7#  doris::BaseBetaRowsetWriter::flush_memtable(doris::vectorized::Block*, int, long*)
        8#  doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
        9#  doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
        10# doris::MemtableFlushTask::run()
        11# doris::ThreadPool::dispatch_thread()
        12# doris::Thread::supervise_thread(void*)
        13# ?
        14# clone
```
```
I20240828 11:41:58.613629 20183 mem_tracker_limiter.cpp:182] [Address Sanitizer] free memory buf size inaccurate, mem tracker label: Load#Id=433657e8b3834e94-ac178bb7ab8ff661, consumption: 3239536, peak consumption: 6385184, buf: 0x6030015390a0, size: 32, old buf: 0x6030015390a0, old size: 20, new stack_trace:
	0#  doris::OwnedSlice::~OwnedSlice()
	1#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	2#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	3#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	4#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	5#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	6#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	7#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	8#  doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	9#  doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	10# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	11# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	12# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	13# doris::MemtableFlushTask::run()
	14# doris::ThreadPool::dispatch_thread()
	15# doris::Thread::supervise_thread(void*)
	16# ?
	17# clone
, old stack_trace:
	0#  Allocator<false, false, false, DefaultMemoryAllocator>::alloc_impl(unsigned long, unsigned long)
	1#  doris::faststring::build()
	2#  doris::segment_v2::BinaryPrefixPageBuilder::finish(doris::OwnedSlice*)
	3#  doris::segment_v2::IndexedColumnWriter::_finish_current_data_page(unsigned long&)
	4#  doris::segment_v2::IndexedColumnWriter::finish(doris::segment_v2::IndexedColumnMetaPB*)
	5#  doris::PrimaryKeyIndexBuilder::finalize(doris::segment_v2::PrimaryKeyIndexMetaPB*)
	6#  doris::segment_v2::VerticalSegmentWriter::_write_primary_key_index()
	7#  doris::segment_v2::VerticalSegmentWriter::finalize_columns_index(unsigned long*)
	8#  doris::segment_v2::VerticalSegmentWriter::finalize(unsigned long*, unsigned long*)
	9#  doris::SegmentFlusher::_flush_segment_writer(std::unique_ptr<doris::segment_v2::VerticalSegmentWriter, std::default_delete<doris::segment_v2::VerticalSegmentWriter> >&, std::shared_ptr<doris::TabletSchema>, long*)
	10# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*)
	11# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*)
	12# doris::BetaRowsetWriterV2::flush_memtable(doris::vectorized::Block*, int, long*)
	13# doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
	14# doris::FlushToken::_flush_memtable(std::unique_ptr<doris::MemTable, std::default_delete<doris::MemTable> >, int, long)
	15# doris::MemtableFlushTask::run()
	16# doris::ThreadPool::dispatch_thread()
	17# doris::Thread::supervise_thread(void*)
	18# ?
	19# clon
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

@jacktengg jacktengg jacktengg approved these changes

@Gabriel39 Gabriel39 Gabriel39 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@xinyiZzz @doris-robot @yiguolei @jacktengg @Gabriel39 @dataroaring