[Bug] be asan block reader _update_agg_value mem leak.

[swjtu-zhanglei]

Search before asking

• I had searched in the issues and found no similar issues.

Version

trunk DEBUG (build git://hk-dev01-ali/mnt/disk1/zhanglei/work/repo/incubator-doris/be/../@d51166dd2aa3ed75429b379a1ea36bc657c50b32)
Built on Fri, 17 Jun 2022 16:13:37 CST by zhanglei@hk-dev01-ali

What's Wrong?

4 ==2027722==ERROR: LeakSanitizer: detected memory leaks
5
6 Direct leak of 192 byte(s) in 3 object(s) allocated from:
7 #0 0x56350e7b7927 in operator new(unsigned long) (/mnt/disk1/zhanglei/work/repo/test/doris_master/be/lib/palo_be+0x55a5927)
8 #1 0x56350eaf42cf in __gnu_cxx::new_allocator<phmap::priv::Allocate<8ul, std::allocator<std::pair<unsigned int const, roaring::Roaring> > >(std::allocator<std::pair<unsigned int const, roaring::Roaring> >, unsigned long)::M>::allocate(u 8 nsigned long, void const) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/ext/new_allocator.h:121
9 #2 0x56350eaed06a in phmap::allocator_traits<std::allocator<phmap::priv::Allocate<8ul, std::allocator<std::pair<unsigned int const, roaring::Roaring> > >(std::allocator<std::pair<unsigned int const, roaring::Roaring> >, unsigned long):: 9 M> >::allocate(std::allocator<phmap::priv::Allocate<8ul, std::allocator<std::pair<unsigned int const, roaring::Roaring> > >(std::allocator<std::pair<unsigned int const, roaring::Roaring> >, unsigned long)::M>&, unsigned long) /mnt/disk1/zha 9 nglei/work/repo/incubator-doris/thirdparty/installed/include/parallel_hashmap/phmap_base.h:1468
10 #3 0x56350eadfbd0 in void* phmap::priv::Allocate<8ul, std::allocator<std::pair<unsigned int const, roaring::Roaring> > >(std::allocator<std::pair<unsigned int const, roaring::Roaring> >, unsigned long) /mnt/disk1/zhanglei/work/repo/incu 10 bator-doris/thirdparty/installed/include/parallel_hashmap/phmap_base.h:4363
11 #4 0x56350ead25b3 in phmap::priv::btree<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, false> >::allocate(long) /mnt/disk1/zhangle 11 i/work/repo/incubator-doris/thirdparty/installed/include/parallel_hashmap/btree.h:1972
12 #5 0x56350eac3696 in phmap::priv::btree<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, false> >::new_leaf_root_node(int) /mnt/disk 12 1/zhanglei/work/repo/incubator-doris/thirdparty/installed/include/parallel_hashmap/btree.h:1986
13 #6 0x56350eabfddf in std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, 13 false> >, std::pair<unsigned int const, roaring::Roaring>&, std::pair<unsigned int const, roaring::Roaring>>, bool> phmap::priv::btree<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pa 13 ir<unsigned int const, roaring::Roaring> >, 256, false> >::insert_unique<std::piecewise_construct_t const&, std::tuple<unsigned int&&>, std::tuple<> >(unsigned int const&, std::piecewise_construct_t const&, std::tuple<unsigned int&&>&&, std: 13 :tuple<>&&) (/mnt/disk1/zhanglei/work/repo/test/doris_master/be/lib/palo_be+0x58adddf)
14 #7 0x56350eab1b1c in std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, 14 false> >, std::pair<unsigned int const, roaring::Roaring>&, std::pair<unsigned int const, roaring::Roaring>>, bool> phmap::priv::btree_map_container<phmap::priv::btree<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less<unsi 14 gned int>, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, false> > >::try_emplace<>(unsigned int&&) /mnt/disk1/zhanglei/work/repo/incubator-doris/thirdparty/installed/include/parallel_hashmap/btree.h:3608
15 #8 0x56350eaae9b7 in phmap::priv::btree_map_container<phmap::priv::btree<phmap::priv::map_params<unsigned int, roaring::Roaring, phmap::Less, std::allocator<std::pair<unsigned int const, roaring::Roaring> >, 256, false> > > 15 ::operator[](unsigned int&&) /mnt/disk1/zhanglei/work/repo/incubator-doris/thirdparty/installed/include/parallel_hashmap/btree.h:3638
16 #9 0x56350ea983b2 in doris::detail::Roaring64Map::add(unsigned long) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/util/bitmap_value.h:154
17 #10 0x56350ea9d619 in doris::BitmapValue::add(unsigned long) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/util/bitmap_value.h:1172
18 #11 0x56350ea9d9ef in doris::BitmapValue::operator|=(doris::BitmapValue const&) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/util/bitmap_value.h:1234
19 #12 0x563510f42e3b in doris::vectorized::AggregateFunctionBitmapUnionOp::add(doris::BitmapValue&, doris::BitmapValue const&, bool&) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/aggregate_functions/aggregate_function_bitmap.h: 19 39
20 #13 0x563510f533a6 in void doris::vectorized::AggregateFunctionBitmapDatadoris::vectorized::AggregateFunctionBitmapUnionOp::adddoris::BitmapValue(doris::BitmapValue const&) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/agg 20 regate_functions/aggregate_function_bitmap.h:73
21 #14 0x563510f52b74 in doris::vectorized::AggregateFunctionBitmapOpdoris::vectorized::AggregateFunctionBitmapUnionOp::add(char, doris::vectorized::IColumn const**, unsigned long, doris::vectorized::Arena*) const /mnt/disk1/zhanglei/wor 21 k/repo/incubator-doris/be/src/vec/aggregate_functions/aggregate_function_bitmap.h:105
22 #15 0x563510f52dc8 in doris::vectorized::IAggregateFunctionHelper<doris::vectorized::AggregateFunctionBitmapOpdoris::vectorized::AggregateFunctionBitmapUnionOp >::add_batch_range(unsigned long, unsigned long, char*, doris::vectorized:: 22 IColumn const**, doris::vectorized::Arena*, bool) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/aggregate_functions/aggregate_function.h:176
23 #16 0x563510b653e7 in doris::vectorized::BlockReader::_update_agg_value(std::vector<COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn, std::allocator<COWdoris::vectorized::IColumn::mutable_ptr<doris::vectorized:: 23 IColumn> > >&, int, int, bool) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/olap/block_reader.cpp:344
24 #17 0x563510b643ac in doris::vectorized::BlockReader::_update_agg_data(std::vector<COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn, std::allocator<COWdoris::vectorized::IColumn::mutable_ptr<doris::vectorized::I 24 Column> > >&) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/olap/block_reader.cpp:291
25 #18 0x563510b63e7b in doris::vectorized::BlockReader::_append_agg_data(std::vector<COWdoris::vectorized::IColumn::mutable_ptrdoris::vectorized::IColumn, std::allocator<COWdoris::vectorized::IColumn::mutable_ptr<doris::vectorized::I 25 Column> > >&) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/olap/block_reader.cpp:269
26 #19 0x563510b63048 in doris::vectorized::BlockReader::_agg_key_next_block(doris::vectorized::Block*, doris::MemPool*, doris::ObjectPool*, bool*) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/olap/block_reader.cpp:211
27 #20 0x56350f072072 in doris::vectorized::BlockReader::next_block_with_aggregation(doris::vectorized::Block*, doris::MemPool*, doris::ObjectPool*, bool*) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/olap/block_reader.h:45
28 #21 0x563511c07c4a in doris::vectorized::VOlapScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/exec/volap_scanner.cpp:286
29 #22 0x563511b92f9d in doris::vectorized::VOlapScanNode::scanner_thread(doris::vectorized::VOlapScanner*) /mnt/disk1/zhanglei/work/repo/incubator-doris/be/src/vec/exec/volap_scan_node.cpp:446
30 #23 0x563511bed1b0 in void std::__invoke_impl<void, void (doris::vectorized::VOlapScanNode::&)(doris::vectorized::VOlapScanner), doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&>(std::__invoke_memfun_deref, void (d 30 oris::vectorized::VOlapScanNode::&)(doris::vectorized::VOlapScanner), doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/bits/invoke.h:74
31 #24 0x563511beccba in std::__invoke_result<void (doris::vectorized::VOlapScanNode::&)(doris::vectorized::VOlapScanner), doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&>::type std::__invoke<void (doris::vectorized: 31 :VOlapScanNode::&)(doris::vectorized::VOlapScanner), doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&>(void (doris::vectorized::VOlapScanNode::&)(doris::vectorized::VOlapScanner), doris::vectorized::VOlapScanNode*&, 31 doris::vectorized::VOlapScanner*&) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/bits/invoke.h:96
32 #25 0x563511bec4b2 in void std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>::__call<void, , 0ul, 1ul>(std::tuple<>&&, std::_Inde 32 x_tuple<0ul, 1ul>) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/functional:420
33 #26 0x563511bebd1c in void std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>::operator()<, void>() /mnt/disk1/zhanglei/work/app/l 33 db_toolchain/include/c++/11/functional:503
34 #27 0x563511beac5f in void std::__invoke_impl<void, std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>&>(std::__invoke_other, std: 34 :_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>&) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/bits/invoke.h:61
35 #28 0x563511be9735 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>&>, void>::type std: 35 :__invoke_r<void, std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>&>(std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vec 35 torized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)>&) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/bits/invoke.h:111
36 #29 0x563511be7a87 in std::_Function_handler<void (), std::_Bind<void (doris::vectorized::VOlapScanNode::(doris::vectorized::VOlapScanNode, doris::vectorized::VOlapScanner*))(doris::vectorized::VOlapScanner*)> >::_M_invoke(std::_Any_da 36 ta const&) /mnt/disk1/zhanglei/work/app/ldb_toolchain/include/c++/11/bits/std_function.h:291

What You Expected?

asan without mem leak report

How to Reproduce?

1、run regression-test with be(asan)
2、kill -15 bepid

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct