Skip to content

[BUG] When the bitmap object is too big, it will cause BE coredump during compaction. #5892

@stdpain

Description

@stdpain

Describe the bug
RT

To Reproduce
Steps to reproduce the behavior:

  1. generate data from 2^32 ~ 2^64
  2. use stream load to bitmap column
  3. wait a monment be will core

log:

*** SIGSEGV (@0xffffffffc9fcdee5) received by PID 27534 (TID 0x7f9c2e74a700) from PID 18446744072803376869; stack trace: ***
    @          0x2d1ff62 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f9ca10e71d0 (unknown)
    @     0x7f9ca0f2f4c3 __memmove_avx_unaligned_erms
    @          0x2bcceb7 LZ4_decompress_safe_usingDict
    @          0x2bd2c98 LZ4F_decompress
    @          0x21a8391 doris::Lz4fBlockCompression::_decompress()
    @          0x21a7d30 doris::Lz4fBlockCompression::decompress()
    @          0x1ee5bc7 doris::segment_v2::PageIO::read_and_decompress_page()
    @          0x289db36 doris::segment_v2::ColumnReader::read_page()
    @          0x28a0f1f doris::segment_v2::FileColumnIterator::_read_data_page()
    @          0x28a02e7 doris::segment_v2::FileColumnIterator::seek_to_ordinal()
    @          0x284753b doris::segment_v2::SegmentIterator::_seek_columns()
    @          0x2847aaa doris::segment_v2::SegmentIterator::next_batch()
    @          0x1f5ee65 doris::BetaRowsetReader::next_block()
    @          0x1e66cba doris::CollectIterator::Level0Iterator::_refresh_current_row_v2()
    @          0x1e669ac doris::CollectIterator::Level0Iterator::init()
    @          0x1e65cb9 doris::CollectIterator::add_child()
    @          0x1e2b018 doris::Reader::_capture_rs_readers()
    @          0x1e29728 doris::Reader::init()
    @          0x1e1c0d2 doris::Merger::merge_rowsets()
    @          0x1dfd4e1 doris::Compaction::do_compaction_impl()
    @          0x1dfcf04 doris::Compaction::do_compaction()
    @          0x1df5c31 doris::BaseCompaction::execute_compact_impl()
    @          0x1dfcdf7 doris::Compaction::execute_compact()
    @          0x1d6c1ff doris::Tablet::execute_compaction()
    @          0x1d183ea _ZZN5doris13StorageEngine35_compaction_tasks_producer_callbackEvENKUlvE0_clEv
    @          0x1d1b852 _ZSt13__invoke_implIvRZN5doris13StorageEngine35_compaction_tasks_producer_callbackEvEUlvE0_JEET_St14__invoke_otherOT0_DpOT1_
    @          0x1d1ac0e _ZSt10__invoke_rIvRZN5doris13StorageEngine35_compaction_tasks_producer_callbackEvEUlvE0_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES5_E4typeEOS6_DpOS7_
    @          0x1d1a217 _ZNSt17_Function_handlerIFvvEZN5doris13StorageEngine35_compaction_tasks_producer_callbackEvEUlvE0_E9_M_invokeERKSt9_Any_data
    @          0x1ff15b8 std::function<>::operator()()
    @          0x225b64e doris::FunctionRunnable::run()
    @          0x225a015 doris::ThreadPool::dispatch_thread()

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/storageIssues or PRs related to storage enginekind/fixCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions