[fix](partial update) fix some bugs about delete sign #25712

zhannngchen · 2023-10-20T14:11:54Z

Proposed changes

Issue Number: close #xxx

Fix core, when there is conflict between multiple partial update loads, we need to write updated rows to an additional segment file through a transient rowset writer. The transient rowset writer don't initialize the _mow_context member(and also is unnecessary), [Enhancement](merge-on-write) use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist #24011 introduce specialized delete mark for rows with delete sign, which caused the following core:

*** Query id: 0-0 ***
*** tablet id: 0 ***
*** Aborted at 1697799372 (unix time) try "date -d @1697799372" if you are using GNU date ***
*** Current BE git commitID: 584111abaf ***
*** SIGSEGV address not mapped to object (@0x18) received by PID 464733 (TID 466003 OR 0x7f03d15f0700) from PID 24; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/zhangchen/doris-dev/be/src/common/signal_handler.h:417
 1# 0x00007F058A018570 in /lib64/libc.so.6
 2# doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/rowset/segment_v2/segment_writer.cpp:703
 3# doris::SegmentFlusher::_add_rows(std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_delete<doris::segment_v2::SegmentWriter> >&, doris::vectorized::Block const*, unsigned long, unsigned long) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/rowset/segment_creator.cpp:78
 4# doris::SegmentFlusher::flush_single_block(doris::vectorized::Block const*, int, long*, std::shared_ptr<doris::TabletSchema>) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/rowset/segment_creator.cpp:56
 5# doris::SegmentCreator::flush_single_block(doris::vectorized::Block const*, int, long*, std::shared_ptr<doris::TabletSchema>) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/rowset/segment_creator.cpp:231
 6# doris::BetaRowsetWriter::flush_single_block(doris::vectorized::Block const*) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/rowset/beta_rowset_writer.cpp:457
 7# doris::Tablet::calc_segment_delete_bitmap(std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, std::shared_ptr<doris::DeleteBitmap>, long, doris::RowsetWriter*) at /mnt/disk2/zhangchen/doris-dev/be/src/olap/tablet.cpp:3105
 8# std::_Function_handler<void (), doris::CalcDeleteBitmapToken::submit(std::shared_ptr<doris::Tablet>, std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, long, doris::RowsetWriter*)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/zhangchen/env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
 9# doris::ThreadPool::dispatch_thread() in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
10# doris::Thread::supervise_thread(void*) at /mnt/disk2/zhangchen/doris-dev/be/src/util/thread.cpp:495
11# start_thread at pthread_create.c:480
12# clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

A bug in [Enhancement](merge-on-write) use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist #24011, missed delete mark for new key with delete sign(which not exists in history data)
To fix issue 1, we need to pass the delete bitmap to CalcDeleteBitmapExecutor

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions · 2023-10-20T14:21:01Z