-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix](load) add null check for memtable after write failure reset #47860
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
398669b to
1b9536a
Compare
|
run buildall |
1b9536a to
14c8459
Compare
|
run buildall |
TPC-H: Total hot run time: 31841 ms |
TPC-DS: Total hot run time: 189717 ms |
ClickBench: Total hot run time: 30.46 s |
|
TeamCity be ut coverage result: |
dataroaring
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
…7860) Related PR: #47610 Problem Summary: SIGSEGV address not mapped to object (@0x0) received by PID 340906 (TID 341622 OR 0x7f7f38784640) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007F80B37D0520 in /lib/x86_64-linux-gnu/libc.so.6 4# doris::MemTableWriter::_flush_memtable_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:157 5# doris::MemTableWriter::flush_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:187 6# doris::MemTableMemoryLimiter::_flush_active_memtables(long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:190 7# doris::MemTableMemoryLimiter::handle_memtable_flush() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:144 8# doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/runtime/load_channel_mgr.cpp:154 9# std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 10# doris::WorkThreadPool<false>::work_thread(int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/work_thread_pool.hpp:159 11# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84 12# start_thread at ./nptl/pthread_create.c:442 13# 0x00007F80B38B4850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83 This PR addresses potential null pointer dereference crashes that could occur when write operations fail and the memtable is reset. The changes add defensive null checks to ensure safe handling of the _mem_table state during flush memtable.
…ache#47860) Related PR: apache#47610 Problem Summary: SIGSEGV address not mapped to object (@0x0) received by PID 340906 (TID 341622 OR 0x7f7f38784640) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007F80B37D0520 in /lib/x86_64-linux-gnu/libc.so.6 4# doris::MemTableWriter::_flush_memtable_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:157 5# doris::MemTableWriter::flush_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:187 6# doris::MemTableMemoryLimiter::_flush_active_memtables(long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:190 7# doris::MemTableMemoryLimiter::handle_memtable_flush() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:144 8# doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/runtime/load_channel_mgr.cpp:154 9# std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 10# doris::WorkThreadPool<false>::work_thread(int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/work_thread_pool.hpp:159 11# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84 12# start_thread at ./nptl/pthread_create.c:442 13# 0x00007F80B38B4850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83 This PR addresses potential null pointer dereference crashes that could occur when write operations fail and the memtable is reset. The changes add defensive null checks to ensure safe handling of the _mem_table state during flush memtable.
…eset (apache#47860)" This reverts commit 7490ea8.
…ache#47860) Related PR: apache#47610 Problem Summary: SIGSEGV address not mapped to object (@0x0) received by PID 340906 (TID 341622 OR 0x7f7f38784640) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007F80B37D0520 in /lib/x86_64-linux-gnu/libc.so.6 4# doris::MemTableWriter::_flush_memtable_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:157 5# doris::MemTableWriter::flush_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:187 6# doris::MemTableMemoryLimiter::_flush_active_memtables(long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:190 7# doris::MemTableMemoryLimiter::handle_memtable_flush() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:144 8# doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/runtime/load_channel_mgr.cpp:154 9# std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 10# doris::WorkThreadPool<false>::work_thread(int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/work_thread_pool.hpp:159 11# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84 12# start_thread at ./nptl/pthread_create.c:442 13# 0x00007F80B38B4850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83 This PR addresses potential null pointer dereference crashes that could occur when write operations fail and the memtable is reset. The changes add defensive null checks to ensure safe handling of the _mem_table state during flush memtable.
### What problem does this PR solve? Issue Number: DORIS-18927 Related PR: apache#47860 and apache#47610
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #47610
Problem Summary:
SIGSEGV address not mapped to object (@0x0) received by PID 340906 (TID 341622 OR 0x7f7f38784640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
3# 0x00007F80B37D0520 in /lib/x86_64-linux-gnu/libc.so.6
4# doris::MemTableWriter::_flush_memtable_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:157
5# doris::MemTableWriter::flush_async() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_writer.cpp:187
6# doris::MemTableMemoryLimiter::_flush_active_memtables(long) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:190
7# doris::MemTableMemoryLimiter::handle_memtable_flush() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/memtable_memory_limiter.cpp:144
8# doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/runtime/load_channel_mgr.cpp:154
9# std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
10# doris::WorkThreadPool::work_thread(int) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/work_thread_pool.hpp:159
11# execute_native_thread_routine at ../../../../../libstdc++-v3/src/c++11/thread.cc:84
12# start_thread at ./nptl/pthread_create.c:442
13# 0x00007F80B38B4850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
This PR addresses potential null pointer dereference crashes that could occur when write operations fail and the memtable is reset. The changes add defensive null checks to ensure safe handling of the _mem_table state during flush memtable.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)