Skip to content

Conversation

@w41ter
Copy link
Contributor

@w41ter w41ter commented Mar 19, 2024

Proposed changes

Issue Number: close #xxx

The standard said that the input parameter pos of std::vector::erase must be valid and dereferenceable, the end() iterator cannot be used as a value of pos. I did some tests and the crash only occurs when the vector is empty. Fortunately local_files is usually not empty.

The crash stack is:

*** Query id: 0-0 ***
*** tablet id: 0 ***
*** Aborted at 1710410771 (unix time) try "date -d @1710410771" if you are using GNU date ***
*** Current BE git commitID: 91efb6a43d ***
*** SIGSEGV address not mapped to object (@0xffffffffffffffe0) received by PID 114167 (TID 116733 OR 0x7fd55292c700) from PID 18446744073709551584; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:417
 1# os::Linux::chained_handler(int, siginfo*, void*) in /mnt/disk1/caoliang/jdk1.8.0_331/jre/lib/amd64/server/libjvm.so
 2# JVM_handle_linux_signal in /mnt/disk1/caoliang/jdk1.8.0_331/jre/lib/amd64/server/libjvm.so
 3# signalHandler(int, siginfo*, void*) in /mnt/disk1/caoliang/jdk1.8.0_331/jre/lib/amd64/server/libjvm.so
 4# 0x00007FD981939B50 in /lib64/libc.so.6
 5# doris::SnapshotLoader::download(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::vector<long, std::allocator<long> >*) at /home/zcp/repo_center/doris_release/doris/be/src/runtime/snapshot_loader.cpp:344
 6# doris::download_callback(doris::StorageEngine&, doris::ExecEnv*, doris::TAgentTaskRequest const&) in /mnt/disk1/caoliang/doris/doris-2.1.0/apache-doris-2.1.0-rc10-bin-x64/be/lib/doris_be
 7# std::_Function_handler<void (), doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
 8# doris::ThreadPool::dispatch_thread() in /mnt/disk1/caoliang/doris/doris-2.1.0/apache-doris-2.1.0-rc10-bin-x64/be/lib/doris_be
 9# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:499
10# start_thread in /lib64/libpthread.so.0
11# __clone in /lib64/libc.so.6

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

The standard said that the input parameter `pos` of std::vector::erase
must be valid and dereferenceable, the `end()` iterator cannot be used
as a value of `pos`. I did some tests and the crash only occurs when the
vector is empty. Fortunately `local_files` is usually not empty.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Mar 19, 2024

run buildall

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.31% (8726/24713)
Line Coverage: 27.12% (71372/263168)
Region Coverage: 26.37% (37030/140423)
Branch Coverage: 23.26% (18930/81368)
Coverage Report: http://coverage.selectdb-in.cc/coverage/93aa52100fcdaf3de2cfa568a3eb12ae00cfef82_93aa52100fcdaf3de2cfa568a3eb12ae00cfef82/report/index.html

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 19, 2024
@yiguolei yiguolei merged commit 1835a8c into apache:master Mar 20, 2024
@w41ter w41ter deleted the fix_download_snapshot_crash branch March 20, 2024 01:56
@xiaokang xiaokang added the p0_c label Mar 20, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Mar 20, 2024
…pache#32489)

The standard said that the input parameter `pos` of std::vector::erase
must be valid and dereferenceable, the `end()` iterator cannot be used
as a value of `pos`. I did some tests and the crash only occurs when the
vector is empty. Fortunately `local_files` is usually not empty.
yiguolei pushed a commit that referenced this pull request Mar 21, 2024
…32489)

The standard said that the input parameter `pos` of std::vector::erase
must be valid and dereferenceable, the `end()` iterator cannot be used
as a value of `pos`. I did some tests and the crash only occurs when the
vector is empty. Fortunately `local_files` is usually not empty.
yiguolei pushed a commit that referenced this pull request Mar 21, 2024
…32489)

The standard said that the input parameter `pos` of std::vector::erase
must be valid and dereferenceable, the `end()` iterator cannot be used
as a value of `pos`. I did some tests and the crash only occurs when the
vector is empty. Fortunately `local_files` is usually not empty.
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Mar 21, 2024
…pache#32489)

The standard said that the input parameter `pos` of std::vector::erase
must be valid and dereferenceable, the `end()` iterator cannot be used
as a value of `pos`. I did some tests and the crash only occurs when the
vector is empty. Fortunately `local_files` is usually not empty.
@yiguolei yiguolei mentioned this pull request Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.7-merged p0_c reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants