branch-2.1: [fix](struct)Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump #49485 #49553

github-actions · 2025-03-26T14:46:13Z

Cherry-picked from #49485

hello-stephen · 2025-03-26T14:46:55Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hello-stephen · 2025-03-26T14:46:59Z

run buildall

doris-robot · 2025-03-26T15:23:28Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	37.39% (9850/26343)
Line Coverage	29.09% (82538/283777)
Region Coverage	27.70% (42337/152860)
Branch Coverage	24.44% (21547/88164)

simges · 2025-03-26T23:17:22Z

be/src/vec/data_types/serde/data_type_struct_serde.cpp

            next.trim_prefix();
-            if (field_pos > elem_size) {
+            // field_pos should always less than elem_size, if not, we should return error
+            if (field_pos >= elem_size) {


This condition appears to be inconsistent with the comment above, as it's unclear whether field_pos could be equal to elem_size.

less than means field_pos < elem_size which is right . equal we should return err, otherwise serders[field_pos] would make crash

After taking a closer look, I noticed I misunderstood the context. All looks good now, thanks for your patience.

github-actions · 2025-03-27T12:15:26Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-03-27T12:15:29Z

PR approved by anyone and no changes requested.

…iteral with one more subfield causing BE coredump (#49485) Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump some situation like this blow will make BE core ``` create table t(a int, b int, s struct<a:int>) PROPERTIES ("replication_allocation" = "tag.location.default: 1"); insert into t values(1,1,'{1,2}'); ``` core info: ``` [WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled` start BE in local mode ================================================================= ==2818976==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000f17a90 at pc 0x55eba7fdd69c bp 0x7f03243dbd30 sp 0x7f03243dbd28 READ of size 8 at 0x603000f17a90 thread T928 (brpc_light) #0 0x55eba7fdd69b in std::__shared_ptr<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2>::get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16 #1 0x55eba7fdd649 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66 #2 0x55eba7fda316 in std::__shared_ptr_access<doris::vectorized::DataTypeSerDe, (__gnu_cxx::_Lock_policy)2, false, false>::operator->() const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9 #3 0x55ebcfc8b19e in doris::vectorized::DataTypeStructSerDe::deserialize_one_cell_from_json(doris::vectorized::IColumn&, doris::Slice&, doris::vectorized::DataTypeSerDe::FormatOptions const&) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_struct_serde.cpp:200:25 #4 0x55ebdac49c8b in doris::vectorized::ConvertImplGenericFromString::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:618:36 #5 0x55ebda2234d0 in doris::Status std::__invoke_impl<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #6 0x55ebda22323a in std::enable_if<is_invocable_r_v<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>, doris::Status>::type std::__invoke_r<doris::Status, doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(doris::Status (*&)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long), doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114:9 #7 0x55ebda222e71 in std::_Function_handler<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long), doris::Status (*)(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)>::_M_invoke(std::_Any_data const&, doris::FunctionContext*&&, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long&&, unsigned long&&) /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #8 0x55ebd9b3f1d2 in std::function<doris::Status (doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long)>::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long) const /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9 #9 0x55ebda1ad7c7 in doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)::operator()(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long) const /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_cast.h:2241:17 #10 0x55ebda1acee2 in doris::Status std::__invoke_impl<doris::Status, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bool) const::'lambda'(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned int, unsigned long)&, doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned int, std::allocator<unsigned int>> const&, unsigned long, unsigned long>(std::__invoke_other, doris::vectorized::FunctionCast::prepare_remove_nullable(doris::FunctionContext*, std::shared_ptr<doris::vectorized::IDataType const> const&, std::shared_ptr<doris::vectorized::IDataType const> const&, bo@@@ ```

yiguolei · 2025-03-29T12:23:33Z

run buildall

doris-robot · 2025-03-29T13:01:39Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	37.41% (9857/26347)
Line Coverage	29.12% (82647/283838)
Region Coverage	27.72% (42385/152897)
Branch Coverage	24.46% (21569/88184)

amorynan · 2025-04-18T08:37:43Z

run buildall

doris-robot · 2025-04-18T09:09:18Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	38.80% (10251/26421)
Line Coverage	29.85% (85105/285071)
Region Coverage	28.50% (43852/153852)
Branch Coverage	25.23% (22418/88856)

github-actions · 2025-10-16T00:13:08Z

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

github-actions bot requested a review from yiguolei as a code owner March 26, 2025 14:46

dataroaring closed this Mar 26, 2025

dataroaring reopened this Mar 26, 2025

simges reviewed Mar 27, 2025

View reviewed changes

yiguolei approved these changes Mar 27, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 27, 2025

github-actions bot added the reviewed label Mar 27, 2025

yiguolei force-pushed the auto-pick-49485-branch-2.1 branch from 48d8d03 to 3658e9e Compare March 29, 2025 12:23

github-actions bot added the Stale label Oct 16, 2025

github-actions bot closed this Oct 26, 2025

branch-2.1: [fix](struct)Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump #49485 #49553

branch-2.1: [fix](struct)Fixed the issue of inserting into a struct type string literal with one more subfield causing BE coredump #49485 #49553

Uh oh!

Conversation

github-actions bot commented Mar 26, 2025

Uh oh!

hello-stephen commented Mar 26, 2025

Uh oh!

hello-stephen commented Mar 26, 2025

Uh oh!

doris-robot commented Mar 26, 2025

BE UT Coverage Report

Uh oh!

simges Mar 26, 2025

Choose a reason for hiding this comment

Uh oh!

amorynan Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

simges Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

yiguolei commented Mar 29, 2025

Uh oh!

doris-robot commented Mar 29, 2025

BE UT Coverage Report

Uh oh!

amorynan commented Apr 18, 2025

Uh oh!

doris-robot commented Apr 18, 2025

BE UT Coverage Report

Uh oh!

github-actions bot commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants