Skip to content

branch-2.1: [fix](ip) fix datatype serde for ipv6 with rowstore #43252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Nov 5, 2024
Merged

branch-2.1: [fix](ip) fix datatype serde for ipv6 with rowstore #43252

yiguolei merged 1 commit into from
Nov 5, 2024

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Nov 5, 2024

Cherry-picked from #43065

@doris-robot
Copy link

doris-robot commented Nov 5, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@doris-robot
Copy link

doris-robot commented Nov 5, 2024

run buildall

@doris-robot
Copy link

doris-robot commented Nov 5, 2024

TeamCity be ut coverage result:
Function Coverage: 36.31% (9462/26061)
Line Coverage: 27.85% (77900/279681)
Region Coverage: 26.53% (39986/150719)
Branch Coverage: 23.31% (20308/87114)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a6e61eb988e3b7dc4742e5d693ae7b03dbf6c3c3_a6e61eb988e3b7dc4742e5d693ae7b03dbf6c3c3/report/index.html

@yiguolei
Copy link
Contributor

yiguolei commented Nov 5, 2024

skip buildall

@yiguolei
[fix](ip) fix datatype serde for ipv6 with rowstore (#43065)
91d0913 
before this pr: 
if we has ipv6 column in table which is support store_row_column 
we insert some data
and then make sql with update we will meet core
```
*** Aborted at 1730367188 (unix time) try "date -d @1730367188" if you are using GNU date ***
*** Current BE git commitID: face753 ***
*** SIGSEGV invalid permissions for mapped object (@0x60c000a339ae) received by PID 4176451 (TID 4187168 OR 0x7ff774f5a700) from PID 10697134; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk1/wangqiannan/amory/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /mnt/disk1/wangqiannan/tool/jdk-17.0.10/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /mnt/disk1/wangqiannan/tool/jdk-17.0.10/lib/server/libjvm.so
 3# 0x00007FFFF63AEB50 in /lib64/libc.so.6
 4# doris::IPv6Value::from_string(unsigned __int128&, char const*, unsigned long) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/runtime/ipv6_value.h:55
 5# bool doris::vectorized::read_ipv6_text_impl<unsigned __int128>(unsigned __int128&, doris::vectorized::ReadBuffer&) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/io/io_helper.h:316
 6# doris::vectorized::DataTypeIPv6SerDe::read_one_cell_from_jsonb(doris::vectorized::IColumn&, doris::JsonbValue const*) const at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/data_types/serde/data_type_ipv6_serde.cpp:76
 7# doris::vectorized::JsonbSerializeUtil::jsonb_to_block(std::vector<std::shared_ptr<doris::vectorized::DataTypeSerDe>, std::allocator<std::shared_ptr<doris::vectorized::DataTypeSerDe> > > const&, char const*, unsigned long, std::unordered_map<unsigned int, unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<std::pair<unsigned int const, unsigned int> > > const&, doris::vectorized::Block&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::unordered_set<int, std::hash<int>, std::equal_to<int>, std::allocator<int> > const&) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/jsonb/serialize.cpp:104
 8# doris::vectorized::JsonbSerializeUtil::jsonb_to_block(std::vector<std::shared_ptr<doris::vectorized::DataTypeSerDe>, std::allocator<std::shared_ptr<doris::vectorized::DataTypeSerDe> > > const&, doris::vectorized::ColumnStr<unsigned int> const&, std::unordered_map<unsigned int, unsigned int, std::hash<unsigned int>, std::equal_to<unsigned int>, std::allocator<std::pair<unsigned int const, unsigned int> > > const&, doris::vectorized::Block&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::unordered_set<int, std::hash<int>, std::equal_to<int>, std::allocator<int> > const&) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/jsonb/serialize.cpp:83
 9# doris::BaseTablet::fetch_value_through_row_column(std::shared_ptr<doris::Rowset>, doris::TabletSchema const&, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> > const&, std::vector<unsigned int, std::allocator<unsigned int> > const&, doris::vectorized::Block&) at /mnt/disk1/wangqiannan/amory/doris/be/src/olap/base_tablet.cpp:885
10# doris::FixedReadPlan::read_columns_by_plan(doris::TabletSchema const&, std::vector<unsigned int, std::allocator<unsigned int> >, std::map<doris::RowsetId, std::shared_ptr<doris::Rowset>, std::less<doris::RowsetId>, std::allocator<std::pair<doris::RowsetId const, std::shared_ptr<doris::Rowset> > > > const&, doris::vectorized::Block&, std::map<unsigned int, unsigned int, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, unsigned int> > >*, signed char const*) const at /mnt/disk1/wangqiannan/amory/doris/be/src/olap/partial_update_info.cpp:295
11# doris::FixedReadPlan::fill_missing_columns(doris::RowsetWriterContext*, std::map<doris::RowsetId, std::shared_ptr<doris::Rowset>, std::less<doris::RowsetId>, std::allocator<std::pair<doris::RowsetId const, std::shared_ptr<doris::Rowset> > > > const&, doris::TabletSchema const&, doris::vectorized::Block&, std::vector<bool, std::allocator<bool> > const&, bool, unsigned long const&, doris::vectorized::Block const*) const at /mnt/disk1/wangqiannan/amory/doris/be/src/olap/partial_update_info.cpp:332
12# doris::segment_v2::VerticalSegmentWriter::_append_block_with_partial_content(doris::segment_v2::RowsInBlock&, doris::vectorized::Block&) at /mnt/disk1/wangqiannan/amory/doris/be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:502
```
@yiguolei yiguolei force-pushed the auto-pick-43065-branch-2.1 branch from a6e61eb to 91d0913 Compare November 5, 2024 12:04
@yiguolei
Copy link
Contributor

yiguolei commented Nov 5, 2024

skip buildall

@yiguolei yiguolei merged commit 46afbfc into branch-2.1 Nov 5, 2024
github-actions[bot]
github-actions bot commented Nov 5, 2024
View reviewed changes
Copy link
Contributor Author

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/test/vec/data_types/serde/data_type_serde_test.cpp
std::string ip = "5be8:dde9:7f0b:d5a7:bd01:b3be:9c69:573b";
auto vec = vectorized::ColumnVector<IPv6>::create();
IPv6Value ipv6;
EXPECT_TRUE(ipv6.from_string(ip));
Copy link
Contributor Author

@github-actions github-actions bot Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: static member accessed through instance [readability-static-accessed-through-instance]

Suggested change
EXPECT_TRUE(ipv6.from_string(ip));
EXPECT_TRUE(std::doris::IPv6Value::from_string(ip));

be/test/vec/data_types/serde/data_type_serde_test.cpp
jsonb_column->insert_data(jsonb_writer.getOutput()->getBuffer(),
jsonb_writer.getOutput()->getSize());
StringRef jsonb_data = jsonb_column->get_data_at(0);
auto pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);
Copy link
Contributor Author

@github-actions github-actions bot Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'auto pdoc' can be declared as 'auto *pdoc' [readability-qualified-auto]

Suggested change
auto pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);
auto *pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);

be/test/vec/data_types/serde/data_type_serde_test.cpp
std::string ip = "192.0.0.1";
auto vec = vectorized::ColumnVector<IPv4>::create();
IPv4Value ipv4;
EXPECT_TRUE(ipv4.from_string(ip));
Copy link
Contributor Author

@github-actions github-actions bot Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: static member accessed through instance [readability-static-accessed-through-instance]

Suggested change
EXPECT_TRUE(ipv4.from_string(ip));
EXPECT_TRUE(std::doris::IPv4Value::from_string(ip));

be/test/vec/data_types/serde/data_type_serde_test.cpp
jsonb_column->insert_data(jsonb_writer.getOutput()->getBuffer(),
jsonb_writer.getOutput()->getSize());
StringRef jsonb_data = jsonb_column->get_data_at(0);
auto pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);
Copy link
Contributor Author

@github-actions github-actions bot Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'auto pdoc' can be declared as 'auto *pdoc' [readability-qualified-auto]

Suggested change
auto pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);
auto *pdoc = JsonbDocument::createDocument(jsonb_data.data, jsonb_data.size);

@dataroaring dataroaring deleted the auto-pick-43065-branch-2.1 branch December 27, 2024 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@doris-robot @yiguolei