Skip to content

Conversation

@xy720
Copy link
Member

@xy720 xy720 commented Feb 22, 2023

Proposed changes

This commit fix two problems:

1、fix insert null for struct-type

how to reproduce:

mysql> desc struct_test;
+-------+-----------------------------------+------+-------+---------+-------+
| Field | Type                              | Null | Key   | Default | Extra |
+-------+-----------------------------------+------+-------+---------+-------+
| k1    | INT                               | Yes  | true  | NULL    |       |
| k2    | STRUCT<f1:int(11),f2:varchar(50)> | Yes  | false | NULL    | NONE  |
+-------+-----------------------------------+------+-------+---------+-------+
2 rows in set (0.01 sec)

mysql> insert into struct_test values(1, {1, 'a'}), (2, null);
Query OK, 2 rows affected (0.15 sec)

mysql> select * from struct_test;
+------+--------------+
| k1   | k2           |
+------+--------------+
|    1 | {1, 'a'}     |
|    2 | {NULL, NULL} |
+------+--------------+
2 rows in set (0.03 sec)

The correct result should be:

+------+--------------+
| k1   | k2           |
+------+--------------+
|    1 | {1, 'a'}     |
|    2 | NULL |
+------+--------------+

2、fix is null predicate in delete stmt for array

how to reproduce:

mysql> desc array_test_tb;
+-------+--------------------+------+-------+---------+-------+
| Field | Type               | Null | Key   | Default | Extra |
+-------+--------------------+------+-------+---------+-------+
| k1    | INT                | Yes  | true  | NULL    |       |
| k2    | ARRAY<VARCHAR(50)> | Yes  | false | NULL    | NONE  |
+-------+--------------------+------+-------+---------+-------+

mysql> insert into array_test_tb values(1, ['a', 'b']), (2, null);
Query OK, 2 rows affected (0.03 sec)

mysql> delete from array_test_tb where k2 is null;
Query OK, 0 rows affected (0.03 sec)

mysql> select * from array_test_tb;
ERROR 1105 (HY000): RpcException, msg: org.apache.doris.rpc.RpcException: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason

There are same problems in struct and map type.

The core dump is like:

*** Query id: b37fbaa5102d41a9-8001aa8823c228b3 ***
*** Aborted at 1677027639 (unix time) try "date -d @1677027639" if you are using GNU date ***
*** Current BE git commitID: df9a0baf3 ***
*** SIGABRT unkown detail explain (@0x1fb00015678) received by PID 87672 (TID 88167 OR 0x7f84bb5ee700) from PID 87672; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/common/signal_handler.h:428
 1# 0x00007F860B8517E0 in /lib64/libpthread.so.0
 2# gsignal in /lib64/libc.so.6
 3# __GI_abort in /lib64/libc.so.6
 4# 0x00007F86166D1F99 in /home/disk4/xuyang/work/be/lib/doris_be
 5# 0x00007F86166C75AD at src/logging.cc:1650
 6# google::LogMessage::SendToLog() at src/logging.cc:1607
 7# google::LogMessage::Flush() at src/logging.cc:1477
 8# google::LogMessageFatal::~LogMessageFatal() at src/logging.cc:2227
 9# doris::vectorized::IColumn::filter_by_selector(unsigned short const*, unsigned long, doris::vectorized::IColumn*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/columns/column.h:402
10# doris::vectorized::ColumnNullable::filter_by_selector(unsigned short const*, unsigned long, doris::vectorized::IColumn*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/columns/column_nullable.cpp:323
11# doris::vectorized::ColumnArray::filter_by_selector(unsigned short const*, unsigned long, doris::vectorized::IColumn*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/columns/column_array.cpp:720
12# doris::vectorized::ColumnNullable::filter_by_selector(unsigned short const*, unsigned long, doris::vectorized::IColumn*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/columns/column_nullable.cpp:323
13# doris::Status doris::segment_v2::SegmentIterator::_output_column_by_sel_idx<std::vector<unsigned int, std::allocator<unsigned int> > >(doris::vectorized::Block*, std::vector<unsigned int, std::allocator<unsigned int> > const&, unsigned short*, unsigned short) at /home
/disk4/xuyang/work/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/segment_iterator.h:205
14# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1600
15# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/olap/rowset/beta_rowset_reader.cpp:239
16# doris::vectorized::VCollectIterator::Level0Iterator::_refresh_current_row() at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:464
17# doris::vectorized::VCollectIterator::Level0Iterator::init_for_union(bool, bool) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:438
18# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > >&) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/olap/vcollect_iterator.cpp:163
19# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/olap/block_reader.cpp:94
20# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/olap/block_reader.cpp:160
21# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/exec/scan/new_olap_scanner.cpp:160
22# doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, doris::vectorized::VScanner*) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/vec/exec/scan/scanner_scheduler.cpp:264
23# doris::PriorityWorkStealingThreadPool::work_thread(int) at /home/disk4/xuyang/work/baidu/bdg/doris/core/be/src/util/priority_work_stealing_thread_pool.hpp:137
24# execute_native_thread_routine in /home/disk4/xuyang/work/be/lib/doris_be
25# start_thread in /lib64/libpthread.so.0
26# __clone in /lib64/libc.so.6

The reason is that the predicate column should be a child in array/struct/map column.

Problem summary

Describe your changes.

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@xy720 xy720 added area/array-type Issues or PRs related to array type struct-type Issues or PRs related to struct type map-type Issues or PRs related to map type labels Feb 22, 2023
@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

cambyzju
cambyzju previously approved these changes Feb 22, 2023
Copy link
Contributor

@cambyzju cambyzju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xy720
Copy link
Member Author

xy720 commented Feb 22, 2023

run buildall

@xy720
Copy link
Member Author

xy720 commented Feb 22, 2023

run p0

1 similar comment
@xy720
Copy link
Member Author

xy720 commented Feb 22, 2023

run p0

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@xy720
Copy link
Member Author

xy720 commented Feb 23, 2023

run buildall

@xy720
Copy link
Member Author

xy720 commented Feb 23, 2023

run p0

Copy link
Contributor

@cambyzju cambyzju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cambyzju cambyzju merged commit 91fc9fa into apache:master Feb 23, 2023
@xy720 xy720 mentioned this pull request Feb 24, 2023
88 tasks
yagagagaga pushed a commit to yagagagaga/doris that referenced this pull request Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/array-type Issues or PRs related to array type area/vectorization kind/test map-type Issues or PRs related to map type struct-type Issues or PRs related to struct type

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants