[feature](sink) support paritition tablet sink shuffle #30914
Conversation
|
Thank you for your contribution to Apache Doris. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| } | ||
|
|
||
| template <typename Channels> | ||
| Status VDataStreamSender::channel_add_rows_with_idx( |
There was a problem hiding this comment.
warning: function 'channel_add_rows_with_idx' has cognitive complexity of 52 (threshold 50) [readability-function-cognitive-complexity]
Status VDataStreamSender::channel_add_rows_with_idx(
^Additional context
be/src/vec/sink/vdata_stream_sender.h:435: +1, including nesting penalty of 0, nesting level increased to 1
for (int i = 0; i < num_channels; ++i) {
^be/src/vec/sink/vdata_stream_sender.h:436: +2, including nesting penalty of 1, nesting level increased to 2
if (!channels[i]->is_receiver_eof() && !channel2rows[i].empty()) {
^be/src/vec/sink/vdata_stream_sender.h:436: +1
if (!channels[i]->is_receiver_eof() && !channel2rows[i].empty()) {
^be/src/vec/sink/vdata_stream_sender.h:438: +3, including nesting penalty of 2, nesting level increased to 3
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:407: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.h:438: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:408: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.h:438: +1, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:410: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.h:438: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.h:438: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.h:442: +1, including nesting penalty of 0, nesting level increased to 1
if (eos) {
^be/src/vec/sink/vdata_stream_sender.h:443: +2, including nesting penalty of 1, nesting level increased to 2
for (int i = 0; i < num_channels; ++i) {
^be/src/vec/sink/vdata_stream_sender.h:444: +3, including nesting penalty of 2, nesting level increased to 3
if (!channels[i]->is_receiver_eof()) {
^be/src/vec/sink/vdata_stream_sender.h:446: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:407: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.h:446: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:408: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.h:446: +1, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:410: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.h:446: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.h:446: +7, including nesting penalty of 6, nesting level increased to 7
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^
| } | ||
|
|
||
| template <typename Channels> | ||
| Status VDataStreamSender::channel_add_rows_with_idx( |
There was a problem hiding this comment.
warning: function 'channel_add_rows_with_idx' has cognitive complexity of 52 (threshold 50) [readability-function-cognitive-complexity]
Status VDataStreamSender::channel_add_rows_with_idx(
^Additional context
be/src/vec/sink/vdata_stream_sender.h:436: +1, including nesting penalty of 0, nesting level increased to 1
for (int i = 0; i < num_channels; ++i) {
^be/src/vec/sink/vdata_stream_sender.h:437: +2, including nesting penalty of 1, nesting level increased to 2
if (!channels[i]->is_receiver_eof() && !channel2rows[i].empty()) {
^be/src/vec/sink/vdata_stream_sender.h:437: +1
if (!channels[i]->is_receiver_eof() && !channel2rows[i].empty()) {
^be/src/vec/sink/vdata_stream_sender.h:439: +3, including nesting penalty of 2, nesting level increased to 3
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:408: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.h:439: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:409: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.h:439: +1, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.h:439: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:412: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.h:439: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:412: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.h:443: +1, including nesting penalty of 0, nesting level increased to 1
if (eos) {
^be/src/vec/sink/vdata_stream_sender.h:444: +2, including nesting penalty of 1, nesting level increased to 2
for (int i = 0; i < num_channels; ++i) {
^be/src/vec/sink/vdata_stream_sender.h:445: +3, including nesting penalty of 2, nesting level increased to 3
if (!channels[i]->is_receiver_eof()) {
^be/src/vec/sink/vdata_stream_sender.h:447: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:408: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.h:447: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:409: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.h:447: +1, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:411: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.h:447: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:412: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.h:447: +7, including nesting penalty of 6, nesting level increased to 7
HANDLE_CHANNEL_STATUS(state, channels[i], status);
^be/src/vec/sink/vdata_stream_sender.h:412: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^
zhangstar333
force-pushed
the
partition_tablet_shuffle
branch
from
2827800 to
cf88168
Compare
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41902 ms |
TPC-DS: Total hot run time: 177078 ms |
ClickBench: Total hot run time: 30.25 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
run buildall |
|
run buildall |
|
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41923 ms |
TPC-DS: Total hot run time: 185865 ms |
ClickBench: Total hot run time: 31.42 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
run buildall |
TPC-H: Total hot run time: 41592 ms |
|
TeamCity be ut coverage result: |
|
run buildall |
| return Status::OK(); | ||
| } | ||
|
|
||
| Status VDataStreamSender::send(RuntimeState* state, Block* block, bool eos) { |
There was a problem hiding this comment.
warning: function 'send' has cognitive complexity of 184 (threshold 50) [readability-function-cognitive-complexity]
Status VDataStreamSender::send(RuntimeState* state, Block* block, bool eos) {
^Additional context
be/src/vec/sink/vdata_stream_sender.cpp:574: +1, including nesting penalty of 0, nesting level increased to 1
if (all_receiver_eof) {
^be/src/vec/sink/vdata_stream_sender.cpp:578: +1, including nesting penalty of 0, nesting level increased to 1
if (_part_type == TPartitionType::UNPARTITIONED || _channels.size() == 1) {
^be/src/vec/sink/vdata_stream_sender.cpp:582: +2, including nesting penalty of 1, nesting level increased to 2
if (_only_local_exchange) {
^be/src/vec/sink/vdata_stream_sender.cpp:583: +3, including nesting penalty of 2, nesting level increased to 3
if (!block->empty()) {
^be/src/vec/sink/vdata_stream_sender.cpp:592: +1, nesting level increased to 2
} else if (_enable_pipeline_exec) {
^be/src/vec/sink/vdata_stream_sender.cpp:594: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_get_next_available_buffer(&block_holder));
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:594: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_get_next_available_buffer(&block_holder));
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:598: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_serializer.next_serialized_block(
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:598: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_serializer.next_serialized_block(
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:600: +3, including nesting penalty of 2, nesting level increased to 3
if (serialized) {
^be/src/vec/sink/vdata_stream_sender.cpp:602: +4, including nesting penalty of 3, nesting level increased to 4
if (!cur_block.empty()) {
^be/src/vec/sink/vdata_stream_sender.cpp:603: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(_serializer.serialize_block(
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:603: +6, including nesting penalty of 5, nesting level increased to 6
RETURN_IF_ERROR(_serializer.serialize_block(
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:605: +1, nesting level increased to 4
} else {
^be/src/vec/sink/vdata_stream_sender.cpp:624: +1, nesting level increased to 2
} else {
^be/src/vec/sink/vdata_stream_sender.cpp:627: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_serializer.next_serialized_block(
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:627: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_serializer.next_serialized_block(
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:629: +3, including nesting penalty of 2, nesting level increased to 3
if (serialized) {
^be/src/vec/sink/vdata_stream_sender.cpp:631: +4, including nesting penalty of 3, nesting level increased to 4
if (!cur_block.empty()) {
^be/src/vec/sink/vdata_stream_sender.cpp:632: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(_serializer.serialize_block(&cur_block, _cur_pb_block,
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:632: +6, including nesting penalty of 5, nesting level increased to 6
RETURN_IF_ERROR(_serializer.serialize_block(&cur_block, _cur_pb_block,
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:652: +1, nesting level increased to 1
} else if (_part_type == TPartitionType::RANDOM) {
^be/src/vec/sink/vdata_stream_sender.cpp:655: +2, including nesting penalty of 1, nesting level increased to 2
if (!current_channel->is_receiver_eof()) {
^be/src/vec/sink/vdata_stream_sender.cpp:657: +3, including nesting penalty of 2, nesting level increased to 3
if (current_channel->is_local()) {
^be/src/vec/sink/vdata_stream_sender.cpp:659: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:419: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:659: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:420: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.cpp:659: +1, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:422: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.cpp:659: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:423: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:659: +7, including nesting penalty of 6, nesting level increased to 7
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:423: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:660: +1, nesting level increased to 3
} else {
^be/src/vec/sink/vdata_stream_sender.cpp:662: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:662: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:666: +4, including nesting penalty of 3, nesting level increased to 4
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:419: expanded from macro 'HANDLE_CHANNEL_STATUS'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:666: +5, including nesting penalty of 4, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:420: expanded from macro 'HANDLE_CHANNEL_STATUS'
if (status.is<ErrorCode::END_OF_FILE>()) { \
^be/src/vec/sink/vdata_stream_sender.cpp:666: +1, nesting level increased to 5
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:422: expanded from macro 'HANDLE_CHANNEL_STATUS'
} else { \
^be/src/vec/sink/vdata_stream_sender.cpp:666: +6, including nesting penalty of 5, nesting level increased to 6
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:423: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:666: +7, including nesting penalty of 6, nesting level increased to 7
HANDLE_CHANNEL_STATUS(state, current_channel, status);
^be/src/vec/sink/vdata_stream_sender.h:423: expanded from macro 'HANDLE_CHANNEL_STATUS'
RETURN_IF_ERROR(status); \
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:671: +1, nesting level increased to 1
} else if (_part_type == TPartitionType::HASH_PARTITIONED ||
^be/src/vec/sink/vdata_stream_sender.cpp:676: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_partitioner->do_partitioning(state, block, _mem_tracker.get()));
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:676: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_partitioner->do_partitioning(state, block, _mem_tracker.get()));
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:678: +2, including nesting penalty of 1, nesting level increased to 2
if (_part_type == TPartitionType::HASH_PARTITIONED) {
^be/src/vec/sink/vdata_stream_sender.cpp:679: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(channel_add_rows(state, _channels, _partition_count,
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:679: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(channel_add_rows(state, _channels, _partition_count,
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:682: +1, nesting level increased to 2
} else {
^be/src/vec/sink/vdata_stream_sender.cpp:683: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(channel_add_rows(state, _channel_shared_ptrs, _partition_count,
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:683: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(channel_add_rows(state, _channel_shared_ptrs, _partition_count,
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:687: +1, nesting level increased to 1
} else if (_part_type == TPartitionType::TABLET_SINK_SHUFFLE_PARTITIONED) {
^be/src/vec/sink/vdata_stream_sender.cpp:689: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_send_new_partition_batch());
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:689: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_send_new_partition_batch());
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:690: +2, including nesting penalty of 1, nesting level increased to 2
if (UNLIKELY(block->rows() == 0)) {
^be/src/vec/sink/vdata_stream_sender.cpp:697: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_row_distribution.generate_rows_distribution(
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:697: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_row_distribution.generate_rows_distribution(
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:706: +2, including nesting penalty of 1, nesting level increased to 2
for (int idx = 0; idx < row_ids.size(); ++idx) {
^be/src/vec/sink/vdata_stream_sender.cpp:712: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(channel_add_rows_with_idx(state, _channels, num_channels, channel2rows,
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:712: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(channel_add_rows_with_idx(state, _channels, num_channels, channel2rows,
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:715: +2, including nesting penalty of 1, nesting level increased to 2
if (eos) {
^be/src/vec/sink/vdata_stream_sender.cpp:717: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_send_new_partition_batch());
^be/src/common/status.h:541: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/sink/vdata_stream_sender.cpp:717: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_send_new_partition_batch());
^be/src/common/status.h:543: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/sink/vdata_stream_sender.cpp:719: +1, nesting level increased to 1
} else {
^be/src/vec/sink/vdata_stream_sender.cpp:728: +1, including nesting penalty of 0, nesting level increased to 1
if (eos && _enable_pipeline_exec) {
^be/src/vec/sink/vdata_stream_sender.cpp:728: +1
if (eos && _enable_pipeline_exec) {
^
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 41512 ms |
TPC-DS: Total hot run time: 176777 ms |
ClickBench: Total hot run time: 30.8 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
TeamCity be ut coverage result: |
|
run buildall |
TPC-H: Total hot run time: 41314 ms |
TPC-DS: Total hot run time: 176545 ms |
ClickBench: Total hot run time: 30.99 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
TeamCity be ut coverage result: |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
Co-authored-by: morrySnow <morrysnow@126.com>
…ange node rather than on TabletWriter when using `TABLET_SINK_SHUFFLE_PARTITIONED` (#36836) ## Proposed changes Issue Number: close #36638 #30914 add partition tablet sink shuffle and the processing of auto_increment column should be handled on exchange node raher than TabletWriter when using partition tablet sink shuffle. branch-2.1-pick: #37029
…ange node rather than on TabletWriter when using `TABLET_SINK_SHUFFLE_PARTITIONED` (#36836) ## Proposed changes Issue Number: close #36638 #30914 add partition tablet sink shuffle and the processing of auto_increment column should be handled on exchange node raher than TabletWriter when using partition tablet sink shuffle. branch-2.1-pick: #37029
4 participants
Proposed changes
this shuffle control by session variables: enable_strict_consistency_dml, and default value is false.
Co-authored-by : morrySnow @morrySnow
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...