Skip to content

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Oct 16, 2024

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return Status::OK();
}

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'update_delete_bitmap' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
                   ^
Additional context

be/src/olap/base_tablet.cpp:1359: +1

            txn_info->partial_update_info && txn_info->partial_update_info->is_partial_update();
                                          ^

be/src/olap/base_tablet.cpp:1360: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1361: nesting level increased to 2

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:708: expanded from macro 'DORIS_TRY'

    ({                                           \
    ^

be/src/olap/base_tablet.cpp:1361: +3, including nesting penalty of 2, nesting level increased to 3

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:711: expanded from macro 'DORIS_TRY'

        if (!res.has_value()) [[unlikely]] {     \
        ^

be/src/olap/base_tablet.cpp:1373: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1373: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1379: +1, including nesting penalty of 0, nesting level increased to 1

        if (self->tablet_state() == TABLET_NOTREADY) {
        ^

be/src/olap/base_tablet.cpp:1384: +1, including nesting penalty of 0, nesting level increased to 1

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1384: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1399: +1, including nesting penalty of 0, nesting level increased to 1

    if (non_visible_rowsets != nullptr) {
    ^

be/src/olap/base_tablet.cpp:1411: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1424: +2, including nesting penalty of 1, nesting level increased to 2

        if (!rowsets_skip_alignment.empty()) {
        ^

be/src/olap/base_tablet.cpp:1429: +1, including nesting penalty of 0, nesting level increased to 1

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1429: +2, including nesting penalty of 1, nesting level increased to 2

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1431: +3, including nesting penalty of 2, nesting level increased to 3

        while (DebugPoints::instance()->is_enable("BaseTablet::update_delete_bitmap.block")) {
        ^

be/src/olap/base_tablet.cpp:1434: +4, including nesting penalty of 3, nesting level increased to 4

            if (block_dp) {
            ^

be/src/olap/base_tablet.cpp:1436: +5, including nesting penalty of 4, nesting level increased to 5

                if (wait_token != token) {
                ^

be/src/olap/base_tablet.cpp:1444: +1, including nesting penalty of 0, nesting level increased to 1

    if (!rowsets_skip_alignment.empty()) {
    ^

be/src/olap/base_tablet.cpp:1447: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1447: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1449: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1449: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1454: +1, including nesting penalty of 0, nesting level increased to 1

    if (segments.size() <= 1) {
    ^

be/src/olap/base_tablet.cpp:1455: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1455: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1458: +1, nesting level increased to 1

    } else {
      ^

be/src/olap/base_tablet.cpp:1460: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1460: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1463: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1463: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1471: +1, including nesting penalty of 0, nesting level increased to 1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
    ^

be/src/olap/base_tablet.cpp:1471: +1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
                                                        ^

be/src/olap/base_tablet.cpp:1476: +2, including nesting penalty of 1, nesting level increased to 2

        if (!st.ok()) {
        ^

be/src/olap/base_tablet.cpp:1481: +1, including nesting penalty of 0, nesting level increased to 1

    if (transient_rs_writer) {
    ^

be/src/olap/base_tablet.cpp:1483: +2, including nesting penalty of 1, nesting level increased to 2

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1483: +3, including nesting penalty of 2, nesting level increased to 3

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1484: +4, including nesting penalty of 3, nesting level increased to 4

            if (rand() % 100 < (100 * dp->param("percent", 0.5))) {
            ^

be/src/olap/base_tablet.cpp:1492: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1492: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1494: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1494: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1511: nesting level increased to 1

            [](size_t sum, const segment_v2::SegmentSharedPtr& s) { return sum += s->num_rows(); });
            ^

be/src/olap/base_tablet.cpp:1513: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1513: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 0d3360e to 7ef30a6 Compare October 16, 2024 03:59
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 16, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9708/25916)
Line Coverage: 28.73% (80617/280589)
Region Coverage: 28.17% (41699/148045)
Branch Coverage: 24.75% (21207/85680)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7ef30a6b67d0f29e90321d373b159892927c2262_7ef30a6b67d0f29e90321d373b159892927c2262/report/index.html

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch 3 times, most recently from f94b88e to 885bbe4 Compare October 16, 2024 10:53
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 16, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9710/25921)
Line Coverage: 28.72% (80609/280685)
Region Coverage: 28.16% (41713/148116)
Branch Coverage: 24.74% (21211/85722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/885bbe4188d8e9d3c9a01c8da95c7674d6d2505e_885bbe4188d8e9d3c9a01c8da95c7674d6d2505e/report/index.html

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 17, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.47% (9712/25921)
Line Coverage: 28.72% (80604/280677)
Region Coverage: 28.17% (41718/148115)
Branch Coverage: 24.74% (21205/85720)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0fd68d4e836ea587e41e74ba290bc06b6d65c139_0fd68d4e836ea587e41e74ba290bc06b6d65c139/report/index.html

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 18, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return status;
}

Status CloudTabletCalcDeleteBitmapTask::_handle_rowset(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_handle_rowset' exceeds recommended size/complexity thresholds [readability-function-size]

Status CloudTabletCalcDeleteBitmapTask::_handle_rowset(
                                        ^
Additional context

be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp:239: 81 lines including whitespace and comments (threshold 80)

Status CloudTabletCalcDeleteBitmapTask::_handle_rowset(
                                        ^

return Status::OK();
}

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'update_delete_bitmap' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
                   ^
Additional context

be/src/olap/base_tablet.cpp:1372: +1

            txn_info->partial_update_info && txn_info->partial_update_info->is_partial_update();
                                          ^

be/src/olap/base_tablet.cpp:1373: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1374: nesting level increased to 2

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:708: expanded from macro 'DORIS_TRY'

    ({                                           \
    ^

be/src/olap/base_tablet.cpp:1374: +3, including nesting penalty of 2, nesting level increased to 3

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:711: expanded from macro 'DORIS_TRY'

        if (!res.has_value()) [[unlikely]] {     \
        ^

be/src/olap/base_tablet.cpp:1386: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1386: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1392: +1, including nesting penalty of 0, nesting level increased to 1

        if (self->tablet_state() == TABLET_NOTREADY) {
        ^

be/src/olap/base_tablet.cpp:1397: +1, including nesting penalty of 0, nesting level increased to 1

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1397: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1412: +1, including nesting penalty of 0, nesting level increased to 1

    if (non_visible_rowsets != nullptr) {
    ^

be/src/olap/base_tablet.cpp:1424: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1437: +2, including nesting penalty of 1, nesting level increased to 2

        if (!rowsets_skip_alignment.empty()) {
        ^

be/src/olap/base_tablet.cpp:1442: +1, including nesting penalty of 0, nesting level increased to 1

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1442: +2, including nesting penalty of 1, nesting level increased to 2

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1444: +3, including nesting penalty of 2, nesting level increased to 3

        while (DebugPoints::instance()->is_enable("BaseTablet::update_delete_bitmap.block")) {
        ^

be/src/olap/base_tablet.cpp:1447: +4, including nesting penalty of 3, nesting level increased to 4

            if (block_dp) {
            ^

be/src/olap/base_tablet.cpp:1449: +5, including nesting penalty of 4, nesting level increased to 5

                if (wait_token != token) {
                ^

be/src/olap/base_tablet.cpp:1457: +1, including nesting penalty of 0, nesting level increased to 1

    if (!rowsets_skip_alignment.empty()) {
    ^

be/src/olap/base_tablet.cpp:1460: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1460: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1463: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1463: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1468: +1, including nesting penalty of 0, nesting level increased to 1

    if (segments.size() <= 1) {
    ^

be/src/olap/base_tablet.cpp:1469: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1469: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1473: +1, nesting level increased to 1

    } else {
      ^

be/src/olap/base_tablet.cpp:1475: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1475: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1478: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1478: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1486: +1, including nesting penalty of 0, nesting level increased to 1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
    ^

be/src/olap/base_tablet.cpp:1486: +1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
                                                        ^

be/src/olap/base_tablet.cpp:1491: +2, including nesting penalty of 1, nesting level increased to 2

        if (!st.ok()) {
        ^

be/src/olap/base_tablet.cpp:1496: +1, including nesting penalty of 0, nesting level increased to 1

    if (transient_rs_writer) {
    ^

be/src/olap/base_tablet.cpp:1498: +2, including nesting penalty of 1, nesting level increased to 2

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1498: +3, including nesting penalty of 2, nesting level increased to 3

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1499: +4, including nesting penalty of 3, nesting level increased to 4

            if (rand() % 100 < (100 * dp->param("percent", 0.5))) {
            ^

be/src/olap/base_tablet.cpp:1507: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1507: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1509: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1509: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1526: nesting level increased to 1

            [](size_t sum, const segment_v2::SegmentSharedPtr& s) { return sum += s->num_rows(); });
            ^

be/src/olap/base_tablet.cpp:1528: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1528: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from fea59b8 to 4817fed Compare October 18, 2024 03:22
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 18, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9710/25921)
Line Coverage: 28.73% (80653/280707)
Region Coverage: 28.17% (41729/148132)
Branch Coverage: 24.74% (21213/85728)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4817fed4ac4af5df8146920e1dd944c5614bec77_4817fed4ac4af5df8146920e1dd944c5614bec77/report/index.html

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 4817fed to 69be38d Compare October 21, 2024 02:22
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 21, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.47% (9710/25914)
Line Coverage: 28.74% (80661/280609)
Region Coverage: 28.17% (41723/148086)
Branch Coverage: 24.75% (21210/85686)
Coverage Report: http://coverage.selectdb-in.cc/coverage/69be38d179357da8b7cee819e061a7465145fa21_69be38d179357da8b7cee819e061a7465145fa21/report/index.html

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 21, 2024

run p0

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 21, 2024

run cloud_p0

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 69be38d to 0a8d05a Compare October 25, 2024 07:27
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 25, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return Status::OK();
}

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'update_delete_bitmap' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
                   ^
Additional context

be/src/olap/base_tablet.cpp:1372: +1

            txn_info->partial_update_info && txn_info->partial_update_info->is_partial_update();
                                          ^

be/src/olap/base_tablet.cpp:1373: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1374: nesting level increased to 2

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:708: expanded from macro 'DORIS_TRY'

    ({                                           \
    ^

be/src/olap/base_tablet.cpp:1374: +3, including nesting penalty of 2, nesting level increased to 3

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:711: expanded from macro 'DORIS_TRY'

        if (!res.has_value()) [[unlikely]] {     \
        ^

be/src/olap/base_tablet.cpp:1386: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1386: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1392: +1, including nesting penalty of 0, nesting level increased to 1

        if (self->tablet_state() == TABLET_NOTREADY) {
        ^

be/src/olap/base_tablet.cpp:1397: +1, including nesting penalty of 0, nesting level increased to 1

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1397: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1412: +1, including nesting penalty of 0, nesting level increased to 1

    if (non_visible_rowsets != nullptr) {
    ^

be/src/olap/base_tablet.cpp:1417: nesting level increased to 2

                  [](RowsetSharedPtr& lhs, RowsetSharedPtr& rhs) {
                  ^

be/src/olap/base_tablet.cpp:1428: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1441: +2, including nesting penalty of 1, nesting level increased to 2

        if (!rowsets_skip_alignment.empty()) {
        ^

be/src/olap/base_tablet.cpp:1446: +1, including nesting penalty of 0, nesting level increased to 1

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1446: +2, including nesting penalty of 1, nesting level increased to 2

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1448: +3, including nesting penalty of 2, nesting level increased to 3

        while (DebugPoints::instance()->is_enable("BaseTablet::update_delete_bitmap.block")) {
        ^

be/src/olap/base_tablet.cpp:1451: +4, including nesting penalty of 3, nesting level increased to 4

            if (block_dp) {
            ^

be/src/olap/base_tablet.cpp:1453: +5, including nesting penalty of 4, nesting level increased to 5

                if (wait_token != token) {
                ^

be/src/olap/base_tablet.cpp:1461: +1, including nesting penalty of 0, nesting level increased to 1

    if (!rowsets_skip_alignment.empty()) {
    ^

be/src/olap/base_tablet.cpp:1464: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1464: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1467: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1467: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1472: +1, including nesting penalty of 0, nesting level increased to 1

    if (segments.size() <= 1) {
    ^

be/src/olap/base_tablet.cpp:1473: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1473: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1477: +1, nesting level increased to 1

    } else {
      ^

be/src/olap/base_tablet.cpp:1479: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1479: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1482: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1482: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1490: +1, including nesting penalty of 0, nesting level increased to 1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
    ^

be/src/olap/base_tablet.cpp:1490: +1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
                                                        ^

be/src/olap/base_tablet.cpp:1495: +2, including nesting penalty of 1, nesting level increased to 2

        if (!st.ok()) {
        ^

be/src/olap/base_tablet.cpp:1500: +1, including nesting penalty of 0, nesting level increased to 1

    if (transient_rs_writer) {
    ^

be/src/olap/base_tablet.cpp:1502: +2, including nesting penalty of 1, nesting level increased to 2

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1502: +3, including nesting penalty of 2, nesting level increased to 3

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1503: +4, including nesting penalty of 3, nesting level increased to 4

            if (rand() % 100 < (100 * dp->param("percent", 0.5))) {
            ^

be/src/olap/base_tablet.cpp:1511: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1511: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1513: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1513: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1530: nesting level increased to 1

            [](size_t sum, const segment_v2::SegmentSharedPtr& s) { return sum += s->num_rows(); });
            ^

be/src/olap/base_tablet.cpp:1532: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1532: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9714/25935)
Line Coverage: 28.71% (80563/280642)
Region Coverage: 28.13% (41656/148060)
Branch Coverage: 24.70% (21169/85718)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0a8d05a141846170d7dd3f5c233504cfae608120_0a8d05a141846170d7dd3f5c233504cfae608120/report/index.html

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 0a8d05a to 41ddebd Compare October 28, 2024 02:59
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Oct 28, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.46% (9717/25939)
Line Coverage: 28.72% (80592/280571)
Region Coverage: 28.16% (41680/148036)
Branch Coverage: 24.71% (21175/85690)
Coverage Report: http://coverage.selectdb-in.cc/coverage/41ddebd583d1ba8b948ebccd7eabca2f3e866405_41ddebd583d1ba8b948ebccd7eabca2f3e866405/report/index.html

public static int calculate_delete_bitmap_task_timeout_seconds = 15;

@ConfField(mutable = true, description = {"存算分离模式下事务导入calculate delete bitmap task 超时时间,默认300s"})
public static int calculate_delete_bitmap_task_timeout_seconds_for_transaction_load = 300;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using calculate_delete_bitmap_task_timeout_seconds and adjust calculate_delete_bitmap_task_timeout_seconds to a larger number.

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 4, 2024

run p0

1 similar comment
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 4, 2024

run p0

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 410bbca to 2476e2a Compare November 12, 2024 07:47
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 12, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return Status::OK();
}

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'update_delete_bitmap' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
                   ^
Additional context

be/src/olap/base_tablet.cpp:1370: +1

            txn_info->partial_update_info && txn_info->partial_update_info->is_partial_update();
                                          ^

be/src/olap/base_tablet.cpp:1371: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1372: nesting level increased to 2

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:708: expanded from macro 'DORIS_TRY'

    ({                                           \
    ^

be/src/olap/base_tablet.cpp:1372: +3, including nesting penalty of 2, nesting level increased to 3

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:711: expanded from macro 'DORIS_TRY'

        if (!res.has_value()) [[unlikely]] {     \
        ^

be/src/olap/base_tablet.cpp:1384: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1384: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1390: +1, including nesting penalty of 0, nesting level increased to 1

        if (self->tablet_state() == TABLET_NOTREADY) {
        ^

be/src/olap/base_tablet.cpp:1395: +1, including nesting penalty of 0, nesting level increased to 1

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1395: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1410: +1, including nesting penalty of 0, nesting level increased to 1

    if (invisible_rowsets != nullptr) {
    ^

be/src/olap/base_tablet.cpp:1415: nesting level increased to 2

                  [](RowsetSharedPtr& lhs, RowsetSharedPtr& rhs) {
                  ^

be/src/olap/base_tablet.cpp:1426: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1439: +2, including nesting penalty of 1, nesting level increased to 2

        if (!rowsets_skip_alignment.empty()) {
        ^

be/src/olap/base_tablet.cpp:1444: +1, including nesting penalty of 0, nesting level increased to 1

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1444: +2, including nesting penalty of 1, nesting level increased to 2

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1446: +3, including nesting penalty of 2, nesting level increased to 3

        while (DebugPoints::instance()->is_enable("BaseTablet::update_delete_bitmap.block")) {
        ^

be/src/olap/base_tablet.cpp:1449: +4, including nesting penalty of 3, nesting level increased to 4

            if (block_dp) {
            ^

be/src/olap/base_tablet.cpp:1451: +5, including nesting penalty of 4, nesting level increased to 5

                if (wait_token != token) {
                ^

be/src/olap/base_tablet.cpp:1459: +1, including nesting penalty of 0, nesting level increased to 1

    if (!rowsets_skip_alignment.empty()) {
    ^

be/src/olap/base_tablet.cpp:1462: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1462: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1465: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1465: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1470: +1, including nesting penalty of 0, nesting level increased to 1

    if (segments.size() <= 1) {
    ^

be/src/olap/base_tablet.cpp:1471: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1471: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1475: +1, nesting level increased to 1

    } else {
      ^

be/src/olap/base_tablet.cpp:1477: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1477: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1480: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1480: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1488: +1, including nesting penalty of 0, nesting level increased to 1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
    ^

be/src/olap/base_tablet.cpp:1488: +1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
                                                        ^

be/src/olap/base_tablet.cpp:1493: +2, including nesting penalty of 1, nesting level increased to 2

        if (!st.ok()) {
        ^

be/src/olap/base_tablet.cpp:1498: +1, including nesting penalty of 0, nesting level increased to 1

    if (transient_rs_writer) {
    ^

be/src/olap/base_tablet.cpp:1500: +2, including nesting penalty of 1, nesting level increased to 2

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1500: +3, including nesting penalty of 2, nesting level increased to 3

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1501: +4, including nesting penalty of 3, nesting level increased to 4

            if (rand() % 100 < (100 * dp->param("percent", 0.5))) {
            ^

be/src/olap/base_tablet.cpp:1509: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1509: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1511: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1511: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1528: nesting level increased to 1

            [](size_t sum, const segment_v2::SegmentSharedPtr& s) { return sum += s->num_rows(); });
            ^

be/src/olap/base_tablet.cpp:1530: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:629: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1530: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:631: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 2476e2a to 42580b4 Compare November 12, 2024 14:08
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 12, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return Status::OK();
}

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'update_delete_bitmap' has cognitive complexity of 86 (threshold 50) [readability-function-cognitive-complexity]

Status BaseTablet::update_delete_bitmap(const BaseTabletSPtr& self, TabletTxnInfo* txn_info,
                   ^
Additional context

be/src/olap/base_tablet.cpp:1370: +1

            txn_info->partial_update_info && txn_info->partial_update_info->is_partial_update();
                                          ^

be/src/olap/base_tablet.cpp:1371: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1372: nesting level increased to 2

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:709: expanded from macro 'DORIS_TRY'

    ({                                           \
    ^

be/src/olap/base_tablet.cpp:1372: +3, including nesting penalty of 2, nesting level increased to 3

        transient_rs_writer = DORIS_TRY(self->create_transient_rowset_writer(
                              ^

be/src/common/status.h:712: expanded from macro 'DORIS_TRY'

        if (!res.has_value()) [[unlikely]] {     \
        ^

be/src/olap/base_tablet.cpp:1384: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1384: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(std::dynamic_pointer_cast<BetaRowset>(rowset)->load_segments(&segments));
    ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1390: +1, including nesting penalty of 0, nesting level increased to 1

        if (self->tablet_state() == TABLET_NOTREADY) {
        ^

be/src/olap/base_tablet.cpp:1395: +1, including nesting penalty of 0, nesting level increased to 1

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1395: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(self->get_all_rs_id_unlocked(next_visible_version - 1, &cur_rowset_ids));
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1410: +1, including nesting penalty of 0, nesting level increased to 1

    if (invisible_rowsets != nullptr) {
    ^

be/src/olap/base_tablet.cpp:1415: nesting level increased to 2

                  [](RowsetSharedPtr& lhs, RowsetSharedPtr& rhs) {
                  ^

be/src/olap/base_tablet.cpp:1426: +1, including nesting penalty of 0, nesting level increased to 1

    if (is_partial_update) {
    ^

be/src/olap/base_tablet.cpp:1439: +2, including nesting penalty of 1, nesting level increased to 2

        if (!rowsets_skip_alignment.empty()) {
        ^

be/src/olap/base_tablet.cpp:1444: +1, including nesting penalty of 0, nesting level increased to 1

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1444: +2, including nesting penalty of 1, nesting level increased to 2

    DBUG_EXECUTE_IF("BaseTablet::update_delete_bitmap.enable_spin_wait", {
    ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1446: +3, including nesting penalty of 2, nesting level increased to 3

        while (DebugPoints::instance()->is_enable("BaseTablet::update_delete_bitmap.block")) {
        ^

be/src/olap/base_tablet.cpp:1449: +4, including nesting penalty of 3, nesting level increased to 4

            if (block_dp) {
            ^

be/src/olap/base_tablet.cpp:1451: +5, including nesting penalty of 4, nesting level increased to 5

                if (wait_token != token) {
                ^

be/src/olap/base_tablet.cpp:1459: +1, including nesting penalty of 0, nesting level increased to 1

    if (!rowsets_skip_alignment.empty()) {
    ^

be/src/olap/base_tablet.cpp:1462: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1462: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, rowsets_skip_alignment,
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1465: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1465: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1470: +1, including nesting penalty of 0, nesting level increased to 1

    if (segments.size() <= 1) {
    ^

be/src/olap/base_tablet.cpp:1471: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1471: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1475: +1, nesting level increased to 1

    } else {
      ^

be/src/olap/base_tablet.cpp:1477: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1477: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(calc_delete_bitmap(self, rowset, segments, specified_rowsets, delete_bitmap,
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1480: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1480: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(token->wait());
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1488: +1, including nesting penalty of 0, nesting level increased to 1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
    ^

be/src/olap/base_tablet.cpp:1488: +1

    if (config::enable_merge_on_write_correctness_check && rowset->num_rows() != 0) {
                                                        ^

be/src/olap/base_tablet.cpp:1493: +2, including nesting penalty of 1, nesting level increased to 2

        if (!st.ok()) {
        ^

be/src/olap/base_tablet.cpp:1498: +1, including nesting penalty of 0, nesting level increased to 1

    if (transient_rs_writer) {
    ^

be/src/olap/base_tablet.cpp:1500: +2, including nesting penalty of 1, nesting level increased to 2

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:36: expanded from macro 'DBUG_EXECUTE_IF'

    if (UNLIKELY(config::enable_debug_points)) {                              \
    ^

be/src/olap/base_tablet.cpp:1500: +3, including nesting penalty of 2, nesting level increased to 3

        DBUG_EXECUTE_IF("Tablet.update_delete_bitmap.partial_update_write_rowset_fail", {
        ^

be/src/util/debug_points.h:38: expanded from macro 'DBUG_EXECUTE_IF'

        if (dp) {                                                             \
        ^

be/src/olap/base_tablet.cpp:1501: +4, including nesting penalty of 3, nesting level increased to 4

            if (rand() % 100 < (100 * dp->param("percent", 0.5))) {
            ^

be/src/olap/base_tablet.cpp:1509: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1509: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->flush());
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1511: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1511: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(transient_rs_writer->build(transient_rowset));
        ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/olap/base_tablet.cpp:1528: nesting level increased to 1

            [](size_t sum, const segment_v2::SegmentSharedPtr& s) { return sum += s->num_rows(); });
            ^

be/src/olap/base_tablet.cpp:1530: +1, including nesting penalty of 0, nesting level increased to 1

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:630: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/olap/base_tablet.cpp:1530: +2, including nesting penalty of 1, nesting level increased to 2

    RETURN_IF_ERROR(self->save_delete_bitmap(txn_info, txn_id, delete_bitmap,
    ^

be/src/common/status.h:632: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

@dataroaring
Copy link
Contributor

run buildall

2 similar comments
@dataroaring
Copy link
Contributor

run buildall

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.00% (9903/26061)
Line Coverage: 29.17% (82728/283614)
Region Coverage: 28.30% (42495/150140)
Branch Coverage: 24.88% (21546/86616)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4553bc3375ff3b59a0b2425b988b8a9bc83d6cad_4553bc3375ff3b59a0b2425b988b8a9bc83d6cad/report/index.html

dataroaring
dataroaring previously approved these changes Nov 19, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 19, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

assertEquals(t2_row_count, result[0][0] as int)

def dbName = "regression_test_insert_p2"
def dbName = "regression_test_insert_p2_transaction"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any cases test sequence column?

@mymeiyi mymeiyi force-pushed the cloud-txn-no-update branch from 4553bc3 to b1664e2 Compare November 26, 2024 07:47
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 26, 2024
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 26, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.36% (9979/26015)
Line Coverage: 29.46% (83533/283583)
Region Coverage: 28.60% (42995/150324)
Branch Coverage: 25.19% (21833/86658)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b1664e217b4485e4d5a423690b634a76456a1521_b1664e217b4485e4d5a423690b634a76456a1521/report/index.html

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 26, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit d3714bf into apache:master Nov 26, 2024
15 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants