tables: force to lock the touched index in DML when DDL merging temp index by lcwangchao · Pull Request #62387 · pingcap/tidb

lcwangchao · 2025-07-14T03:59:19Z

What problem does this PR solve?

Issue Number: close #62337

Problem Summary:

see descriptions in issue. We choose the second way to fix it.

What changed and how does it work?

When the index is in state BackfillStateReadyToMerge or BackfillStateMerging, we also lock the touched non-unique indices to ensure the correctness, even if the DDL does not lock the row.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

…index

codecov · 2025-07-14T04:21:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.4431%. Comparing base (c765578) to head (6142d72).
Report is 7 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #62387        +/-   ##
================================================
+ Coverage   72.9000%   73.4431%   +0.5431%     
================================================
  Files          1754       1755         +1     
  Lines        484967     488074      +3107     
================================================
+ Hits         353541     358457      +4916     
+ Misses       109800     107982      -1818     
- Partials      21626      21635         +9

Flag	Coverage Δ
integration	`42.7042% <51.5151%> (?)`
unit	`72.1538% <100.0000%> (-0.0067%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.7804% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`46.3170% <ø> (+0.0076%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

lcwangchao · 2025-07-14T07:11:02Z

+// to make sure the serialization between the DDL and DML transactions.
+func (c *index) mayDDLMergingTempIndex() bool {
+	return c.idxInfo.BackfillState == model.BackfillStateReadyToMerge ||
+		c.idxInfo.BackfillState == model.BackfillStateMerging


@tangenta Please to review this method. It means when ddl worker enters the state BackfillStateReadyToMerge , all the DML should have seen the BackfillStateReadyToMerge or the previous state

@lcwangchao Better to add comments about the constraint ensured by DDL or the MDL mechanism

lcwangchao · 2025-07-14T07:16:46Z

/retest

ti-chi-bot · 2025-07-14T07:45:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [D3Hunter,tangenta]
~~pkg/table/OWNERS~~ [tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-07-14T07:45:13Z

[LGTM Timeline notifier]

Timeline:

2025-07-14 07:36:52.297599184 +0000 UTC m=+2503665.020778166: ☑️ agreed by tangenta.
2025-07-14 07:45:11.81643049 +0000 UTC m=+2504164.539609494: ☑️ agreed by D3Hunter.

Co-authored-by: D3Hunter <jujj603@gmail.com>

ti-chi-bot · 2025-07-14T09:30:48Z

In response to a cherrypick label: new pull request created to branch release-7.5: #62397.
But this PR has conflicts, please resolve them!

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2025-07-14T09:31:32Z

In response to a cherrypick label: new pull request created to branch release-8.1: #62398.
But this PR has conflicts, please resolve them!

ti-chi-bot · 2025-07-14T09:32:19Z

In response to a cherrypick label: new pull request created to branch release-8.5: #62400.
But this PR has conflicts, please resolve them!

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

…index (#62387) (#62400) close #62337

…index (#62387) (#62397) close #62337

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

…index (#62387) (#62398) close #62337

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

…DL merging temp index (pingcap#62387)"" This reverts commit 41508e4.

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

…DL merging temp index (pingcap#62387)"" This reverts commit 41508e4.

ti-chi-bot Bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 14, 2025

tables: force to lock the touched index in DML when DDL merging temp …

444606e

…index

lcwangchao force-pushed the force_lock_index_when_merge_tempidx branch from 4761250 to 444606e Compare July 14, 2025 04:03

lcwangchao requested review from D3Hunter, MyonKeminta, cfzjywxk, ekexium and tangenta July 14, 2025 04:03

lcwangchao commented Jul 14, 2025

View reviewed changes

tangenta approved these changes Jul 14, 2025

View reviewed changes

ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 14, 2025

D3Hunter approved these changes Jul 14, 2025

View reviewed changes

Comment thread pkg/session/test/tidb_test.go Outdated

ti-chi-bot Bot added the lgtm label Jul 14, 2025

lcwangchao requested a review from Benjamin2037 July 14, 2025 07:45

ti-chi-bot Bot removed the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jul 14, 2025

Update pkg/session/test/tidb_test.go

6142d72

Co-authored-by: D3Hunter <jujj603@gmail.com>

lcwangchao added needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. require-LGT3 Indicates that the PR requires three LGTM. labels Jul 14, 2025

Benjamin2037 added needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. labels Jul 14, 2025

ti-chi-bot Bot merged commit 95b5aa9 into pingcap:master Jul 14, 2025
24 of 27 checks passed

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 14, 2025

This is an automated cherry-pick of pingcap#62387

0770f1a

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jul 14, 2025

tables: force to lock the touched index in DML when DDL merging temp index (#62387) #62397

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 14, 2025

This is an automated cherry-pick of pingcap#62387

0728c10

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jul 14, 2025

tables: force to lock the touched index in DML when DDL merging temp index (#62387) #62398

Merged

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 14, 2025

This is an automated cherry-pick of pingcap#62387

8f7b451

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jul 14, 2025

tables: force to lock the touched index in DML when DDL merging temp index (#62387) #62400

Merged

13 tasks

lcwangchao deleted the force_lock_index_when_merge_tempidx branch July 14, 2025 09:32

ti-chi-bot Bot pushed a commit that referenced this pull request Jul 15, 2025

tables: force to lock the touched index in DML when DDL merging temp …

2dff758

…index (#62387) (#62400) close #62337

ti-chi-bot Bot pushed a commit that referenced this pull request Jul 17, 2025

tables: force to lock the touched index in DML when DDL merging temp …

90a9d63

…index (#62387) (#62397) close #62337

tangenta mentioned this pull request Jul 17, 2025

ddl: optimize temp index worker in highly conflicting case #61445

Merged

13 tasks

This was referenced Jul 18, 2025

ddl: optimize temp index worker in highly conflicting case (#61445) #62491

Merged

ddl: optimize temp index worker in highly conflicting case (#61445) #62492

Merged

ddl: optimize temp index worker in highly conflicting case (#61445) #62495

Open

wjhuang2016 added a commit to wjhuang2016/tidb that referenced this pull request Jul 18, 2025

Revert "tables: force to lock the touched index in DML when DDL mergi…

f5807b5

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

wjhuang2016 mentioned this pull request Jul 18, 2025

for build #62504

Closed

13 tasks

ti-chi-bot Bot pushed a commit that referenced this pull request Jul 21, 2025

tables: force to lock the touched index in DML when DDL merging temp …

b2bb841

…index (#62387) (#62398) close #62337

tangenta added a commit to tangenta/tidb that referenced this pull request Jul 24, 2025

Revert "tables: force to lock the touched index in DML when DDL mergi…

41508e4

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

tangenta added a commit to tangenta/tidb that referenced this pull request Jul 28, 2025

Revert "Revert "tables: force to lock the touched index in DML when D…

de00f78

…DL merging temp index (pingcap#62387)"" This reverts commit 41508e4.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 30, 2025

Revert "tables: force to lock the touched index in DML when DDL mergi…

497101d

…ng temp index (pingcap#62387)" This reverts commit 95b5aa9.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 30, 2025

Revert "Revert "tables: force to lock the touched index in DML when D…

0eb9901

…DL merging temp index (pingcap#62387)"" This reverts commit 41508e4.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tables: force to lock the touched index in DML when DDL merging temp index#62387

tables: force to lock the touched index in DML when DDL merging temp index#62387
ti-chi-bot[bot] merged 2 commits into
pingcap:masterfrom
lcwangchao:force_lock_index_when_merge_tempidx

lcwangchao commented Jul 14, 2025

Uh oh!

codecov Bot commented Jul 14, 2025 •

edited

Loading

Uh oh!

lcwangchao Jul 14, 2025 •

edited

Loading

Uh oh!

tangenta Jul 14, 2025

Uh oh!

cfzjywxk Jul 14, 2025

Uh oh!

lcwangchao commented Jul 14, 2025

Uh oh!

Uh oh!

ti-chi-bot Bot commented Jul 14, 2025

Uh oh!

ti-chi-bot Bot commented Jul 14, 2025

Uh oh!

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

lcwangchao commented Jul 14, 2025

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Uh oh!

codecov Bot commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lcwangchao Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tangenta Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

cfzjywxk Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

lcwangchao commented Jul 14, 2025

Uh oh!

Uh oh!

ti-chi-bot Bot commented Jul 14, 2025

Uh oh!

ti-chi-bot Bot commented Jul 14, 2025

[LGTM Timeline notifier]

Uh oh!

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

ti-chi-bot commented Jul 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov Bot commented Jul 14, 2025 •

edited

Loading

lcwangchao Jul 14, 2025 •

edited

Loading