ddl: optimize temp index worker in highly conflicting case#61445
Conversation
ti-chi-bot
Bot
added
do-not-merge/needs-triage-completed
release-note-none
|
Hi @tangenta. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #61445 +/- ##
================================================
+ Coverage 72.8617% 74.9535% +2.0918%
================================================
Files 1755 1806 +51
Lines 485565 498079 +12514
================================================
+ Hits 353791 373328 +19537
+ Misses 110115 101555 -8560
- Partials 21659 23196 +1537
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
ti-chi-bot
Bot
added
needs-cherry-pick-release-7.5
ti-chi-bot
Bot
added
lgtm
and removed
needs-1-more-lgtm
[LGTM Timeline notifier]Timeline:
|
|
/retest |
|
@tangenta: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Signed-off-by: tangenta <tangenta@126.com>
Signed-off-by: tangenta <tangenta@126.com>
| // Lock the corresponding row keys so that it doesn't modify the index KVs | ||
| // that are changing by a pessimistic transaction. | ||
| rowKey := tablecodec.EncodeRecordKey(w.table.RecordPrefix(), idxRecord.handle) | ||
| err := txn.LockKeys(context.Background(), new(kv.LockCtx), rowKey) |
There was a problem hiding this comment.
Is it risky to remove locking row key for all back fill cases?
There was a problem hiding this comment.
This PR(#39936) adds LockKeys. Is it being removed here because the related issues have been resolved?
Additionally, the PR description doesn't seem to mention removing the LockKeys.
There was a problem hiding this comment.
I see the reason has been updated in the PR description. If you can, please briefly describe the manual testing
|
/retest |
|
@tangenta: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| return nil | ||
| }) | ||
| if attempts <= 1 { | ||
| w.batchCnt = min(int(vardef.GetDDLReorgBatchSize()), w.batchCnt*2) |
There was a problem hiding this comment.
If attempts <= 1, it follows the original logic. So, can this assignment be moved before the for loop, and could the condition check be removed?"
There was a problem hiding this comment.
I prefer to put here so it is close to w.batchCnt /= 2 logic. It indicates that we will multiply by two in each iteration if we encountered a lot of conflicts previously.
There was a problem hiding this comment.
There was no -- operation in this attempt, so <=1 was the first attempt, and there was no need to revert to the original value the first time
| // Lock the corresponding row keys so that it doesn't modify the index KVs | ||
| // that are changing by a pessimistic transaction. | ||
| rowKey := tablecodec.EncodeRecordKey(w.table.RecordPrefix(), idxRecord.handle) | ||
| err := txn.LockKeys(context.Background(), new(kv.LockCtx), rowKey) |
There was a problem hiding this comment.
This PR(#39936) adds LockKeys. Is it being removed here because the related issues have been resolved?
Additionally, the PR description doesn't seem to mention removing the LockKeys.
Signed-off-by: tangenta <tangenta@126.com>
Signed-off-by: tangenta <tangenta@126.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Benjamin2037, D3Hunter, zimulala The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
In response to a cherrypick label: new pull request created to branch |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
In response to a cherrypick label: new pull request created to branch |
ti-chi-bot
Bot
added
the
needs-cherry-pick-release-8.1
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
|
In response to a cherrypick label: new pull request created to branch |
…1445) Signed-off-by: tangenta <tangenta@126.com>
6 participants
What problem does this PR solve?
Issue Number: close #61433
Problem Summary:
When there are lots of conflicts between DDL internal txn and DML txn, temp index worker can hardly make any progress.
What changed and how does it work?
LockKeysin DDL internal txn from merge index worker.tidb_ddl_temp_index_write: the counter of writing to temp index (single write).tidb_ddl_temp_index_double_write: the counter of writing to temp index (double write).tidb_ddl_temp_index_scan: the counter of scanned temp index value records.tidb_ddl_temp_index_merge: the counter of merged temp index value records.Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.