[fix] [admin] Fix get topic stats fail if a subscription catch up concurrently #20971
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
poorbarcode
added
release/3.0.2
release/2.11.3
release/2.10.6
type/bug
poorbarcode
requested review from
Technoboy-,
codelipenghui,
coderzc and
congbobo184
Technoboy-
approved these changes
Aug 11, 2023
poorbarcode
force-pushed
the
fix/topic_stats_ealiest_msg
branch
from
f80ff40 to
a81ec2f
Compare
Contributor
Author
|
rebase master |
coderzc
approved these changes
Aug 14, 2023
congbobo184
approved these changes
Aug 15, 2023
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #20971 +/- ##
=============================================
+ Coverage 37.05% 73.06% +36.00%
- Complexity 12241 32361 +20120
=============================================
Files 1698 1875 +177
Lines 129816 140982 +11166
Branches 14157 15730 +1573
=============================================
+ Hits 48109 103009 +54900
+ Misses 75377 29821 -45556
- Partials 6330 8152 +1822
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Technoboy-
pushed a commit
that referenced
this pull request
Aug 21, 2023
…currently (#20971) ### Motivation **Background**: when calling `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>`, Pulsar will read the first entry which is not acknowledged, and respond with the entry write time. The flow is like this: - get the mark deleted position of the subscription - if no backlog, response `-1` - else read the next position of the mark deleted position, and respond with the entry write time. **Issue**: if the command `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` and `consumer.acknowledge` are executed at the same time, the step 2 in above flow will get a position which is larger than the last confirmed position, lead a read entry error. | time | `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` | `consumer.acknowledge` | | --- | --- | --- | | 1 | mark deleted position is `3:1` and LAC is `3:2` now | | 2 | the check `whether has backlog` is passed | | 3 | | acknowledged `3:2`, mark deleted position is `3:2` now | | 4 | calculate next position: `3:3` | | 5 | Read `3:3` and get an error: `read entry failed` | Note: the test in PR is not intended to reproduce the issue. ### Modifications Respond `-1` if the next position of the mark deleted position is larger than the LAC
Technoboy-
pushed a commit
that referenced
this pull request
Aug 21, 2023
…currently (#20971) ### Motivation **Background**: when calling `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>`, Pulsar will read the first entry which is not acknowledged, and respond with the entry write time. The flow is like this: - get the mark deleted position of the subscription - if no backlog, response `-1` - else read the next position of the mark deleted position, and respond with the entry write time. **Issue**: if the command `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` and `consumer.acknowledge` are executed at the same time, the step 2 in above flow will get a position which is larger than the last confirmed position, lead a read entry error. | time | `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` | `consumer.acknowledge` | | --- | --- | --- | | 1 | mark deleted position is `3:1` and LAC is `3:2` now | | 2 | the check `whether has backlog` is passed | | 3 | | acknowledged `3:2`, mark deleted position is `3:2` now | | 4 | calculate next position: `3:3` | | 5 | Read `3:3` and get an error: `read entry failed` | Note: the test in PR is not intended to reproduce the issue. ### Modifications Respond `-1` if the next position of the mark deleted position is larger than the LAC
shibd
pushed a commit
to shibd/pulsar
that referenced
this pull request
Oct 22, 2023
…currently (apache#20971) **Background**: when calling `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>`, Pulsar will read the first entry which is not acknowledged, and respond with the entry write time. The flow is like this: - get the mark deleted position of the subscription - if no backlog, response `-1` - else read the next position of the mark deleted position, and respond with the entry write time. **Issue**: if the command `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` and `consumer.acknowledge` are executed at the same time, the step 2 in above flow will get a position which is larger than the last confirmed position, lead a read entry error. | time | `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` | `consumer.acknowledge` | | --- | --- | --- | | 1 | mark deleted position is `3:1` and LAC is `3:2` now | | 2 | the check `whether has backlog` is passed | | 3 | | acknowledged `3:2`, mark deleted position is `3:2` now | | 4 | calculate next position: `3:3` | | 5 | Read `3:3` and get an error: `read entry failed` | Note: the test in PR is not intended to reproduce the issue. Respond `-1` if the next position of the mark deleted position is larger than the LAC (cherry picked from commit 7c96a36)
shibd
pushed a commit
to shibd/pulsar
that referenced
this pull request
Oct 24, 2023
…currently (apache#20971) **Background**: when calling `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>`, Pulsar will read the first entry which is not acknowledged, and respond with the entry write time. The flow is like this: - get the mark deleted position of the subscription - if no backlog, response `-1` - else read the next position of the mark deleted position, and respond with the entry write time. **Issue**: if the command `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` and `consumer.acknowledge` are executed at the same time, the step 2 in above flow will get a position which is larger than the last confirmed position, lead a read entry error. | time | `pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>` | `consumer.acknowledge` | | --- | --- | --- | | 1 | mark deleted position is `3:1` and LAC is `3:2` now | | 2 | the check `whether has backlog` is passed | | 3 | | acknowledged `3:2`, mark deleted position is `3:2` now | | 4 | calculate next position: `3:3` | | 5 | Read `3:3` and get an error: `read entry failed` | Note: the test in PR is not intended to reproduce the issue. Respond `-1` if the next position of the mark deleted position is larger than the LAC (cherry picked from commit 7c96a36)
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Background: when calling
pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>, Pulsar will read the first entry which is not acknowledged, and respond with the entry write time. The flow is like this:-1Error logs
Issue: if the command
pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>andconsumer.acknowledgeare executed at the same time, the step 2 in above flow will get a position which is larger than the last confirmed position, lead a read entry error.pulsar-admin topics stats --get-earliest-time-in-backlog <topic name>consumer.acknowledge3:1and LAC is3:2nowwhether has backlogis passed3:2, mark deleted position is3:2now3:33:3and get an error:read entry failedNote: the test in PR is not intended to reproduce the issue.
Modifications
Respond
-1if the next position of the mark deleted position is larger than the LACDocumentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: x