Skip to content

[fix][broker] Fix race condition while updating partition number#19199

Merged
mattisonchao merged 1 commit intoapache:masterfrom
mattisonchao:fix_race_condition
Jan 12, 2023
Merged

[fix][broker] Fix race condition while updating partition number#19199
mattisonchao merged 1 commit intoapache:masterfrom
mattisonchao:fix_race_condition

Conversation

@mattisonchao
Copy link
Copy Markdown
Member

@mattisonchao mattisonchao commented Jan 12, 2023

Motivation

We use latest to copy subscriptions from another topic; there is a gap between the topic creation and subscription. In this period, If some new producer sends messages to the new topic partition. There may look like some messages are lost on the user side.

Because there is a new topic partition, I think it's Ok to change latest to earliest.

Modifications

  • Use earliest instead of latest.

Verifying this change

It's hard to add the test case.

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@mattisonchao mattisonchao added type/bug The PR fixed a bug or issue reported a bug area/admin labels Jan 12, 2023
@mattisonchao mattisonchao added this to the 2.12.0 milestone Jan 12, 2023
@mattisonchao mattisonchao self-assigned this Jan 12, 2023
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Jan 12, 2023
@mattisonchao mattisonchao reopened this Jan 12, 2023
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 12, 2023

Codecov Report

Merging #19199 (c930a84) into master (b05fddb) will increase coverage by 0.27%.
The diff coverage is 83.33%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #19199      +/-   ##
============================================
+ Coverage     45.64%   45.91%   +0.27%     
+ Complexity    11043    10406     -637     
============================================
  Files           773      713      -60     
  Lines         74463    69728    -4735     
  Branches       8018     7495     -523     
============================================
- Hits          33986    32017    -1969     
+ Misses        36687    34021    -2666     
+ Partials       3790     3690     -100     
Flag Coverage Δ
unittests 45.91% <83.33%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../apache/pulsar/broker/admin/impl/ClustersBase.java 76.68% <81.81%> (+0.02%) ⬆️
...pulsar/broker/admin/impl/PersistentTopicsBase.java 59.20% <100.00%> (-0.47%) ⬇️
...pache/pulsar/client/impl/CompactionReaderImpl.java 0.00% <0.00%> (-90.00%) ⬇️
...g/apache/pulsar/client/impl/RawBatchConverter.java 3.12% <0.00%> (-89.07%) ⬇️
.../pulsar/compaction/StrategicTwoPhaseCompactor.java 0.00% <0.00%> (-76.20%) ⬇️
.../pulsar/broker/service/SharedConsumerAssignor.java 3.70% <0.00%> (-74.08%) ⬇️
...lsar/client/impl/RawBatchMessageContainerImpl.java 0.00% <0.00%> (-73.14%) ⬇️
...lsar/broker/service/StickyKeyConsumerSelector.java 50.00% <0.00%> (-50.00%) ⬇️
...ion/buffer/metadata/TransactionBufferSnapshot.java 42.85% <0.00%> (-42.86%) ⬇️
...apache/pulsar/broker/service/EntryAndMetadata.java 0.00% <0.00%> (-40.75%) ⬇️
... and 136 more

Copy link
Copy Markdown
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cbornet you may be interested on this fix, as we talked one day about this kind of problems

@cbornet
Copy link
Copy Markdown
Contributor

cbornet commented Jan 12, 2023

LGTM

Copy link
Copy Markdown
Contributor

@HQebupt HQebupt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mattisonchao mattisonchao merged commit 4d57828 into apache:master Jan 12, 2023
@mattisonchao mattisonchao deleted the fix_race_condition branch January 12, 2023 10:12
liangyepianzhou pushed a commit that referenced this pull request Feb 10, 2023
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Feb 28, 2023
coderzc pushed a commit to coderzc/pulsar that referenced this pull request Mar 2, 2023
@coderzc coderzc added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Mar 3, 2023
Annavar-satish pushed a commit to pandio-com/pulsar that referenced this pull request Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants