[improve][broker] Add ref count for sticky hash to optimize the performance of Key_Shared subscription#19167
Merged
codelipenghui merged 4 commits intoapache:masterfrom Jan 11, 2023
Conversation
…rmance of Key_Shared subscription
Technoboy-
approved these changes
Jan 10, 2023
mattisonchao
approved these changes
Jan 10, 2023
Member
mattisonchao
left a comment
There was a problem hiding this comment.
Overall LGTM, It will be better to make it to be thread-safe? Because we use two maps to store the state, In the thread race condition, maybe it will introduce a non-consensus state.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #19167 +/- ##
============================================
+ Coverage 45.64% 47.22% +1.58%
+ Complexity 11043 10703 -340
============================================
Files 773 713 -60
Lines 74463 69716 -4747
Branches 8018 7492 -526
============================================
- Hits 33986 32926 -1060
+ Misses 36687 33074 -3613
+ Partials 3790 3716 -74
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
coderzc
approved these changes
Jan 10, 2023
poorbarcode
approved these changes
Jan 10, 2023
Contributor
poorbarcode
left a comment
There was a problem hiding this comment.
LGTM
I wonder if there has a better data structure to work with than ConcurrentLongLongHashMap if the operation on hashesRefCount is thread-safe.
mattisonchao
added a commit
that referenced
this pull request
Jan 16, 2023
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Feb 28, 2023
…rmance of Key_Shared subscription (apache#19167) (cherry picked from commit 7431381) (cherry picked from commit 0ac194b)
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Add ref count for sticky hash to optimize the performance of the Key_Shared subscription.
The root cause is here. We are going to go through all the redelivery messages to check if contains the hash from a given sticky hash set.
Modifications
Introduce a ref count map for the sticky hash set to make the method
containsStickyKeyHashesworks more efficiently.With this fix
Verifying this change
Updated the existing test to cover the new changes.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-complete