[Transaction] Adopt single thread pool in TC #14238
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Motivation Optimize code and improve maintainability. ### Modification * Option 1 (the way I use) Create a thread pool at peer TC. * advantage Each TC has a single thread pool to perform its own tasks, and will not be blocked due to sharing a single thread with other TCs * disadvantage Too many thread pools may be created * Option 2 Create an ExecuteProvider in the TC service. It create some single-threaded pools when the TC Service is created, and then assign a single-threaded pool to TC when the TC is created * The advantages and disadvantages are opposite to the option one
5 tasks
congbobo184
requested review from
codelipenghui,
congbobo184,
eolivelli,
gaoran10 and
merlimat
Contributor
Author
|
/pulsarbot run-failure-checks |
codelipenghui
approved these changes
Feb 24, 2022
congbobo184
approved these changes
Feb 24, 2022
codelipenghui
pushed a commit
that referenced
this pull request
Feb 25, 2022
### Motivation Optimize code and improve maintainability. ### Modification * Option 1 (the way I use) Create a thread pool at peer TC. * advantage Each TC has a single thread pool to perform its own tasks, and will not be blocked due to sharing a single thread with other TCs * disadvantage Too many thread pools may be created * Option 2 Create an ExecuteProvider in the TC service. It create some single-threaded pools when the TC Service is created, and then assign a single-threaded pool to TC when the TC is created * The advantages and disadvantages are opposite to the option one (cherry picked from commit ced5786)
gaoran10
pushed a commit
that referenced
this pull request
Mar 1, 2022
### Motivation Optimize code and improve maintainability. ### Modification * Option 1 (the way I use) Create a thread pool at peer TC. * advantage Each TC has a single thread pool to perform its own tasks, and will not be blocked due to sharing a single thread with other TCs * disadvantage Too many thread pools may be created * Option 2 Create an ExecuteProvider in the TC service. It create some single-threaded pools when the TC Service is created, and then assign a single-threaded pool to TC when the TC is created * The advantages and disadvantages are opposite to the option one (cherry picked from commit ced5786)
lhotari
added a commit
to lhotari/pulsar
that referenced
this pull request
Mar 1, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by apache#14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code.
Member
|
This PR introduced a thread leak. Please review #14524 which is a fix for that particular problem. |
lhotari
added a commit
to lhotari/pulsar
that referenced
this pull request
Mar 2, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by apache#14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code.
lhotari
added a commit
that referenced
this pull request
Mar 2, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by #14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code. * Close TransactionMetadataStoreService after the broker service has been closed
codelipenghui
pushed a commit
that referenced
this pull request
Mar 2, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by #14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code. * Close TransactionMetadataStoreService after the broker service has been closed (cherry picked from commit 0ddec86)
gaoran10
pushed a commit
that referenced
this pull request
Mar 11, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by #14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code. * Close TransactionMetadataStoreService after the broker service has been closed (cherry picked from commit 0ddec86)
Nicklee007
pushed a commit
to Nicklee007/pulsar
that referenced
this pull request
Apr 20, 2022
### Motivation Optimize code and improve maintainability. ### Modification * Option 1 (the way I use) Create a thread pool at peer TC. * advantage Each TC has a single thread pool to perform its own tasks, and will not be blocked due to sharing a single thread with other TCs * disadvantage Too many thread pools may be created * Option 2 Create an ExecuteProvider in the TC service. It create some single-threaded pools when the TC Service is created, and then assign a single-threaded pool to TC when the TC is created * The advantages and disadvantages are opposite to the option one
Nicklee007
pushed a commit
to Nicklee007/pulsar
that referenced
this pull request
Apr 20, 2022
- MLTransactionMetadataStore.internalPinnedExecutor wasn't closed when MLTransactionMetadataStore.closeAsync was called - problem was introduced by apache#14238 changes - this issue causes tests to fail with OOME. Most likely this also impacts production code. * Close TransactionMetadataStoreService after the broker service has been closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Optimize code and improve maintainability.
Modification
Create a thread pool at peer TC.
Each TC has a single thread pool to perform its own tasks, and will not be blocked due to sharing a single thread with other TCs
Too many thread pools may be created
Create an ExecuteProvider in the TC service. It create some single-threaded pools when the TC Service is created, and then assign a single-threaded pool to TC when the TC is created
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)