This repository was archived by the owner on Jan 24, 2024. It is now read-only.
Fix compaction threshold not configured in metadata namespace #1273
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
Author
|
This PR relies on the changes of #1274. |
Fixes streamnative#1272 ### Motivation streamnative#448 introduced a bug that the compaction threshold was not set in metadata namespace. ```java // NOTE: compactionThreshold is always null unless it has been configured Long compactionThreshold = namespaces.getCompactionThreshold(kafkaMetadataNamespace); if (compactionThreshold != null && compactionThreshold != MAX_COMPACTION_THRESHOLD) { // it never gets here unless compaction threshold has been configured namespaces.setCompactionThreshold(kafkaMetadataNamespace, MAX_COMPACTION_THRESHOLD); } ``` In addition, currently there is no test that ensures the metadata has been initialized correctly. ### Modifications - Configure the compaction threshold when the metadata namespace is created. Since the policies could only be configured when the namespace didn't exist before, there is no need to check the existing policy. - Add `MetadataInitTest` to test the metadata namespace has been configured correctly. It also verifies the `kafkaManageSystemNamespaces` config, which disables the initialization of the metadata namespace. - Refactor the `internalSetup` method to avoid the redundant start of broker.
BewareMyPower
force-pushed
the
bewaremypower/fix-metadata-ns-compaction
branch
from
69ee493 to
52b76dd
Compare
BewareMyPower
added a commit
that referenced
this pull request
May 7, 2022
Fixes #1272 ### Motivation #448 introduced a bug that the compaction threshold was not set in metadata namespace. ```java // NOTE: compactionThreshold is always null unless it has been configured Long compactionThreshold = namespaces.getCompactionThreshold(kafkaMetadataNamespace); if (compactionThreshold != null && compactionThreshold != MAX_COMPACTION_THRESHOLD) { // it never gets here unless compaction threshold has been configured namespaces.setCompactionThreshold(kafkaMetadataNamespace, MAX_COMPACTION_THRESHOLD); } ``` In addition, currently there is no test that ensures the metadata has been initialized correctly. ### Modifications - Configure the compaction threshold when the metadata namespace is created. Since the policies could only be configured when the namespace didn't exist before, there is no need to check the existing policy. - Add `MetadataInitTest` to test the metadata namespace has been configured correctly. It also verifies the `kafkaManageSystemNamespaces` config, which disables the initialization of the metadata namespace. - Refactor the `internalSetup` method to avoid the redundant start of broker. (cherry picked from commit 82211b9)
BewareMyPower
added a commit
that referenced
this pull request
May 7, 2022
Fixes #1272 ### Motivation #448 introduced a bug that the compaction threshold was not set in metadata namespace. ```java // NOTE: compactionThreshold is always null unless it has been configured Long compactionThreshold = namespaces.getCompactionThreshold(kafkaMetadataNamespace); if (compactionThreshold != null && compactionThreshold != MAX_COMPACTION_THRESHOLD) { // it never gets here unless compaction threshold has been configured namespaces.setCompactionThreshold(kafkaMetadataNamespace, MAX_COMPACTION_THRESHOLD); } ``` In addition, currently there is no test that ensures the metadata has been initialized correctly. ### Modifications - Configure the compaction threshold when the metadata namespace is created. Since the policies could only be configured when the namespace didn't exist before, there is no need to check the existing policy. - Add `MetadataInitTest` to test the metadata namespace has been configured correctly. It also verifies the `kafkaManageSystemNamespaces` config, which disables the initialization of the metadata namespace. - Refactor the `internalSetup` method to avoid the redundant start of broker. (cherry picked from commit 82211b9)
BewareMyPower
added a commit
that referenced
this pull request
May 7, 2022
Fixes #1272 ### Motivation #448 introduced a bug that the compaction threshold was not set in metadata namespace. ```java // NOTE: compactionThreshold is always null unless it has been configured Long compactionThreshold = namespaces.getCompactionThreshold(kafkaMetadataNamespace); if (compactionThreshold != null && compactionThreshold != MAX_COMPACTION_THRESHOLD) { // it never gets here unless compaction threshold has been configured namespaces.setCompactionThreshold(kafkaMetadataNamespace, MAX_COMPACTION_THRESHOLD); } ``` In addition, currently there is no test that ensures the metadata has been initialized correctly. ### Modifications - Configure the compaction threshold when the metadata namespace is created. Since the policies could only be configured when the namespace didn't exist before, there is no need to check the existing policy. - Add `MetadataInitTest` to test the metadata namespace has been configured correctly. It also verifies the `kafkaManageSystemNamespaces` config, which disables the initialization of the metadata namespace. - Refactor the `internalSetup` method to avoid the redundant start of broker. (cherry picked from commit 82211b9)
michaeljmarshall
pushed a commit
to michaeljmarshall/kop
that referenced
this pull request
Dec 13, 2022
…native#1273) Fixes streamnative#1272 streamnative#448 introduced a bug that the compaction threshold was not set in metadata namespace. ```java // NOTE: compactionThreshold is always null unless it has been configured Long compactionThreshold = namespaces.getCompactionThreshold(kafkaMetadataNamespace); if (compactionThreshold != null && compactionThreshold != MAX_COMPACTION_THRESHOLD) { // it never gets here unless compaction threshold has been configured namespaces.setCompactionThreshold(kafkaMetadataNamespace, MAX_COMPACTION_THRESHOLD); } ``` In addition, currently there is no test that ensures the metadata has been initialized correctly. - Configure the compaction threshold when the metadata namespace is created. Since the policies could only be configured when the namespace didn't exist before, there is no need to check the existing policy. - Add `MetadataInitTest` to test the metadata namespace has been configured correctly. It also verifies the `kafkaManageSystemNamespaces` config, which disables the initialization of the metadata namespace. - Refactor the `internalSetup` method to avoid the redundant start of broker. (cherry picked from commit 82211b9)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1272
Motivation
#448 introduced a bug that the
compaction threshold was not set in metadata namespace.
In addition, currently there is no test that ensures the metadata has
been initialized correctly.
Modifications
created. Since the policies could only be configured when the
namespace didn't exist before, there is no need to check the existing
policy.
MetadataInitTestto test the metadata namespace has beenconfigured correctly. It also verifies the
kafkaManageSystemNamespacesconfig, which disables theinitialization of the metadata namespace.
internalSetupmethod to avoid the redundant start ofbroker.
Documentation
Check the box below.
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)