KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties#8608
Merged
kkonstantine merged 2 commits intoapache:trunkfrom May 21, 2020
Merged
Conversation
Contributor
Author
|
@ryannedolan based on your experience MM2, would you be willing to take a look at this? |
ryannedolan
approved these changes
May 4, 2020
Contributor
ryannedolan
left a comment
ryannedolan
left a comment
There was a problem hiding this comment.
looks great, thanks!
Contributor
Author
|
Thanks Ryanne! @rhauch, @kkonstantine could one of you take a look at this when you have a chance? |
Contributor
|
ok to test |
Contributor
|
jdk8: single failure on known flaky test: |
kkonstantine
approved these changes
May 21, 2020
Contributor
kkonstantine
left a comment
kkonstantine
left a comment
There was a problem hiding this comment.
LGTM
Yet, to be precise, the new unit test does not reproduce the original issue. It just adds a safeguard around the fix.
kkonstantine
pushed a commit
that referenced
this pull request
May 21, 2020
…ing new properties (#8608) `MirrorTaskConfig` class mutates the `ConfigDef` by defining additional properties, which leads to a potential `ConcurrentModificationException` during worker configuration validation and unintended inclusion of those new properties in the `ConfigDef` for the connectors which in turn is then visible via the REST API's `/connectors/{name}/config/validate` endpoint. The fix here is a one-liner that just creates a copy of the `ConfigDef` before defining new properties. Reviewers: Ryanne Dolan <ryannedolan@gmail.com>, Konstantine Karantasis <konstantine@confluent.io>
kkonstantine
pushed a commit
that referenced
this pull request
May 21, 2020
…ing new properties (#8608) `MirrorTaskConfig` class mutates the `ConfigDef` by defining additional properties, which leads to a potential `ConcurrentModificationException` during worker configuration validation and unintended inclusion of those new properties in the `ConfigDef` for the connectors which in turn is then visible via the REST API's `/connectors/{name}/config/validate` endpoint. The fix here is a one-liner that just creates a copy of the `ConfigDef` before defining new properties. Reviewers: Ryanne Dolan <ryannedolan@gmail.com>, Konstantine Karantasis <konstantine@confluent.io>
Kvicii
pushed a commit
to Kvicii/kafka
that referenced
this pull request
May 22, 2020
* 'trunk' of github.com:apache/kafka: KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas (apache#8658) KAFKA-9780: Deprecate commit records without record metadata (apache#8379) MINOR: Deploy VerifiableClient in constructor to avoid test timeouts (apache#8651) MINOR: Added unit tests for ConnectionQuotas (apache#8650) MINOR: Correct MirrorMaker2 integration test configs for Connect internal topics (apache#8653) KAFKA-9855 - return cached Structs for Schemas with no fields (apache#8472) KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties (apache#8608) KAFKA-8869: Remove task configs for deleted connectors from config snapshot (apache#8444) KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect (apache#7942)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira
MM2 is currently sharing the same
ConfigDefobject for all its connectors and tasks, which would be fine if that object were used as-is. However, theMirrorTaskConfigclass mutates theConfigDefby defining additional properties, which leads to a potentialConcurrentModificationExceptionduring worker configuration validation and unintended inclusion of those new properties in theConfigDeffor the connectors which in turn is then visible via the REST API's/connectors/{name}/config/validateendpoint.The fix here is a one-liner that just creates a copy of the
ConfigDefbefore defining new properties.A unit test is added that fails without this fix and passes with it.
Committer Checklist (excluded from commit message)