Skip to content

KAFKA-18419: KIP-891 Connect Multiversion Support (Transformation and Predicate Changes)#17742

Merged
gharris1727 merged 147 commits intoapache:trunkfrom
snehashisp:mvn-validation-transforms
Jan 6, 2025
Merged

KAFKA-18419: KIP-891 Connect Multiversion Support (Transformation and Predicate Changes)#17742
gharris1727 merged 147 commits intoapache:trunkfrom
snehashisp:mvn-validation-transforms

Conversation

@snehashisp
Copy link
Copy Markdown
Contributor

@snehashisp snehashisp commented Nov 10, 2024

The is one of a set of PRs for KIP-891. The list of total PRs given below all build one the previous one in the list. They can be reviewed individually, or if the complete set of changes is preferrable, please refer to the last PR.

This is PR#3 and contains changes to support multiple versions of transformation and predicates with appropriate defaults and validations.

  1. KAFKA-18182: KIP-891 Connect Multiversion Support (Base PR with Plugin Loading Isolation Changes) #16984 - Plugins Loading Isolation Changes
  2. KAFKA-18215: KIP-891 Connect Multiversioning Support (Configs and Validation changes for Connectors and Converters) #17741 - Versioned Configs and Validation for Connector and Converters
  3. KAFKA-18419: KIP-891 Connect Multiversion Support (Transformation and Predicate Changes) #17742 - Transformation and Predicate Support
  4. KAFKA-18863: Connect Multiversion Support (Versioned Connector Creation and related changes) #17743 - Versioned Connector Creation
  5. KAFKA-18988: Connect Multiversion Support (Updates to status and metrics) #17988 - Updates to status and metrics

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Copy link
Copy Markdown
Contributor

@gharris1727 gharris1727 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything that stands out to me right now. Once the build is fixed i think we can merge.

@snehashisp
Copy link
Copy Markdown
Contributor Author

snehashisp commented Dec 15, 2024

Hi @gharris1727. I have updated the code a bit. PTAL and LMK if you have any queries.

// The value will be null if the class couldn't be found; no point in performing follow-up validation
if (value != null) {
getConfigDefFromConfigProvidingClass(typeConfig, (Class<?>) value);
getConfigDefFromPlugin(typeConfig, ((Class<?>) value).getName(), null, plugins);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this validator needs to include the error when the version is incorrect, similar to the converters.

@snehashisp
Copy link
Copy Markdown
Contributor Author

Updated the code to include the latest suggestion. The failed test looks unrelated.

@snehashisp
Copy link
Copy Markdown
Contributor Author

Hi @gharris1727, please follow up on this review and the subsequent ones for multiversioning.

Copy link
Copy Markdown
Contributor

@gharris1727 gharris1727 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable to me.

@gharris1727 gharris1727 changed the title [WIP] KIP-891: Connect Multiversion Support (Transformation and Predicate Changes) KAFKA-18419: KIP-891 Connect Multiversion Support (Transformation and Predicate Changes) Jan 6, 2025
@gharris1727 gharris1727 merged commit ad33698 into apache:trunk Jan 6, 2025
manoj-mathivanan pushed a commit to manoj-mathivanan/kafka that referenced this pull request Feb 19, 2025
… Predicate Changes) (apache#17742)

Reviewers: Greg Harris <greg.harris@aiven.io>
gharris1727 pushed a commit that referenced this pull request Jun 5, 2025
…ion Tests (#18325)

This adds tests for [KIP-891](https://cwiki.apache.org/confluence/display/KAFKA/KIP-891%3A+Running+multiple+versions+of+Connector+plugins).
It primarily focuses on tests for the new additions in plugin loading
isolation. It has dependency on the actual KIP implementation PRs and
should be merged post #17742

Reviewers: Greg Harris <greg.harris@aiven.io>
Mirai1129 pushed a commit to Mirai1129/kafka that referenced this pull request Jun 5, 2025
…ion Tests (apache#18325)

This adds tests for [KIP-891](https://cwiki.apache.org/confluence/display/KAFKA/KIP-891%3A+Running+multiple+versions+of+Connector+plugins).
It primarily focuses on tests for the new additions in plugin loading
isolation. It has dependency on the actual KIP implementation PRs and
should be merged post apache#17742

Reviewers: Greg Harris <greg.harris@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Gradle build or GitHub Actions ci-approved clients connect

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants