Add schema compatibility strategy on topic level #13297

nodece · 2021-12-14T14:18:12Z

Motivation

Currently, the compatibility strategy for a schema is set at namespace level, which means that all the topics of a namespace will need to adhere to the same compatibility check, this level particle size is relatively large, so add schema compatibility strategy on topic level.

Modifications

Add set, get and remove schema compatibility strategy API to admin API and client tool
Add schema compatibility strategy action to TopicOperation

Verifying this change

Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): no
The public API: no
The schema: no
The default values of configurations: no
The wire protocol: no
The rest endpoints: no
The admin cli options: no
Anything that affects deployment: no

Documentation

Check the box below and label this PR (if you have committer privilege).

Need to update docs?

doc-required
no-need-doc
doc

github-actions · 2021-12-14T14:50:17Z

@nodece:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

Anonymitaet · 2021-12-15T02:43:31Z

Hi @nodece please do not delete backticks (for example, doc-required ✅, doc-required ❌) or else Bot does not recognize the label, thanks.

codelipenghui · 2021-12-15T14:56:44Z

@Jason918 Could you please help review this PR? Please make sure the new change follows for enhancement way.

nodece · 2021-12-16T04:10:09Z

/pulsarbot run-failure-checks

Jason918

Please refer to org.apache.pulsar.broker.service.AbstractTopic#topicPolicies, and try to make it consistent with other policy handling.

pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/Topics.java

pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/TopicPolicies.java

pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java

pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/TopicPolicies.java

pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java

...r/src/test/java/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckTest.java

nodece · 2021-12-20T06:02:37Z

/pulsarbot run-failure-checks

nodece · 2021-12-20T07:02:32Z

/pulsarbot run-failure-checks

nodece · 2021-12-20T09:16:30Z

/pulsarbot run-failure-checks

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/SchemasResourceBase.java

...ava/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckOnTopicLevelTest.java

nodece · 2021-12-22T14:12:53Z

/pulsarbot run-failure-checks

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java

...r/src/test/java/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckTest.java

nodece · 2022-01-19T08:16:01Z

/pulsarbot run-failure-checks

nodece · 2022-01-19T09:42:40Z

@codelipenghui @congbobo184 @Jason918 @315157973 could you take a look this PR?

pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java

nodece · 2022-01-27T06:17:46Z

/pulsarbot run-failure-checks

### Motivation Currently, the compatibility strategy for a schema is set at namespace level, which means that all the topics of a namespace will need to adhere to the same compatibility check, this level particle size is relatively large, so add schema compatibility strategy on topic level. ### Modifications - Add set, get and remove schema compatibility strategy API to admin API and client tool - Add schema compatibility strategy action to TopicOperation ### Verifying this change - [x] Make sure that the change passes the CI checks. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): no - The public API: no - The schema: no - The default values of configurations: no - The wire protocol: no - The rest endpoints: no - The admin cli options: no - Anything that affects deployment: no ### Documentation - [x] `doc-required` Signed-off-by: Zixuan Liu <nodeces@gmail.com>

congbobo184

LGTM! nice work

congbobo184 · 2022-01-27T18:30:11Z

@Jason918 please review again, thanks.

Jason918

Left a small comment, LGTM overall.

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java

Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Motivation Currently, the compatibility strategy for a schema is set at namespace level, which means that all the topics of a namespace will need to adhere to the same compatibility check, this level particle size is relatively large, so add schema compatibility strategy on topic level. ### Modifications - Add set, get and remove schema compatibility strategy API to admin API and client tool - Add schema compatibility strategy action to TopicOperation

github-actions bot assigned nodece Dec 14, 2021

github-actions bot added the doc-label-missing label Dec 14, 2021

Anonymitaet added doc-required Your PR changes impact docs and you will update later. and removed doc-label-missing labels Dec 15, 2021

nodece force-pushed the schema_strategy_topic_level branch from 76233bc to 087502c Compare December 15, 2021 09:37

codelipenghui requested review from 315157973, BewareMyPower, Technoboy-, codelipenghui, gaoran10, hangc0276 and merlimat December 15, 2021 14:56

nodece force-pushed the schema_strategy_topic_level branch 2 times, most recently from c6c25b8 to d8a84e4 Compare December 16, 2021 03:46

Jason918 requested changes Dec 16, 2021

View reviewed changes

pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/Topics.java Outdated Show resolved Hide resolved

nodece commented Dec 16, 2021

View reviewed changes

pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/TopicPolicies.java Outdated Show resolved Hide resolved

Jason918 reviewed Dec 17, 2021

View reviewed changes

315157973 reviewed Dec 17, 2021

View reviewed changes

...r/src/test/java/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckTest.java Outdated Show resolved Hide resolved

...r/src/test/java/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckTest.java Outdated Show resolved Hide resolved

Jason918 reviewed Dec 22, 2021

View reviewed changes

nodece commented Dec 22, 2021

View reviewed changes

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/AdminResource.java Outdated Show resolved Hide resolved

nodece commented Dec 22, 2021

View reviewed changes

...r/src/test/java/org/apache/pulsar/schema/compatibility/SchemaTypeCompatibilityCheckTest.java Show resolved Hide resolved

nodece force-pushed the schema_strategy_topic_level branch from 8c524d5 to 1e26beb Compare December 29, 2021 02:06

nodece force-pushed the schema_strategy_topic_level branch 3 times, most recently from 24dd4f1 to 1a1d0a5 Compare January 19, 2022 02:39

momo-jun mentioned this pull request Jan 19, 2022

[Doc] Add content for topic-level schema compatibility check strategy #13832

Merged

1 task

nodece requested a review from congbobo184 January 19, 2022 09:42

Jason918 reviewed Jan 20, 2022

View reviewed changes

pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java Outdated Show resolved Hide resolved

nodece force-pushed the schema_strategy_topic_level branch 3 times, most recently from 42cfa6e to 498f9de Compare January 20, 2022 08:17

Anonymitaet added doc-complete Your PR changes impact docs and the related docs have been already added. and removed doc-required Your PR changes impact docs and you will update later. labels Jan 24, 2022

nodece force-pushed the schema_strategy_topic_level branch 2 times, most recently from f40705f to 332ad9b Compare January 27, 2022 02:09

nodece force-pushed the schema_strategy_topic_level branch from a19abda to 8268c38 Compare January 27, 2022 08:27

nodece force-pushed the schema_strategy_topic_level branch from 8268c38 to 4053e29 Compare January 27, 2022 10:13

congbobo184 approved these changes Jan 27, 2022

View reviewed changes

nodece requested a review from Jason918 January 28, 2022 02:22

Jason918 approved these changes Jan 28, 2022

View reviewed changes

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java Outdated Show resolved Hide resolved

Use async call

e682ff3

Signed-off-by: Zixuan Liu <nodeces@gmail.com>

codelipenghui approved these changes Jan 28, 2022

View reviewed changes

codelipenghui merged commit d22ff4f into apache:master Jan 28, 2022

This was referenced Jan 28, 2022

[Broker] Set default value of applied to false in getSchemaCompatibilityStrategy #14012

Merged

[Broker] Adjust the validation for policy schemaCompatibilityStrategy #14061

Merged

RobertIndie mentioned this pull request Feb 11, 2022

[Broker] Move schema compatibility strategy cmd from topics to topicPolicies #14225

Merged

1 task

Add schema compatibility strategy on topic level #13297

Add schema compatibility strategy on topic level #13297

Uh oh!

Conversation

nodece commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Uh oh!

github-actions bot commented Dec 14, 2021

Uh oh!

Anonymitaet commented Dec 15, 2021

Uh oh!

codelipenghui commented Dec 15, 2021

Uh oh!

nodece commented Dec 16, 2021

Uh oh!

Jason918 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nodece commented Dec 20, 2021

Uh oh!

nodece commented Dec 20, 2021

Uh oh!

nodece commented Dec 20, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nodece commented Dec 22, 2021

Uh oh!

Uh oh!

Uh oh!

nodece commented Jan 19, 2022

Uh oh!

nodece commented Jan 19, 2022

Uh oh!

Uh oh!

nodece commented Jan 27, 2022

Uh oh!

congbobo184 left a comment

Choose a reason for hiding this comment

Uh oh!

congbobo184 commented Jan 27, 2022

Uh oh!

Jason918 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nodece commented Dec 14, 2021 •

edited

Loading