Skip to content

KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0#4761

Merged
mjsax merged 4 commits intoapache:0.11.0from
mjsax:kafka-6054-fix-upgrade-0110
Mar 27, 2018
Merged

KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0#4761
mjsax merged 4 commits intoapache:0.11.0from
mjsax:kafka-6054-fix-upgrade-0110

Conversation

@mjsax
Copy link
Copy Markdown
Member

@mjsax mjsax commented Mar 23, 2018

No description provided.

@mjsax mjsax added the streams label Mar 23, 2018
@mjsax mjsax requested review from dguy and guozhangwang March 23, 2018 04:39
@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Mar 23, 2018

This is the port of #4746 (0.10.1) and #4758 (0.10.2) to branch 0.11.0.

\cc @bbejeck @vvcephei

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Mar 23, 2018

Upgrade system test passed: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1573/

Triggered all Streams system tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1574/ (Passed)

Note: Java 9 does not work on old 0.11.0 branch -- Java9 was set up later and we can ignore failing Java9 builds.

@mjsax mjsax force-pushed the kafka-6054-fix-upgrade-0110 branch 2 times, most recently from 4e82244 to 8e9597b Compare March 23, 2018 23:21
@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Mar 23, 2018

Updated this. System test passed: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1593/

Github comments
John's review
Guozhang's follow up
Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just left some comments on the web docs.

Comment thread docs/upgrade.html
<h5><a id="upgrade_1020_streams_from_0100" href="#upgrade_1020_streams_from_0100">Upgrading a 0.10.0 Kafka Streams Application</a></h5>
<ul>
<li> Upgrading your Streams application from 0.10.0 to 0.10.2 does require a <a href="#upgrade_10_1">broker upgrade</a> because a Kafka Streams 0.10.2 application can only connect to 0.10.2 or 0.10.1 brokers. </li>
<li> There are couple of API changes, that are not backward compatible (cf. <a href="/{{version}}/documentation/streams#streams_api_changes_0102">Streams API changes in 0.10.2</a> for more details).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we just ref-link the streams/upgrade-guide.html here than copying the content again in the main upgrade section?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can. It's "break" the established pattern though. Up to now, those older sections are just the unchanged release notes from the version before -- I just followed the structure of the existing docs. Let me know what you think. Also, I am wondering, it this changes anyway if we rework the web page from scratch as anticipated.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. We can leave it as is then.

Comment thread docs/upgrade.html
<li> Upgrading your Streams application from 0.10.0 to 0.10.1 does require a <a href="#upgrade_10_1">broker upgrade</a> because a Kafka Streams 0.10.1 application can only connect to 0.10.1 brokers. </li>
<li> There are couple of API changes, that are not backward compatible (cf. <a href="/{{version}}/documentation/streams#streams_api_changes_0101">Streams API changes in 0.10.1</a> for more details).
Thus, you need to update and recompile your code. Just swapping the Kafka Streams library jar file will not work and will break your application. </li>
<li> Upgrading from 0.10.0.x to 0.10.1.2 requires two rolling bounces with config <code>upgrade.from="0.10.0"</code> set for first upgrade phase
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking we cannot yet talk about 0.10.1.2 in 0.11.0.3 yet since we may not release 0.10.1.2 before releasing 0.11.0.3. I think it may not be of a big issue still leaving this here, but just try to point it out.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right -- not sure how to resolve this... Maybe we can only fix this manually when an older bug fix release is done... It's a little annoying :(

Copy link
Copy Markdown
Member

@bbejeck bbejeck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @mjsax LGTM

@bbejeck
Copy link
Copy Markdown
Member

bbejeck commented Mar 26, 2018

retest this please

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Mar 27, 2018

Retest this please.

@mjsax mjsax force-pushed the kafka-6054-fix-upgrade-0110 branch from 4a7d4a6 to 31dca1a Compare March 27, 2018 01:33
@mjsax mjsax merged commit 13dbcad into apache:0.11.0 Mar 27, 2018
@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Mar 27, 2018

Merged to 0.11.0.

@mjsax mjsax deleted the kafka-6054-fix-upgrade-0110 branch June 5, 2018 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants