MINOR: Add statmenet about ZK deprecation to 3.5 release blog post#528
MINOR: Add statmenet about ZK deprecation to 3.5 release blog post#528mjsax merged 3 commits intoapache:asf-sitefrom
Conversation
| <p>We are proud to announce the release of Apache Kafka 3.5.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://downloads.apache.org/kafka/3.5.0/RELEASE_NOTES.html">release notes</a>.</p> | ||
| <p>See the <a href="https://kafka.apache.org/35/documentation.html#upgrade_3_5_0">Upgrading to 3.5.0 from any version 0.8.x through 3.4.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p> | ||
| <p>The ability to migrate Kafka clusters from ZK to KRaft mode with no downtime is still an early access feature. It is currently only suitable for testing in non production environments. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-866+ZooKeeper+to+KRaft+Migration">KIP-866</a> for more details.</p> | ||
| <p><it>Note: ZooKeeper is now marked deprecated in this release. ZooKeeper is planned to be removed in Apache Kafka 4.0. (Cf <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5">KIP-833</a>)</it><p> |
There was a problem hiding this comment.
I would probably make it more clear over there what deprecated means. We will continue to support and backport bug fixes for Zk in 3.x releases. With 4.x release, Zk will be removed from Apache Kafka.
A natural next question in the mind of reader comes as when is 4.0 happening. We should also mention that release is to-be-decided.
There was a problem hiding this comment.
IMO, it's not great to add this important note after the 3.5.0 announcement published.
There was a problem hiding this comment.
Agreed about making clear what it means.
We should have added this before and it was a miss. But it's better to add it now than not at all. The key reason for doing this is to give as much advance notice as possible. The KIP linked to does have a target date for 4.0, but it's subject to change (as usual).
Also, the video does mention this even though the blog post does not: https://youtu.be/BVxDFL5iTx8?t=70
|
I agree with Divij that we need to be careful about the wording. Deprecated typically means "do not use for anything new" and "migrate to the alternative as soon as possible". For both cases there are currently issues. A few features are still missing in KRaft (for example JBOD) and the migration process is still in early access. Also since KIP-833 was voted, there has been significant changes to the KIP document. The version that was voted (revision 10) proposed deprecating ZooKeeper in 3.4. Somehow we thought 3.4 was not ready. 3.5 gets us a bit closer (with support for SCRAM for example) but migration is still early access and some features are still missing. Clearly we messed up with the communication on this. Sure we can update the blog post but we need to update the documentation too. Marking ZooKeeper as deprecated is significant for many of our users so we want to be sure it's clearly described in the official channels. Doing it post-release is not nice to our users. |
@mimaison Right and that's true here too. For something like kafka, migration is not something that can be done very quickly. This note is meant to help users plan the migration work. They can start by testing the migration support we have now in non prod environments and provide feedback.
+1 for updating the docs too. cc @cmccabe @jsancio @rondagostino
Agreed, but it's better than waiting until 3.6. Also, as I said in the other comment, the video does mention this. Finally, we will reinforce it in subsequent releases too - there's no downside to also updating the docs/release post now. |
The video is not from Apache so I'm not sure we should use that as authority. |
|
IMO, we should have a living document "Zk deprecation roadmap" which is updated with every release. Every release blog post will have a section pointing to this document. This solves our requirement of notifying users in advance that Ismael referred to and also pushed official "deprecation" to 3.6. This blog should talk about Zk removal in 4.0 and should answer the following:
If we don't have consensus on answers to these questions yet, we should discuss them in the community. Ideally, I would like to have a conclusion on the topic of longer EOL support for last minor version in 3.x (conversation here) but fine to post it in the blog as to be decided. For 3.5, since we didn't post this in the release blog, my preference will be to have a one-time announcement/blog which will contain the snapshot of the living document valid as of v3.5. |
That's fair - I thought these release videos had been moved to the Apache account, but it was a different set of videos (example). Even so, I still think our users are better off if we update the announcement and docs now vs a few months later. Can someone explain why the latter would be better? |
|
@divijvaidya The KIP linked from this PR is the living document right now. I think we're overthinking this, but if anyone here wants to volunteer to write an additional blog post - please go ahead. The reality is that people don't follow these announcements that closely. Many more will read them over the coming weeks - it's better for these people to get the additional information than not. |
For me, I don't have a concern with marking Zk as deprecated starting 3.5. But I do want to ensure that the manner in which we convey it covers potential FAQs and demonstrates a path forward to the users. We don't (correct me if I am wrong) publish timeline for 4.0 release anywhere except nested inside a KIP which has a misleading title. From the title, I would expect the KIP to talk about state of kraft production readiness and not Zk deprecation. Kraft production readiness is an independent topic than Zk deprecation. That is why I think we should have a separate KIP/living document for Zk deprecation timeline and a more verbose communication to the users (instead of one line in this announcement). We can either expand on the existing blog post or I would be happy to volunteer to write a new one once everyone is in agreement (I don't have a strong opinion either ways). |
|
To clarify I'm not against updating the blog post. I hope my replies are not perceived as confrontational. I think we (and that includes me) messed up the external communication due to a lack of internal communication. As none of us can see in the future, it's expected dates will change and things will shift. We have to communicate clearly our intents so we're all on the same page. For example, for me, it's not obvious why we decided to do it in 3.5 instead of 3.4. I'm not asking for explanations for all past actions (it's past now), it's just to highlight the gap in communications. We can keep updating KIP-833 but let's communicate each time we make changes. As I said this deprecation is significant for a lot of users (most Kafka environments out there still run ZooKeeper) so we need to be clear about what it means and also what we want users to do. Let's create a new section in the docs. |
|
@divijvaidya @mimaison I agree with what you both said. Also, I take this whole debate as constructive conversation to do the right thing for our users. :)
Yes, I believe an email was sent to the mailing list thread after non trivial updates. At least that was the intent, I didn't verify every single instance and it's possible that something may have been missed.
+1
Would fixing the title help or do you think it would be better to have a separate KIP?
Yeah, I totally agree we should have a section in the docs. I think we all aligned on this right at the start. |
My personal take is that we should do bug fix releases for the last 3.x for 12 months if there is need (important bugs or security issues). Let's try to close the mailing list thread on this topic. |
|
Haha. This "escalated" quickly 🤣 I agree with everything discussed. I only don't understand
To me, both are totally related. I don't think we should decouple them. I believe the most important points are the following:
I agree with Ismael that many people don't read the blog post right away, and updating it is valuable; I also second Mickael's concern that "deprecation" for this case has a different meaning. I don't think we need to get overboard and have an additional KIP or blog post about it though. Just my 2ct. If we can agree to this proposal, the next step would be to update the docs, and we can update this PR to link to the docs afterwards. Looking forward to your comments. |
|
@mimaison @ijuma @divijvaidya -- any thoughts? Would not nice to move this forward. |
My concerns are covered by the action items here, so I am good. |
|
Yes, that plan looks good to me. |
|
We got a PR for the ZK section: apache/kafka#14031 In case you want to take a look. |
|
@mimaison @divijvaidya @ijuma @forlack Updated this PR, including the changes of apache/kafka#14031 |
mimaison
left a comment
mimaison
left a comment
There was a problem hiding this comment.
There are some rendering issues. If it's copied straight from apache/kafka we'll need to fix them there too
| <p>We are proud to announce the release of Apache Kafka 3.5.0. This release contains many new features and improvements. This blog post will highlight some of the more prominent features. For a full list of changes, be sure to check the <a href="https://downloads.apache.org/kafka/3.5.0/RELEASE_NOTES.html">release notes</a>.</p> | ||
| <p>See the <a href="https://kafka.apache.org/35/documentation.html#upgrade_3_5_0">Upgrading to 3.5.0 from any version 0.8.x through 3.4.x</a> section in the documentation for the list of notable changes and detailed upgrade steps.</p> | ||
| <p>The ability to migrate Kafka clusters from ZK to KRaft mode with no downtime is still an early access feature. It is currently only suitable for testing in non production environments. See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-866+ZooKeeper+to+KRaft+Migration">KIP-866</a> for more details.</p> | ||
| <p><it>Note: ZooKeeper is now marked deprecated in this release. ZooKeeper is planned to be removed in Apache Kafka 4.0. (Cf <a href="/documentation#zk_dep">ZooKeeper Deprecation</a>)</it><p> |
| <h4 class="anchor-heading"><a id="zkversion" class="anchor-link"></a><a href="#zkversion">Stable version</a></h4> | ||
| The current stable branch is 3.5. Kafka is regularly updated to include the latest release in the 3.5 series. | ||
|
|
||
| <h4 class="anchor-heading"><a href="#zk_depr" class="anchor-link">ZooKeeper Deprecation</a></h4> |
There was a problem hiding this comment.
This does not render well. You need to use the same syntax than the other headers.
My diff to make it render correctly:
diff --git 35/ops.html 35/ops.html
index 37e6a9b9..37d1ffdf 100644
--- 35/ops.html
+++ 35/ops.html
@@ -3444,16 +3444,16 @@ for built-in state stores, currently we have:
<h4 class="anchor-heading"><a id="zkversion" class="anchor-link"></a><a href="#zkversion">Stable version</a></h4>
The current stable branch is 3.5. Kafka is regularly updated to include the latest release in the 3.5 series.
- <h4 class="anchor-heading"><a href="#zk_depr" class="anchor-link">ZooKeeper Deprecation</a></h4>
- <p>With the release of Apache Kafka 3.5, Zookeeper is now marked deprecated. Removal of ZooKeeper is planned in the next major release of Apache Kafka (version 4.0), which is scheduled to happen no sooner than April 2024. During the deprecation phase, ZooKeeper is still supported for metadata management of Kafka clusters, but it is not recommended for new deployments. There is a small subset of features that remain to be implemented in KRaft, see <a href="#kraft_missing" class="anchor-link">current missing features</a> for more information.</p>
+ <h4 class="anchor-heading"><a id="zk_depr" class="anchor-link"></a><a href="#zk_depr">ZooKeeper Deprecation</a></h4>
+ <p>With the release of Apache Kafka 3.5, Zookeeper is now marked deprecated. Removal of ZooKeeper is planned in the next major release of Apache Kafka (version 4.0), which is scheduled to happen no sooner than April 2024. During the deprecation phase, ZooKeeper is still supported for metadata management of Kafka clusters, but it is not recommended for new deployments. There is a small subset of features that remain to be implemented in KRaft, see <a href="#kraft_missing">current missing features</a> for more information.</p>
- <h5 class="anchor-heading"><a href="#zk_depr_migration" class="anchor-link">Migration</a></h5>
+ <h5 class="anchor-heading"><a id="zk_depr_migration" class="anchor-link"></a><a href="#zk_drep_migration">Migration</a></h5>
<p>Migration of an existing ZooKeeper based Kafka cluster to KRaft is currently Preview and we expect it to be ready for production usage in version 3.6. Users are recommended to begin planning for migration to KRaft and also begin testing to provide any feedback. Refer to <a href="#kraft_zk_migration">ZooKeeper to KRaft Migration</a> for details on how to perform a live migration from ZooKeeper to KRaft and current limitations.</p>
- <h5 class="anchor-heading"><a href="#zk_depr_3xsupport" class="anchor-link">3.x and ZooKeeper Support</a></h5>
+ <h5 class="anchor-heading"><a id="zk_depr_3xsupport" class="anchor-link"></a><a href="#zk_depr_3xsupport">3.x and ZooKeeper Support</a></h5>
<p>The final 3.x minor release, that supports ZooKeeper mode, will receive critical bug fixes and security fixes for 12 months after its release.</p>
- <h5 class="anchor-heading"><a href="#zk_depr_timeline" class="anchor-link">ZooKeeper and KRaft timeline</a></h5>
+ <h5 class="anchor-heading"><a id="zk_depr_timeline" class="anchor-link"></a><a href="#zk_depr_timeline">ZooKeeper and KRaft timeline</a></h5>
<p>For details and updates on tentative timelines for ZooKeeper removal and planned KRaft feature releases, refer to <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready">KIP-833</a>.</p>
<h4 class="anchor-heading"><a id="zkops" class="anchor-link"></a><a href="#zkops">Operationalizing ZooKeeper</a></h4>
There was a problem hiding this comment.
Should we also add an entry to the ToC since we're adding a section? Currently the ToC only has:
6.9 ZooKeeper
- Stable Version
- Operationalization
There was a problem hiding this comment.
Thanks. I did c&p from the PR against trunk -- if this does not render as expected, we need to part the change to trunk and 3.5 branches, too. (I can do another PR...)
Good idea to update TOC.
|
Follow up PR to get the latest comments back ported to |
5 participants
No description provided.