KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0#4773
KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0#4773mjsax merged 10 commits intoapache:1.1from
Conversation
|
This is the port of #4746 (
If we decide to cleanup tests in 1.0, those changes need to be added to this branch, too. System tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1605/ |
|
System tests passed: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1605/ |
mjsax
force-pushed
the
kafka-6054-fix-upgrade-11
branch
from
1f51055 to
fe8b254
Compare
There was a problem hiding this comment.
Similar to my other PR's comment, I'd suggest we update this test to get rid of using StreamsSmokeTestDriverService and use StreamsUpgradeTestJobRunnerService.
There was a problem hiding this comment.
Can you elaborate? StreamsSmokeTestDriverService is used the generate the input data in this test and there is no alternative implementation, and StreamsUpgradeTestJobRunnerService is used already.
There was a problem hiding this comment.
This PR LGTM.
I do agree that we could consider consolidating system tests, but I also think we can call the scope on this PR big enough already. I think it's a better idea to make a separate task to step back and look at our whole testing strategy to try and globally optimize it.
There was a problem hiding this comment.
Should these target versions be like 1.1.x?
(I'll not repeat this question on every occurrence)
Update upgrade docs Github comments John's review Guozhang's follow up
remove generics move config Consolidating system tests
mjsax
force-pushed
the
kafka-6054-fix-upgrade-11
branch
from
fe8b254 to
1183029
Compare
|
Rebased, cherry-picked missing changes from #4768, and address review comments. Triggered upgrade system test: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1648/ Call for review. |
|
The system test 1648 failed, is it related? |
|
I think so. Need to investigate... (maybe I introduced an issue rebasing or cherry-picking) |
vvcephei
left a comment
vvcephei
left a comment
There was a problem hiding this comment.
This looks right to me.
|
Retriggered system tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1651/ 1.1.0 release broke the tests -- needed to upgrade version for tests to 1.1.1-SNAPSHOT |
|
The failing tests ( on broker startup affecting version 1.0.1 only. We disabled this test in |
|
I think this error is saying that Kafka was compiled targeting JDK8 but you're attempting to run it with JDK7. If that sounds like the kind of thing we control in the system tests, I think we should fix it rather than disabling the test. |
|
@mjsax I forgot to @-mention you about this^^ |
|
System test passed. Call for review. @vvcephei Scala version was the issue -- I downgraded to use 2.11. System test are executed using JDK7 in branch builder. |
* confluent/1.1: (60 commits) MINOR: Fix kafka-run-class for Java 10 (apache#4895) KAFKA-6772: Load credentials from ZK before accepting connections (apache#4867) KAFKA-6742: TopologyTestDriver error when dealing with stores from GlobalKTable MINOR: Mention that -1 disables retention by time (apache#4881) KAFKA-6790: Fix Streams processor node broken link (apache#4874) MINOR: Java 10 fixes so that the build passes (apache#4839) MINOR: Update Jackson to 2.9.5 (apache#4776) MINOR: Downgrade to Gradle 4.5.1 (apache#4791) MINOR: Java 9/10 fixes, gradle and minor deps update (apache#4725) KAFKA-6752: Enable unclean leader election metric (apache#4838) KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0 (apache#4773) KAFKA-6747 Check whether there is in-flight transaction before aborting transaction (apache#4826) KAFKA-6748: double check before scheduling a new task after the punctuate call (apache#4827) KAFKA-6739; Ignore headers when down-converting from V2 to V0/V1 (apache#4813) KAFKA-6728: Corrected the worker’s instantiation of the HeaderConverter KAFKA-6731: waitOnState should check the state to be the target start. (apache#4808) HOTFIX: Enforce a rebalance upon task migration (apache#4802) MINOR: Remove 1.2.0 changes from streams doc (apache#4784) MINOR: Update version numbers to 1.1.1-SNAPSHOT MINOR: Fix ReassignPartitionsClusterTest.testHwAfterPartitionReassignment test (apache#4781) ...
…:1.1-nflx to 1.1-nflx * commit '84eeea7fe4b3a64b84b87d231969acfee4fb7544': Fix a bug where the ReqeustPerSec API version hash map is not updated. KAFKA-6772: Load credentials from ZK before accepting connections (apache#4867) KAFKA-6742: TopologyTestDriver error when dealing with stores from GlobalKTable MINOR: Mention that -1 disables retention by time (apache#4881) KAFKA-6790: Fix Streams processor node broken link (apache#4874) MINOR: Java 10 fixes so that the build passes (apache#4839) MINOR: Update Jackson to 2.9.5 (apache#4776) MINOR: Downgrade to Gradle 4.5.1 (apache#4791) MINOR: Java 9/10 fixes, gradle and minor deps update (apache#4725) KAFKA-6752: Enable unclean leader election metric (apache#4838) KAFKA-6054: Fix upgrade path from Kafka Streams v0.10.0 (apache#4773) KAFKA-6747 Check whether there is in-flight transaction before aborting transaction (apache#4826) KAFKA-6748: double check before scheduling a new task after the punctuate call (apache#4827)
4 participants
No description provided.