[Task]: Debezium IO Upgrade to 3.1.1 (and Java 17 Build Enablement) #34747
Description
What needs to happen?
Upgrade the Debezium IO connector (sdks:java:io:debezium) to Debezium version 3.1.1.Final. This version of Debezium requires its dependencies (like debezium-core) to be compiled against Java 17 bytecode.
@damccorm started doing something similar for Debezium 2.7.4 and created a WIP branch
To manage this upgrade gracefully and make the review process smoother, split the work into a two-step PR:
- PR 1 (Build Infrastructure): A smaller, focused PR to enable Java 17 compilation specifically for the sdks:java:io:debezium module. We already have infrastructure for the goal of "PR 1". We use BeamPluginModulePlugin to initialize java modules and there is a setJavaVerOptions method. Here is a module could be an example how compiling with different Java version works in Beam.
- PR 2 (Debezium 3.1.1 Upgrade & API Adaptation): A subsequent PR that would: Actually update the Debezium dependencies to 3.1.1.Final.
Incorporate the necessary code changes within sdks:java:io:debezium to adapt to any API changes in Debezium 3.1.1.
We'll need to handle Debezium with newer Java version as well (That's the reason test currently failing on [WIP] Upgrade io.debezium using #33526 as a blueprint #34733). We'll need to modify test workflow file. However due to GitHub limitation changes to .github/workflows won't be effective on unsubmitted PRs. Changes need to be tested locally first.
Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner