From 2e7b78edd3339e1a4e6feb71d371d2d772e43e2b Mon Sep 17 00:00:00 2001 From: "S.Y. Wang" Date: Wed, 9 Apr 2025 14:49:16 +0900 Subject: [PATCH 1/5] KAFKA-18983 Ensure all README.md(s) are mentioned by the root README.md --- README.md | 22 ++++++++++++++++++++++ docker/README.md | 2 +- trogdor/README.md | 4 ++-- 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 338f81e17c7f5..4a7cb6c648fea 100644 --- a/README.md +++ b/README.md @@ -99,6 +99,8 @@ fail due to code changes. You can just run: ./gradlew processMessages processTestMessages +See [Apache Kafka Message Definitions](/clients/src/main/resources/common/message/README.md) for details on Apache Kafka message protocol + ### Running a Kafka broker Using compiled files: @@ -111,6 +113,8 @@ Using docker image: docker run -p 9092:9092 apache/kafka:3.7.0 +See [docker/README.md] for detailed information. + ### Cleaning the build ### ./gradlew clean @@ -263,10 +267,28 @@ default. See https://www.lightbend.com/blog/scala-inliner-optimizer for more det See [tests/README.md](tests/README.md). +### Using Trogdor for testing ### + +We Trogdor as a test framework for Apache Kafka. You can use it to run benchmarks and other workloads. + +See [trogdor/README.md](trogdor/README.md). + ### Running in Vagrant ### See [vagrant/README.md](vagrant/README.md). +### Release Kafka ### + +See [release/README.md](release/README.md). + +### Official Documentation ### + +See [docs/README.md](docs/README.md). + +### Kafka client examples ### + +See [examples/README.md](examples/README.md). + ### Contribution ### Apache Kafka is interested in building the community; we would welcome any thoughts or [patches](https://issues.apache.org/jira/browse/KAFKA). You can reach us [on the Apache mailing lists](http://kafka.apache.org/contact.html). diff --git a/docker/README.md b/docker/README.md index 5c46ef954edcf..6923f0f2d59ee 100644 --- a/docker/README.md +++ b/docker/README.md @@ -31,7 +31,7 @@ Building image and running tests using github actions - image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka) - - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming) + - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. - Example(jvm):- To build and test a jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended. diff --git a/trogdor/README.md b/trogdor/README.md index daebf2f50aa09..dda1cd687d4e0 100644 --- a/trogdor/README.md +++ b/trogdor/README.md @@ -6,9 +6,9 @@ Trogdor can run benchmarks and other workloads. Trogdor can also inject faults i Quickstart ========================================================= -First, we want to [start a single-node Kafka cluster in KRaft mode](https://github.com/apache/kafka/blob/trunk/README.md#running-a-kafka-broker-in-kraft-mode) +First, we want to [start a single-node Kafka cluster](https://github.com/apache/kafka/blob/trunk/README.md#running-a-kafka-broker) -Running Kafka in Kraft mode: +Running Kafka: ``` KAFKA_CLUSTER_ID="$(./bin/kafka-storage.sh random-uuid)" From 4089ba0f79e320d462b94334c9b15d23d1ad0538 Mon Sep 17 00:00:00 2001 From: "S.Y. Wang" Date: Wed, 9 Apr 2025 17:50:20 +0900 Subject: [PATCH 2/5] Address feedback --- README.md | 6 +++--- docker/README.md | 8 ++++---- docker/native/README.md | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 4a7cb6c648fea..131adb9979893 100644 --- a/README.md +++ b/README.md @@ -99,7 +99,7 @@ fail due to code changes. You can just run: ./gradlew processMessages processTestMessages -See [Apache Kafka Message Definitions](/clients/src/main/resources/common/message/README.md) for details on Apache Kafka message protocol +See [Apache Kafka Message Definitions](/clients/src/main/resources/common/message/README.md) for details on Apache Kafka message protocol. ### Running a Kafka broker @@ -113,7 +113,7 @@ Using docker image: docker run -p 9092:9092 apache/kafka:3.7.0 -See [docker/README.md] for detailed information. +See [docker/README.md](docker/README.md) for detailed information. ### Cleaning the build ### ./gradlew clean @@ -269,7 +269,7 @@ See [tests/README.md](tests/README.md). ### Using Trogdor for testing ### -We Trogdor as a test framework for Apache Kafka. You can use it to run benchmarks and other workloads. +We use Trogdor as a test framework for Apache Kafka. You can use it to run benchmarks and other workloads. See [trogdor/README.md](trogdor/README.md). diff --git a/docker/README.md b/docker/README.md index 6923f0f2d59ee..ce8cb99603460 100644 --- a/docker/README.md +++ b/docker/README.md @@ -30,8 +30,8 @@ Building image and running tests using github actions - kafka-url - This is the url to download kafka tarball from. For example kafka tarball url from (https://archive.apache.org/dist/kafka). For building RC image this will be an RC tarball url. - image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. - - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka) - - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. + - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/x/z5izDw) + - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. - Example(jvm):- To build and test a jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended. @@ -210,6 +210,6 @@ python generate_kafka_pr_template.py --image-type=jvm ``` - kafka-version - This is the version to create the Docker official images static Dockerfile and assets for, as well as the version to build and test the Docker official image for. -- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka). - - **NOTE:** As of now [KIP-1028](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Image+for+Apache+Kafka) only aims to release JVM based Docker Official Images and not GraalVM based native Apache Kafka docker image. +- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/x/z5izDw). + - **NOTE:** As of now [KIP-1028](https://cwiki.apache.org/confluence/x/0AmpEQ) only aims to release JVM based Docker Official Images and not GraalVM based native Apache Kafka docker image. diff --git a/docker/native/README.md b/docker/native/README.md index 42f38bf4e922a..4bb56261e5148 100644 --- a/docker/native/README.md +++ b/docker/native/README.md @@ -4,7 +4,7 @@ - The Native Apache Kafka Docker Image can launch brokers with sub-second startup time and minimal memory footprint by leveraging native Kafka executable. - The native Kafka executable is built by compiling Apache Kafka code ahead-of-time using the [GraalVM native-image tool](https://www.graalvm.org/jdk21/reference-manual/native-image/). - This image is experimental and intended for local development and testing purposes only; it is not recommended for production use. -- This is introduced with [KIP-974](https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker). +- This is introduced with [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw). ## Native-Image reachability metadata The native-image tool performs static analysis while building a native binary to determine the dynamic features(the dynamic language features of the JVM, including reflection and resource handling, compute the dynamically-accessed program elements such as invoked methods or resource URLs at runtime), but it cannot always exhaustively predict all uses. From 2725f421364fa173dde4f1e1f3608b0213e97755 Mon Sep 17 00:00:00 2001 From: "S.Y. Wang" Date: Thu, 10 Apr 2025 15:13:31 +0900 Subject: [PATCH 3/5] Address feedback --- docker/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/README.md b/docker/README.md index ce8cb99603460..1b8b3a0d9785c 100644 --- a/docker/README.md +++ b/docker/README.md @@ -27,11 +27,11 @@ Building image and running tests using github actions - This is the recommended way to build, test and get a CVE report for the docker image. - Just choose the image type and provide kafka url to `Docker Build Test` workflow. It will generate a test report and CVE report that can be shared with the community. -- kafka-url - This is the url to download kafka tarball from. For example kafka tarball url from (https://archive.apache.org/dist/kafka). For building RC image this will be an RC tarball url. +- kafka-url - This is the url to download kafka tarball from. For example kafka tarball url from [Kafka archive](https://archive.apache.org/dist/kafka). For building RC image this will be an RC tarball url. - image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/x/z5izDw) - - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. + - `native` image type is for graalvm based `native` Kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. - Example(jvm):- To build and test a jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended. From 3747ed5065f8074eef2e188b6f31f7d409b7dc3c Mon Sep 17 00:00:00 2001 From: "S.Y. Wang" Date: Thu, 10 Apr 2025 18:14:32 +0900 Subject: [PATCH 4/5] Address feedback --- docker/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/README.md b/docker/README.md index 1b8b3a0d9785c..0942cfd05bb56 100644 --- a/docker/README.md +++ b/docker/README.md @@ -31,7 +31,7 @@ Building image and running tests using github actions - image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/x/z5izDw) - - `native` image type is for graalvm based `native` Kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming). Or you can see [native/README.md](native/README.md) for more information. + - `native` image type is for graalvm based `native` Kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/x/KZizDw). Or you can see [native/README.md](native/README.md) for more information. - Example(jvm):- To build and test a jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended. From cc731ed6b07a2a193fab4e4686caa4862defcc77 Mon Sep 17 00:00:00 2001 From: "S.Y. Wang" Date: Tue, 15 Apr 2025 16:35:20 +0900 Subject: [PATCH 5/5] Fix --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 131adb9979893..fa151846a409c 100644 --- a/README.md +++ b/README.md @@ -99,7 +99,7 @@ fail due to code changes. You can just run: ./gradlew processMessages processTestMessages -See [Apache Kafka Message Definitions](/clients/src/main/resources/common/message/README.md) for details on Apache Kafka message protocol. +See [Apache Kafka Message Definitions](clients/src/main/resources/common/message/README.md) for details on Apache Kafka message protocol. ### Running a Kafka broker