KAFKA-7855: Kafka Streams Maven Archetype quickstart fails to compile out of the box

[aurlien]

In the LineSplit.java example, the untyped KStream is resolved to KStream<Object, Object>. value.split(..) then fails to build, because value is of type Object.

Other possible solutions would be to add the type to the builder:
builder.<String, String>stream(...).flatmap(..).to(...), or builder.stream(..., Consumed.with(Serdes.String(), Serdes.String())).

This change matches the code in the tutorial.

WIP: It should be tested during compile that the Java resource files compiles. However, I haven't found any way to achieve this. Feedback appreciated!

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[guozhangwang]

@aurlien thanks for the PR! Can you try build the project just as a normal mvn one and see if it compiles?

[aurlien]

Hi @guozhangwang, thanks for reaching out.

Trying to compile the quickstart project (mvn clean compile), I get the following error: Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-artifacts) on project streams-quickstart: Unable to execute gpg command: Error while executing process. Cannot run program "gpg": error=2, No such file or directory

I haven't managed to fix this yet, but guess that has something to do with my local setup?

However, making the same change as this PR does, to the output of the following, makes the built example project compile:

mvn archetype:generate \
    -DarchetypeGroupId=org.apache.kafka \
    -DarchetypeArtifactId=streams-quickstart-java \
    -DarchetypeVersion=2.1.0 \
    -DgroupId=streams.examples \
    -DartifactId=streams.examples \
    -Dversion=0.1 \
    -Dpackage=myapps

Please tell me if I'm doing something wrong here - this is my first Kafka PR, and I'm eager to learn :)

[mjsax]

@aurlien Thanks for the PR. And welcome to the community!

I personally would prefer the solution to specify the type as builder.<String, String>stream(...) -- the reason it, that many developers don't know that this is possible. Thus, using builder.<String, String>stream(...) has an additional educational value IMHO :) -- I don't insist on it and the current PR LGTM, too.

About the error: yes, it seems you don't have gpg installed or not setup correctly -- maven can't find it.

@guozhangwang I am wondering why this was not detected via tests. Seems we don't have a test pipeline for this. I am not sure atm how hard it would be to get test coverage for this? Thoughts?

[guozhangwang]

@guozhangwang I am wondering why this was not detected via tests. Seems we don't have a test pipeline for this. I am not sure atm how hard it would be to get test coverage for this? Thoughts?

It is a bit hard.. note for architype project, it declares the streams jar as external dependency, so to test this we need to have a unit test that takes in the streams jar. Maybe we can have a system test style case, like StreamsUpgradeTest but it should not be part of o.a.k. package.

[mjsax]

@guozhangwang What is the conclusion? We skip adding a test and merge?

[guozhangwang]

Yeah although it is not ideal I felt it is very hard, if possible to add a test covering similar issues as part of AK, since it would involve adding a new procedure as part of unit test that 1) download an artifact, 2) mvn compile on that artifact. We can probably add a script doing so and add it as part of a jenkins job, but would hard to be part of ./gradlew test cmd.

[aurlien]

Thanks for the feedback! And sorry for the late reply, for some reason this conversation ended in my spam folder.

If you prefer, I can add the type to the builder - I see your point about the educational value.

[guozhangwang]

cc @mjsax for final reviews.

[mjsax]

I just tested this locally. It's does still not compile, because we also need to import KStream.

I leave it up to you how you fix it -- either add the import or get rid of KStream and add the type to the builder.

[mjsax]

@aurlien Any update on this?

[aurlien]

@mjsax Sorry for the delay, I will fix it tomorrow!

[aurlien]

@mjsax I updated to use your recommended builder.<String, String> syntax.

However, I couldn't get the previous solution to fail, when compiling this project. I ran > mvn clean compile install -Dgpg.skip in the kafka/streams/quickstart/java directory. Am i doing something wrong?

[mjsax]

The command you execute, creates the archetype. It only fails if you use the archetype, ie, you need to do a second step:

mvn archetype:generate \
    -DarchetypeCatalog=local \
    -DarchetypeGroupId=org.apache.kafka \
    -DarchetypeArtifactId=streams-quickstart-java \
    -DarchetypeVersion=2.3.0-SNAPSHOT \
    -DgroupId=streams.examples \
    -DartifactId=streams.examples \
    -Dversion=0.1 \
    -Dpackage=myapps

cd streams.examples

mvn compile 

The creates a new project based on the archetype and you cannot compile this project.

[mjsax]

Java8 Passed. Java11 failed with flaky tests.

Retest this please.

[mjsax]

LGTM.

Will merge after Jenkins passed.

[mjsax]

Thanks for the fix @aurlien!

[mjsax]

Merged to trunk and cherry-picked to 2.2, 2.1, and 2.0.