KAFKA-13092: Perf regression in LISR requests

[jolshan]

Cherry pick of 584213e

Main difference is that we do not try to write to partition metadata file when initializing the Log. Since we only do so in checkOrSet topic ID and there is a small bug in that path, the benchmark attempts to run that method last -- so that the log is created and we can actually record the ID to write.

Some quick runs of the benchmarks --
Master

Benchmark                                   (numPartitions)  (useTopicIds)  Mode  Cnt     Score   Error  Units
PartitionCreationBench.makeFollower                    2000          false  avgt    2  2576.278          ms/op
PartitionCreationBench.makeFollower                    2000           true  avgt    2  5378.075          ms/op

PR

Benchmark                                   (numPartitions)  (useTopicIds)  Mode  Cnt     Score   Error  Units
PartitionCreationBench.makeFollower                    2000          false  avgt    2  2678.498          ms/op
PartitionCreationBench.makeFollower                    2000           true  avgt    2  2962.268          ms/op

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[dajac]

@jolshan is it something that we must add to 2.8.1?

[jolshan]

Ah yes. I had a feeling I still had a pr open for this. I think I can clean it up a bit and it should be ready for review by tomorrow.

[dajac]

@jolshan All the builds have failed. Could you take a look?

[jolshan]

Looks like an issue with more recent changes. I'll fix it up

[jolshan]

Fixed my issue locally but looks like 2.8 branch is broken in general.

> Task :streams:compileTestJava
/Users/jolshan/kafka/streams/src/test/java/org/apache/kafka/streams/integration/MetricsReporterIntegrationTest.java:73: error: stop() has private access in EmbeddedKafkaCluster
        CLUSTER.stop();
               ^
1 error

Will try to contact relevant person.

[jolshan]

fix here: #11257

[dajac]

Do you need to rebase the PR to include it?

[jolshan]

I think you may be able to just restart the build. (I think only committers can do that). If that doesn't work, I'll rebase.

[dajac]

I think you may be able to just restart the build. (I think only committers can do that). If that doesn't work, I'll rebase.

Done, thanks.

[dajac]

@jolshan All tests failed, again. I think that you have to rebase to include the fix.

[jolshan]

Ok will do.

[dajac]

@lbradstreet @junrao Could you take a look at this one as you both reviewed the original PR?

[junrao]

@jolshan : Thanks for the PR. LGTM

[lbradstreet]

LGTM

[dajac]

LGTM

[dajac]

@junrao @lbradstreet Thanks for your reviews.

The failures are not related. I will merge it to 2.8.