KAFKA-8855; Collect and Expose Client's Name and Version in the Brokers (KIP-511 Part 1)

[dajac]

This PR implements the first part of KIP-511, namely:

• update the ApiVersionsRequest/Response to the auto-generated ones;
• bump the versions of ApiVersionsRequest/Response;
• add ClientSoftwareName and ClientSoftwareVersions in the ApiVersionsRequest;
• add validation for those two fields and handle error in the client;
• implement the proposed fail-back mechanism.

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.

Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[mumrah]

Looks good, a few small nits.

[cmccabe]

Thanks, @dajac . This looks pretty close to being ready to go. I left a few comments.

[dajac]

@cmccabe Thanks for the review. I have addressed all your comments.

[cmccabe]

LGTM

[cmccabe]

retest this please

[dajac]

retest this please

[dajac]

Failing tests

JDK 8 and Scala 2.11

• org.apache.kafka.connect.integration.ConnectWorkerIntegrationTest.testAddAndRemoveWorker
• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSourceConnector
• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSinkConnector
• kafka.api.SslAdminClientIntegrationTest.testCreateTopicsResponseMetadataAndConfig

JDK 11 and Scala 2.12

• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSourceConnector
• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSinkConnector

JDK 11 and Scala 2.13

• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSourceConnector
• org.apache.kafka.connect.integration.ExampleConnectIntegrationTest.testSinkConnector

They are unrelated to the changes made in this PR.

[dajac]

I ran tests locally and got another failing test:

• ReassignPartitionsClusterTest. shouldPerformMultipleReassignmentOperationsOverVariousTopics

I wanted to confirm that the failing tests in CI are flaky ones.

[cmccabe]

Those tests pass for me locally. Committing

[hxfghgh]

hi my friend,i don't know why you have to "add ClientSoftwareName and ClientSoftwareVersions in the ApiVersionsRequest;"。 This does not seem to be a mandatory field to add.