KAFKA-9073: check assignment in requestFailed to avoid NPE

[guozhangwang]

This is a cherry-pick of the bug-fix included in #6884 to 2.3 and older branch.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[guozhangwang]

@ableegoldman @hachikuji

[ableegoldman]

Thanks for the quick PR, LGTM

By the way, I vaguely recall another minor bug you found during that long PR: Also found a minor bug in MetadataUpdate that removed topic would still be retained with null value of num.partitions.

I don't remember the specifics (got that from PR description) but is it worth tacking on to this PR as well? Doesn't seem like anyone has noticed that one or needs the fix, but it wouldn't hurt

[mjsax]

Can we add a test for this case?

[guozhangwang]

By the way, I vaguely recall another minor bug you found during that long PR: Also found a minor bug in MetadataUpdate that removed topic would still be retained with null value of num.partitions.

Ack, I think you were referring to this: https://github.com/apache/kafka/pull/6884/files/ddfe28f9dd5ce26f8f189946e5c5d90b627e33ca#r307557509 Will update this PR as well.

[guozhangwang]

@mjsax I tried to add a unit test but this is not very straight-forward to test the exact root cause since it has to be a race condition between the assignment / subscription change and the request callback / metadata snapshot construction. I feel it is relatively safe to merge to old branches since it is a cherry-pick from trunk / 2.4. WDYT?

[mjsax]

@guozhangwang Feel free to merge.