[DO NOT MERGE] investigate flaky test

[dengziming]

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.

Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[dajac]

Have you been able to reproduce it locally? I've tried but couldn't.

[dengziming]

I tried multiple versions of JDK but still failed to reproduce it, it's weird so I tried to reproduce it here, 💀. @dajac

[dengziming]

Refer to #10389