MINOR: revert change in log level for skipped record

[vvcephei]

It looks like it was a mistake during #6521

Specifically, while addressing code review comments
to change other logs from debugs to warnings, this
one seems to have been included by accident.

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[vvcephei]

Ping @ableegoldman or @cadonna for a review

[ableegoldman]

I replied on the user mailing list thread about my hesitation to "fix" this, but if we do ultimately decide to go back to debug note that we should also fix this in InMemorySessionStore and AbstractRocksDBSegmentedBytesStores

[vvcephei]

After discussing this with @cadonna , he filled me in that this change was actually intentional as part of KIP-444. Closing this PR.

[IndeedSi]

@vvcephei @ableegoldman Hi, I'm seeing large amount of this WARN log every time our instances restart. I think it's just the same case as described in the mail list. These logs are causing high disk and CPU consumption.

Is there any followup after that mail thread? Thanks!

[vvcephei]

Hi @IndeedSi ,

No, as far as I know Jiri's solution was to increase the join window size. Judging from the investigation, it seemed like there really were out-of-order records in the repartition topic that needed to be buffered so they could be processed instead of dropped (as opposed to some kind of superfluous logging as we initially thought).

Does you investigation point to the same cause? If so, then really the only thing that can be done is to increase the join window (aka buffer more records), so that you aren't dropping any data.

I hope this helps,
-John

[IndeedSi]

@vvcephei Thanks, John
I think you are right. We did end up with larger window size (for some other reason) and this log seems no longer happening during restarts.