MINOR: Improve Log layer segment iteration logic and few other areas

[kowshik]

1. In Log.collectAbortedTransactions() I've restored a previously used logic, such that it would handle the case where the starting segment could be null. This was the case previously, but the PR KAFKA-12552: Introduce LogSegments class abstracting the segments map #10401 accidentally changed the behavior causing the code to assume that the starting segment won't be null.
2. In Log.rebuildProducerState() I've removed usage of the allSegments local variable. The logic looks a bit simpler after I removed it.
3. I've introduced a new LogSegments.higherSegments() API. This is now used to make the logic a bit more readable in Log. collectAbortedTransactions() and Log.deletableSegments() APIs.
4. I've removed the unnecessary use of java.lang.Long in LogSegments class' segments map definition.
5. I've converted a few LogSegments API from public to private, as they need not be public.

Tests:
Relying on existing unit tests.

[kowshik]

@ijuma If you think this API is a good fit to the requirement, I can add unit tests for it.

[kowshik]

cc @junrao

[ccding]

can we simply use while (fetchDataInfo == null && segmentOpt.isDefined) rather than while (!done)?

[kowshik]

Done. Please see 80c40171302546be2dcf2444167d7f375f0b820d.

[ccding]

LGTM

Thanks @kowshik

[junrao]

@kowshik : Thanks for the PR. A couple of comments below.

[junrao]

The old logic supports skipping forward multiple segments to find the right data. The new logic seems to only support skipping forward once. It would be useful to preserve the original semantic.

[kowshik]

That is an excellent catch. Sorry I missed this. Done, changed it now.

[junrao]

Could we return a constant empty map?

[kowshik]

Done.

[kowshik]

Thanks for the review @junrao! I have addressed your comments in 4b836034415c3d5f6b84384ef9be1e75b66edc4b.

[junrao]

@kowshik : Thanks for the updated PR. There seems to be a compilation error in java 8.

[kowshik]

@junrao Thanks for the review and for catching this issue! I've fixed the build now. The issue was that the code in Log.scala used the Iterator.nextOption()API that was introduced only in scala v2.13, so the build failed for scala v2.12.

I have now also added unit tests for the newly introduced LogSegments.higherSegments() API.

The PR is ready for review again.

[junrao]

@kowshik : Thanks for the updated PR. LGTM