Skip to content

Conversation

@coderzc
Copy link
Member

@coderzc coderzc commented Jan 3, 2023

PIP: #16763

Motivation

In #19035, the skipCondition parameter returns need to skip entry when reading entry, but the logic now is reversed.

Modifications

Fix skip logic and add test to cover it.

Verifying this change

  • Make sure that the change passes the CI checks.

testReadEntriesWithSkip

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jan 3, 2023
@coderzc coderzc self-assigned this Jan 3, 2023
@coderzc coderzc added this to the 2.12.0 milestone Jan 3, 2023
@coderzc coderzc changed the title [fix][broker] fix cursor skip read [fix][broker][PIP-195] fix cursor skip read Jan 3, 2023
@coderzc coderzc closed this Jan 3, 2023
@coderzc coderzc reopened this Jan 3, 2023
@coderzc coderzc force-pushed the fix_cursor_read_skip branch from 4a4a1a4 to f1a7bd7 Compare January 3, 2023 15:27
@codecov-commenter
Copy link

codecov-commenter commented Jan 3, 2023

Codecov Report

Merging #19124 (a32996f) into master (9ec1d07) will increase coverage by 0.27%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #19124      +/-   ##
============================================
+ Coverage     47.43%   47.70%   +0.27%     
- Complexity     9535     9594      +59     
============================================
  Files           632      632              
  Lines         59839    59839              
  Branches       6234     6234              
============================================
+ Hits          28384    28548     +164     
+ Misses        28375    28194     -181     
- Partials       3080     3097      +17     
Flag Coverage Δ
unittests 47.70% <ø> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...lsar/broker/loadbalance/impl/ThresholdShedder.java 3.27% <0.00%> (-27.05%) ⬇️
...r/service/AbstractDispatcherMultipleConsumers.java 56.45% <0.00%> (-14.52%) ⬇️
...org/apache/pulsar/broker/loadbalance/LoadData.java 58.33% <0.00%> (-8.34%) ⬇️
...g/apache/pulsar/client/impl/ConnectionHandler.java 50.00% <0.00%> (-5.32%) ⬇️
...roker/loadbalance/impl/ModularLoadManagerImpl.java 60.39% <0.00%> (-3.33%) ⬇️
...ersistentStickyKeyDispatcherMultipleConsumers.java 58.33% <0.00%> (-1.97%) ⬇️
...g/apache/pulsar/compaction/CompactedTopicImpl.java 77.14% <0.00%> (-1.43%) ⬇️
...he/pulsar/broker/admin/v2/NonPersistentTopics.java 60.64% <0.00%> (-1.39%) ⬇️
...rg/apache/pulsar/broker/web/PulsarWebResource.java 55.29% <0.00%> (-0.94%) ⬇️
...rg/apache/pulsar/compaction/TwoPhaseCompactor.java 72.81% <0.00%> (-0.93%) ⬇️
... and 28 more

completableFuture.complete(entries.size());
try {
entries.forEach(entry -> {
assertNotEquals(entry.getEntryId() % 2, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be assertEquals(entry.getEntryId() % 2, 0);

And due to this one is executed by the SafeRun, so the test will not get failed event if the assert fails.

Copy link
Member Author

@coderzc coderzc Jan 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And due to this one is executed by the SafeRun, so the test will not get failed event if the assert fails.

I already use completableFuture.completeExceptionally(e) to receive exception, it should be can work.

But AssertionError is not a subclass of Exception, so we should be caught Throwable.

image

completableFuture2.complete(entries.size());
try {
entries.forEach(entry -> {
assertEquals(entry.getEntryId() % 2, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the above comment. You might need to consider using a different way to assert the result.

@coderzc coderzc requested a review from codelipenghui January 4, 2023 03:42
long entryId = firstEntry;
for (; entryId <= lastEntry; entryId++) {
if (opReadEntry.skipCondition.test(PositionImpl.get(ledger.getId(), entryId))) {
if (!opReadEntry.skipCondition.test(PositionImpl.get(ledger.getId(), entryId))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest Invert if condition to help simple understand this logic. : )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

}
}

if (firstValidEntry != -1L) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this logic just works for the else branch. Can we merge it?

Copy link
Member Author

@coderzc coderzc Jan 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can move this logic to the else branch

@coderzc coderzc force-pushed the fix_cursor_read_skip branch from 5dc2bcc to a32996f Compare January 4, 2023 11:23
@codelipenghui codelipenghui merged commit 3bb93cb into apache:master Jan 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/broker doc-not-needed Your PR changes do not impact docs ready-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants