Skip to content

Conversation

@codelipenghui
Copy link
Contributor

Fixes #7384

Motivation

Fix batch ackset recycled multiple times. The root cause is a race condition in group ack flush and cumulative Ack. So add recycled state check for the ackset.

Verifying this change

New unit test added.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

@codelipenghui codelipenghui self-assigned this Jul 1, 2020
@codelipenghui codelipenghui added this to the 2.7.0 milestone Jul 1, 2020
@codelipenghui codelipenghui added release/2.6.1 type/bug The PR fixed a bug or issue reported a bug labels Jul 1, 2020

public void recycle() {
if (recyclerHandle != null) {
if (recyclerHandle != null && recycled.compareAndSet(false, true)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still not safe. There cannot be a race condition to resolve the recycling.

In this case it can happen:

  1. thread-1 calls recycle and CAS succeeds
  2. thread-2 gets a the object from pool
  3. thread-3 sees recycled==false and does the CAS again, recycle a different object that is in use.

You cannot use Recyclable for objects with a shared ownership. Either change the logic to avoid race conditions, or use ref-counting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense. Thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@merlimat I have addressed your comments, please take a look again, thanks.

@codelipenghui
Copy link
Contributor Author

/pulsarbot run-failure-checks

@codelipenghui
Copy link
Contributor Author

/pulsarbot run-failure-checks

@merlimat merlimat merged commit 25a6907 into apache:master Jul 6, 2020
codelipenghui added a commit to streamnative/pulsar-archived that referenced this pull request Jul 14, 2020
* Fix batch ackset recycled multiple times.

* Apply comments.

* Update pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/BitSetRecyclable.java

(cherry picked from commit 25a6907)
cdbartholomew pushed a commit to kafkaesque-io/pulsar that referenced this pull request Jul 21, 2020
* Fix batch ackset recycled multiple times.

* Apply comments.

* Update pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/BitSetRecyclable.java
cdbartholomew pushed a commit to kafkaesque-io/pulsar that referenced this pull request Jul 24, 2020
* Fix batch ackset recycled multiple times.

* Apply comments.

* Update pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/BitSetRecyclable.java
wolfstudy pushed a commit that referenced this pull request Jul 29, 2020
* Fix batch ackset recycled multiple times.

* Apply comments.

* Update pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/BitSetRecyclable.java

(cherry picked from commit 25a6907)
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
* Fix batch ackset recycled multiple times.

* Apply comments.

* Update pulsar-common/src/main/java/org/apache/pulsar/common/util/collections/BitSetRecyclable.java
@codelipenghui codelipenghui deleted the penghui/fix-7384 branch December 4, 2020 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release/2.6.1 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pulsar reader throws exception java.lang.IllegalStateException: recycled multiple times

2 participants