Skip to content

Update to Curator 4.2.0, ZooKeeper 3.4.14.#8177

Closed
gianm wants to merge 1 commit intoapache:masterfrom
gianm:curator-update
Closed

Update to Curator 4.2.0, ZooKeeper 3.4.14.#8177
gianm wants to merge 1 commit intoapache:masterfrom
gianm:curator-update

Conversation

@gianm
Copy link
Copy Markdown
Contributor

@gianm gianm commented Jul 27, 2019

Other than generally wanting to use the latest Curator and ZK, this change is motivated by an outage I encountered last night. I was debugging a cluster last night that was acting bizarrely, and in the end it turned out that it had two overlords that both thought they were leader. Shortly before they both gained leadership, the ZK quorum was unavailable for about 20 seconds. It doesn't look like Druid itself was doing anything particularly wrong: logs indicated the overlords weren't ignoring stopBeingLeader calls or anything like that.

For these reasons, I believe the cause of the outage was https://issues.apache.org/jira/browse/CURATOR-498. This comment indicates the bug could cause two LeaderLatch users to become leaders at once: https://issues.apache.org/jira/browse/CURATOR-498?focusedCommentId=16732419&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16732419.

The bug was fixed in Curator 4.2.0.

@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Jul 27, 2019

The test failures look suspicious. There is no such method as ByteBuffer position(int) in java.nio.ByteBuffer in Java 8, which is what the error message is referring to. I wonder if the curator-framework jar was built with JDK 9+ (unintentionally? since I don't think they intended to drop Java 8 support).

[ERROR] testSanity(org.apache.druid.curator.announcement.AnnouncerTest)  Time elapsed: 1.158 s  <<< ERROR!
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
	at org.apache.curator.framework.imps.GzipCompressionProvider.skip(GzipCompressionProvider.java:319)
	at org.apache.curator.framework.imps.GzipCompressionProvider.doReadHeader(GzipCompressionProvider.java:290)
	at org.apache.curator.framework.imps.GzipCompressionProvider.readGzipHeader(GzipCompressionProvider.java:273)
	at org.apache.curator.framework.imps.GzipCompressionProvider.decompress(GzipCompressionProvider.java:195)

@asdf2014
Copy link
Copy Markdown
Member

@gianm Yep, this is a known bug for curator, more details:
https://issues.apache.org/jira/browse/CURATOR-523

@himanshug
Copy link
Copy Markdown
Contributor

+1 when the build works

@fjy
Copy link
Copy Markdown
Contributor

fjy commented Jul 30, 2019

@gianm can you fix test failures?

@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Aug 1, 2019

@gianm can you fix test failures?

It's broken because of the upstream Curator issue that @asdf2014 linked.

@stale
Copy link
Copy Markdown

stale Bot commented Sep 30, 2019

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

@stale stale Bot added the stale label Sep 30, 2019
@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Sep 30, 2019

Still waiting for a new release of Curator.

@stale
Copy link
Copy Markdown

stale Bot commented Sep 30, 2019

This issue is no longer marked as stale.

@stale stale Bot removed the stale label Sep 30, 2019
@stale
Copy link
Copy Markdown

stale Bot commented Nov 30, 2019

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

@stale stale Bot added the stale label Nov 30, 2019
@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Nov 30, 2019

Fine, let's close this until a new Apache Curator is released.

@gianm gianm closed this Nov 30, 2019
@gianm gianm deleted the curator-update branch November 30, 2019 00:30
@ccaominh
Copy link
Copy Markdown
Contributor

ccaominh commented Apr 1, 2020

Looks like curator 4.2.0 and 4.3.0 are now available.

@maytasm
Copy link
Copy Markdown
Contributor

maytasm commented Apr 1, 2020

curator 4.3.0 should have the fix. I can make a PR for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants