MINOR: drop dbAccessor reference on close by vvcephei · Pull Request #6254 · apache/kafka

vvcephei · 2019-02-11T18:53:53Z

Very minor, but we should drop the reference to dbAccessor in the RocksDBStore after we close it.

Committer Checklist (excluded from commit message)

Verify design and implementation
Verify test coverage and CI build status
Verify documentation (including upgrade notes)

vvcephei · 2019-02-11T18:54:29Z

@mjsax @bbejeck ,

Do you mind taking a look at this one-liner? I just happened to notice it while I was debugging something else.

mjsax · 2019-02-12T01:51:51Z

https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2263/

org.apache.kafka.common.KafkaException: Socket server failed to bind to localhost:45180: Address already in use.
	at kafka.network.Acceptor.openServerSocket(SocketServer.scala:573)
	at kafka.network.Acceptor.<init>(SocketServer.scala:451)
	at kafka.network.SocketServer.createAcceptor(SocketServer.scala:245)
	at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1(SocketServer.scala:215)
	at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1$adapted(SocketServer.scala:214)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:214)
	at kafka.network.SocketServer.startup(SocketServer.scala:114)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:253)
	at kafka.utils.TestUtils$.createServer(TestUtils.scala:135)
	at kafka.integration.KafkaServerTestHarness.$anonfun$setUp$1(KafkaServerTestHarness.scala:101)
	at scala.collection.Iterator.foreach(Iterator.scala:941)
	at scala.collection.Iterator.foreach$(Iterator.scala:941)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
	at kafka.integration.KafkaServerTestHarness.setUp(KafkaServerTestHarness.scala:100)
	at kafka.api.IntegrationTestHarness.doSetup(IntegrationTestHarness.scala:81)
	at kafka.api.IntegrationTestHarness.setUp(IntegrationTestHarness.scala:73)
	at kafka.api.ConsumerBounceTest.setUp(ConsumerBounceTest.scala:72)

Retest this please.

mjsax

LGTM.

bbejeck

Thanks @vvcephei LGTM

mjsax · 2019-02-12T08:05:58Z

https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2275/

java.lang.AssertionError: The consumer group deletion did not work as expected
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at kafka.admin.DeleteConsumerGroupsTest.testDeleteCmdWithShortInitialization(DeleteConsumerGroupsTest.scala:211)

https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/19374/

org.junit.runners.model.TestTimedOutException: test timed out after 120000 milliseconds
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:502)
	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:92)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
	at kafka.utils.TestUtils$.assertFutureExceptionTypeEquals(TestUtils.scala:1429)
	at kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts(AdminClientIntegrationTest.scala:1071)

Retest this please.

mjsax · 2019-02-12T18:35:36Z

Build failed because trunk was broken, but should be fixed now via #6257

Retest this please.

mjsax · 2019-02-13T01:37:54Z

https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2292/

java.lang.AssertionError: Unclean leader not elected
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at kafka.server.DynamicBrokerReconfigurationTest.testUncleanLeaderElectionEnable(DynamicBrokerReconfigurationTest.scala:488)

https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/19391/

kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
	at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply$mcV$sp(ZooKeeperClient.scala:242)
	at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:238)
	at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:238)
	at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
	at kafka.zookeeper.ZooKeeperClient.kafka$zookeeper$ZooKeeperClient$$waitUntilConnected(ZooKeeperClient.scala:238)
	at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96)
	at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1824)
	at kafka.zk.ZooKeeperTestHarness.setUp(ZooKeeperTestHarness.scala:59)
	at kafka.integration.KafkaServerTestHarness.setUp(KafkaServerTestHarness.scala:90)

and

java.lang.AssertionError: Consumed more records than expected expected:<4> but was:<6>
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at kafka.utils.TestUtils$.consumeRecords(TestUtils.scala:1262)
	at kafka.utils.TestUtils$.consumeTopicRecords(TestUtils.scala:1240)
	at kafka.metrics.MetricsTest.testBrokerTopicMetricsBytesInOut(MetricsTest.scala:128)

Retest this please.

vvcephei · 2019-02-13T14:09:19Z

Java 11 passed.
https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2318/

Java 8 failed on:
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/19417/

Error Message
java.lang.AssertionError: Received 0, expected at least 102
Stacktrace
java.lang.AssertionError: Received 0, expected at least 102
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at kafka.api.ConsumerBounceTest.kafka$api$ConsumerBounceTest$$receiveAndCommit(ConsumerBounceTest.scala:562)
	at kafka.api.ConsumerBounceTest$$anonfun$testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup$3.apply(ConsumerBounceTest.scala:383)
	at kafka.api.ConsumerBounceTest$$anonfun$testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup$3.apply(ConsumerBounceTest.scala:382)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup(ConsumerBounceTest.scala:382)

Retest this, please.

mjsax · 2019-02-13T22:20:16Z

It's a trivial change -- I'll try one more time and merge even if tests don't pass afterwards.

Retest this please.

vvcephei · 2019-02-14T16:32:04Z

Geez. Java 11 build failed again:
https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2345/

Error Message
java.lang.AssertionError: Expected an exception of type org.apache.kafka.common.errors.TimeoutException; got type org.apache.kafka.common.errors.SslAuthenticationException
Stacktrace
java.lang.AssertionError: Expected an exception of type org.apache.kafka.common.errors.TimeoutException; got type org.apache.kafka.common.errors.SslAuthenticationException
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at kafka.utils.TestUtils$.assertFutureExceptionTypeEquals(TestUtils.scala:1435)
	at kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts(AdminClientIntegrationTest.scala:1071)

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>

mjsax · 2019-02-14T18:04:12Z

Merged to trunk and cherry-picked to 2.2.

vvcephei · 2019-02-14T18:06:04Z

Thanks, @mjsax !

* ak/trunk: (45 commits) KAFKA-7487: DumpLogSegments misreports offset mismatches (apache#5756) MINOR: improve JavaDocs about auto-repartitioning in Streams DSL (apache#6269) KAFKA-7935: UNSUPPORTED_COMPRESSION_TYPE if ReplicaManager.getLogConfig returns None (apache#6274) KAFKA-7895: Fix stream-time reckoning for suppress (apache#6278) KAFKA-6569: Move OffsetIndex/TimeIndex logger to companion object (apache#4586) MINOR: add log indicating the suppression time (apache#6260) MINOR: Make info logs for KafkaConsumer a bit more verbose (apache#6279) KAFKA-7758: Reuse KGroupedStream/KGroupedTable with named repartition topics (apache#6265) KAFKA-7884; Docs for message.format.version should display valid values (apache#6209) MINOR: Save failed test output to build output directory MINOR: add test for StreamsSmokeTestDriver (apache#6231) MINOR: Fix bugs identified by compiler warnings (apache#6258) KAFKA-6474: Rewrite tests to use new public TopologyTestDriver [part 4] (apache#5433) MINOR: fix bypasses in ChangeLogging stores (apache#6266) MINOR: Make MockClient#poll() more thread-safe (apache#5942) MINOR: drop dbAccessor reference on close (apache#6254) KAFKA-7811: Avoid unnecessary lock acquire when KafkaConsumer commits offsets (apache#6119) KAFKA-7916: Unify store wrapping code for clarity (apache#6255) MINOR: Add missing Alter Operation to Topic supported operations list in AclCommand KAFKA-7921: log at error level for missing source topic (apache#6262) ...

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>

MINOR: drop dbAccessor reference on close

d46803a

mjsax approved these changes Feb 12, 2019

View reviewed changes

bbejeck approved these changes Feb 12, 2019

View reviewed changes

mjsax merged commit b12b7f0 into apache:trunk Feb 14, 2019

mjsax pushed a commit that referenced this pull request Feb 14, 2019

MINOR: drop dbAccessor reference on close (#6254)

faaa84e

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>

vvcephei deleted the close-db-accessor branch February 14, 2019 18:05

pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019

MINOR: drop dbAccessor reference on close (apache#6254)

8034296

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MINOR: drop dbAccessor reference on close#6254

MINOR: drop dbAccessor reference on close#6254
mjsax merged 1 commit intoapache:trunkfrom
vvcephei:close-db-accessor

vvcephei commented Feb 11, 2019

Uh oh!

vvcephei commented Feb 11, 2019

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax left a comment

Uh oh!

bbejeck left a comment

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax commented Feb 13, 2019

Uh oh!

vvcephei commented Feb 13, 2019

Uh oh!

mjsax commented Feb 13, 2019

Uh oh!

vvcephei commented Feb 14, 2019

Uh oh!

mjsax commented Feb 14, 2019

Uh oh!

vvcephei commented Feb 14, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

vvcephei commented Feb 11, 2019

Committer Checklist (excluded from commit message)

Uh oh!

vvcephei commented Feb 11, 2019

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax left a comment

Choose a reason for hiding this comment

Uh oh!

bbejeck left a comment

Choose a reason for hiding this comment

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax commented Feb 12, 2019

Uh oh!

mjsax commented Feb 13, 2019

Uh oh!

vvcephei commented Feb 13, 2019

Uh oh!

mjsax commented Feb 13, 2019

Uh oh!

vvcephei commented Feb 14, 2019

Uh oh!

mjsax commented Feb 14, 2019

Uh oh!

vvcephei commented Feb 14, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants