KAFKA-2055; Fix transient ConsumerBounceTest.testSeekAndCommitWithBro…

[lvfangmin]

…kerFailures failure;

[hachikuji]

Do you think it's worthwhile to mention the JIRA for the issue that this check gets around (i.e. KAFKA-1211)?

[lvfangmin]

Yes, I think it's better to track this work around, as we may have to remove this in the future.

[guozhangwang]

I think KAFKA-1211 is not related to this check, but KAFKA-2334?

[hachikuji]

@guozhangwang Perhaps I was mistaken, but I thought the issue was that the writes were getting lost before the first seekToEnd, which means that it fails to find the expected position. I don't think it was because of inconsistency in the ordering of offsets visible to the user.

[guozhangwang]

@hachikuji the write was not lost, but just that the HW on the new leader was older than on the new leader, and that seekToEnd is guarded by the HW of the leader. I think it is irrelevant to the messages on the replicas.

[hachikuji]

@guozhangwang That makes sense, thanks for the explanation. But is it not also possible that the writes could get lost in a scenario like that of KAFKA-1211 where the log gets truncated to the HW after a second leader failure? That would also reveal itself in the first seekToEnd returning an earlier than expected offset.

[asfbot]

kafka-trunk-git-pr #50 FAILURE
Looks like there's a problem with this pull request

[guozhangwang]

@lvfangmin I saw the following test failure from git-pr #50:

kafka.api.ProducerSendTest > testCloseWithZeroTimeoutFromCallerThread FAILED
kafka.common.InconsistentBrokerIdException: Configured brokerId 0 doesn't match stored brokerId 1 in meta.properties
at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:474)
at kafka.server.KafkaServer.startup(KafkaServer.scala:134)
at kafka.utils.TestUtils$.createServer(TestUtils.scala:126)
at kafka.integration.KafkaServerTestHarness$$anonfun$setUp$1.apply(KafkaServerTestHarness.scala:59)
at kafka.integration.KafkaServerTestHarness$$anonfun$setUp$1.apply(KafkaServerTestHarness.scala:59)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.Iterator$class.foreach(Iterator.scala:750)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1202)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at kafka.integration.KafkaServerTestHarness$class.setUp(KafkaServerTestHarness.scala:59)
at kafka.api.ProducerSendTest.setUp(ProducerSendTest.scala:53)

Could you rerun the tests and see if they are transient? If yes we can file a separate JIRA for it.

[lvfangmin]

@guozhangwang Running kafka.api.ProducerSendTest.testCloseWithZeroTimeoutFromCallerThread, will fire a separate JIRA if it's a transient one.

[lvfangmin]

Encounter another error while running this test after 15 minutes:

kafka.api.ProducerSendTest > testCloseWithZeroTimeoutFromCallerThread STANDARD_OUT
[2015-07-24 23:13:05,148] WARN fsync-ing the write ahead log in SyncThread:0 took 1084ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide (org.apache.zookeeper.s
erver.persistence.FileTxnLog:334)

kafka.api.ProducerSendTest > testCloseWithZeroTimeoutFromCallerThread FAILED
java.lang.AssertionError: No request is complete.
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:44)
at kafka.api.ProducerSendTest$$anonfun$testCloseWithZeroTimeoutFromCallerThread$1.apply$mcVI$sp(ProducerSendTest.scala:343)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at kafka.api.ProducerSendTest.testCloseWithZeroTimeoutFromCallerThread(ProducerSendTest.scala:340)
Gradle Test Executor 2 finished executing tests.

Will run once more to verify.

[guozhangwang]

@lvfangmin I saw your filed another JIRA for this error, did you find it transient with and without your patch?

[lvfangmin]

@guozhangwang The transient error occurred when with/without my patch. It doesn't related with my patch, so it's safe to merge it.

[guozhangwang]

LGTM