Skip to content

KAFKA-10084: Fix EosTestDriver end offset#8785

Merged
vvcephei merged 2 commits intoapache:trunkfrom
vvcephei:kafka-10084-fix-eos-test-end-offset
Jun 3, 2020
Merged

KAFKA-10084: Fix EosTestDriver end offset#8785
vvcephei merged 2 commits intoapache:trunkfrom
vvcephei:kafka-10084-fix-eos-test-end-offset

Conversation

@vvcephei
Copy link
Copy Markdown
Contributor

@vvcephei vvcephei commented Jun 2, 2020

Check the uncommitted end offset after the committed end offset,
so we can be sure never to miss a pending end-transaction marker.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@vvcephei vvcephei requested a review from mjsax June 2, 2020 19:12
@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jun 2, 2020

I ran the system tests, and they pass:

================================================================================
test_id:    kafkatest.tests.streams.streams_eos_test.StreamsEosTest.test_failure_and_recovery_complex.processing_guarantee=exactly_once
status:     PASS
run time:   13 minutes 14.424 seconds
--------------------------------------------------------------------------------
test_id:    kafkatest.tests.streams.streams_eos_test.StreamsEosTest.test_failure_and_recovery_complex.processing_guarantee=exactly_once_beta
status:     PASS
run time:   13 minutes 12.365 seconds
--------------------------------------------------------------------------------

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jun 2, 2020

Hey @mjsax , thanks for the help on this. Can you take a look at the PR?

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jun 2, 2020

oh, great:

00:14:19.016 > Task :streams:checkstyleTest
00:14:19.017 [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/kafka-pr-jdk11-scala2.13/streams/src/test/java/org/apache/kafka/streams/tests/EosTestDriver.java:47:8: Unused import - java.util.HashSet. [UnusedImports]
00:14:19.017 [ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/kafka-pr-jdk11-scala2.13/streams/src/test/java/org/apache/kafka/streams/tests/EosTestDriver.java:55:8: Unused import - java.util.Set. [UnusedImports]

mjsax
mjsax approved these changes Jun 2, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@mjsax mjsax left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jun 3, 2020

All test failures were unrelated (looks like some new flaky tests)

    kafka.server.GssapiAuthenticationTest.testServerAuthenticationFailure
    org.apache.kafka.clients.consumer.CooperativeStickyAssignorTest.testReassignmentWithRandomSubscriptionsAndChanges
    org.apache.kafka.clients.consumer.CooperativeStickyAssignorTest.testReassignmentWithRandomSubscriptionsAndChanges
    org.apache.kafka.clients.consumer.StickyAssignorTest.testReassignmentWithRandomSubscriptionsAndChanges
    org.apache.kafka.clients.consumer.StickyAssignorTest.testReassignmentWithRandomSubscriptionsAndChanges

@vvcephei vvcephei merged commit cb88be4 into apache:trunk Jun 3, 2020
@vvcephei vvcephei deleted the kafka-10084-fix-eos-test-end-offset branch June 3, 2020 16:13
vvcephei added a commit that referenced this pull request Jun 3, 2020
@vvcephei
KAFKA-10084: Fix EosTestDriver end offset (#8785)
432c540 
Check the uncommitted end offset after the committed end offset,
so we can be sure never to miss a pending end-transaction marker.

Reviewers: Matthias J. Sax <matthias@confluent.io>
@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Jun 3, 2020

cherry-picked to 2.6

ijuma added a commit that referenced this pull request Jun 4, 2020
@ijuma
Merge remote-tracking branch 'apache-github/trunk' into KIP-602_make-…
11277b2 
…use_all_dns_ips-as-default

* apache-github/trunk:
  KAFKA-9788; Use distinct names for transaction and group load time sensors (#8784)
  KAFKA-9514; The protocol generator generated useless condition when a field is made nullable and flexible version is used (#8793)
  MINOR: Update to Gradle 6.5 and tweak build jvm config (#8751)
  MINOR: Upgrade spotbugs and spotbugsPlugin (#8790)
  KAFKA-10089 The stale ssl engine factory is not closed after reconfigure (#8792)
  KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends (#8782)
  KAFKA-10084: Fix EosTestDriver end offset (#8785)
  KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests (#8786)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mjsax mjsax mjsax approved these changes

Assignees

No one assigned

Labels

streams

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vvcephei @mjsax