Skip to content

KAFKA-7921: log at error level for missing source topic#6262

Merged
guozhangwang merged 2 commits intoapache:trunkfrom
vvcephei:upgrade-shudown-log-level
Feb 13, 2019
Merged

KAFKA-7921: log at error level for missing source topic#6262
guozhangwang merged 2 commits intoapache:trunkfrom
vvcephei:upgrade-shudown-log-level

Conversation

@vvcephei
Copy link
Copy Markdown
Contributor

@vvcephei vvcephei commented Feb 12, 2019

This condition is a fatal error, so error level is warranted, to provide more context on why Streams shuts down.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Feb 12, 2019

@guozhangwang @mjsax ,

Is this what you all had in mind?

guozhangwang
guozhangwang reviewed Feb 12, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@guozhangwang guozhangwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I'd suggest that on the leader: https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java#L466

When setting the error code also add a log entry.

@mjsax mjsax added the streams label Feb 12, 2019
@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Feb 13, 2019

Java 8 failure unrelated:
https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/19433/

Error Message
java.lang.AssertionError: Expected an exception of type org.apache.kafka.common.errors.TimeoutException; got type org.apache.kafka.common.errors.SslAuthenticationException
Stacktrace
java.lang.AssertionError: Expected an exception of type org.apache.kafka.common.errors.TimeoutException; got type org.apache.kafka.common.errors.SslAuthenticationException
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at kafka.utils.TestUtils$.assertFutureExceptionTypeEquals(TestUtils.scala:1434)
	at kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts(AdminClientIntegrationTest.scala:1071)

@vvcephei
Copy link
Copy Markdown
Contributor Author

vvcephei commented Feb 13, 2019

Ok, @guozhangwang , I've added the log on the leader as well. I wasn't sure if it'd be helpful to log any additional metadata, aside from the topic name.

@mjsax
Copy link
Copy Markdown
Member

mjsax commented Feb 13, 2019

https://builds.apache.org/job/kafka-pr-jdk8-scala2.11/19433/

java.lang.AssertionError: Expected an exception of type org.apache.kafka.common.errors.TimeoutException; got type org.apache.kafka.common.errors.SslAuthenticationException
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at kafka.utils.TestUtils$.assertFutureExceptionTypeEquals(TestUtils.scala:1434)
	at kafka.api.AdminClientIntegrationTest.testMinimumRequestTimeouts(AdminClientIntegrationTest.scala:1071)

Retest this please.

mjsax
mjsax approved these changes Feb 13, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@mjsax mjsax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@guozhangwang guozhangwang merged commit 1aae604 into apache:trunk Feb 13, 2019
guozhangwang pushed a commit that referenced this pull request Feb 13, 2019
@vvcephei @guozhangwang
KAFKA-7921: log at error level for missing source topic (#6262)
b4a20e8 
This condition is a fatal error, so error level is warranted, to provide more context on why Streams shuts down.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
@guozhangwang
Copy link
Copy Markdown
Contributor

guozhangwang commented Feb 13, 2019

Also cherry-picked to 2.2.

jarekr pushed a commit to confluentinc/kafka that referenced this pull request Apr 18, 2019
@lbradstreet
Merge remote-tracking branch 'ak/trunk' into cos-trunk
0eacf48 
* ak/trunk: (45 commits)
  KAFKA-7487: DumpLogSegments misreports offset mismatches (apache#5756)
  MINOR: improve JavaDocs about auto-repartitioning in Streams DSL (apache#6269)
  KAFKA-7935: UNSUPPORTED_COMPRESSION_TYPE if ReplicaManager.getLogConfig returns None (apache#6274)
  KAFKA-7895: Fix stream-time reckoning for suppress (apache#6278)
  KAFKA-6569: Move OffsetIndex/TimeIndex logger to companion object  (apache#4586)
  MINOR: add log indicating the suppression time (apache#6260)
  MINOR: Make info logs for KafkaConsumer a bit more verbose (apache#6279)
  KAFKA-7758: Reuse KGroupedStream/KGroupedTable with named repartition topics (apache#6265)
  KAFKA-7884; Docs for message.format.version should display valid values (apache#6209)
  MINOR: Save failed test output to build output directory
  MINOR: add test for StreamsSmokeTestDriver (apache#6231)
  MINOR: Fix bugs identified by compiler warnings (apache#6258)
  KAFKA-6474: Rewrite tests to use new public TopologyTestDriver [part 4] (apache#5433)
  MINOR: fix bypasses in ChangeLogging stores (apache#6266)
  MINOR: Make MockClient#poll() more thread-safe (apache#5942)
  MINOR: drop dbAccessor reference on close (apache#6254)
  KAFKA-7811: Avoid unnecessary lock acquire when KafkaConsumer commits offsets (apache#6119)
  KAFKA-7916: Unify store wrapping code for clarity (apache#6255)
  MINOR: Add missing Alter Operation to Topic supported operations list in AclCommand
  KAFKA-7921: log at error level for missing source topic (apache#6262)
  ...
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
@vvcephei @pengxiaolong
KAFKA-7921: log at error level for missing source topic (apache#6262)
2c047c6 
This condition is a fatal error, so error level is warranted, to provide more context on why Streams shuts down.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mjsax mjsax mjsax approved these changes

@guozhangwang guozhangwang guozhangwang approved these changes

Assignees

No one assigned

Labels

streams

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vvcephei @mjsax @guozhangwang