Skip to content

MINOR: Improve usage of LogCaptureAppender#8508

Merged
mjsax merged 2 commits intoapache:trunkfrom
mjsax:minor-imporve-log-capturing
Apr 21, 2020
Merged

MINOR: Improve usage of LogCaptureAppender#8508
mjsax merged 2 commits intoapache:trunkfrom
mjsax:minor-imporve-log-capturing

Conversation

@mjsax
Copy link
Copy Markdown
Member

@mjsax mjsax commented Apr 18, 2020

I started a "small" cleanup, and it ended up to become a "big" cleanup...

  • limit the usage of LogCaptureAppender to the class under test (if possible)
  • ensure that the registered appended is unregistered (via try-with-resource)
  • limit the "scope" of creating/unregistering the appended to a minimum
  • cleanup all warning of all classes touched
  • some code reformatting
  • some class renaming

Call for review @vvcephei

@mjsax mjsax force-pushed the minor-imporve-log-capturing branch from 422aae9 to ccab82c Compare April 18, 2020 00:30
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an actual change -- in all other processor, we use the "top level class" for the logger and not the Processor class -- changing this for alignment.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because unused.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because unused.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because the parameter is variadic

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is only called once and thus its merged into the main test method

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't capture any debug logs and thus it's getting removed

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is only called once and thus its merged into the main test method

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Apr 18, 2020

Did you consider using try with resources?

Copy link
Copy Markdown
Contributor

@vvcephei vvcephei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @mjsax ! This is awesome. I'm wondering if we can formalize the pattern you've adopted by implementing AutoCloseable in LogCaptureAppender, and making the close method unregister the appender, so we can convert all those try/finally blocks to try-with-resources? Of course, we'd have to move the assertions inside the block, which we have sometimes and not other times, but it doesn't seem like a big deal either way.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this appender could also be scoped to the class that logs the message we want to capture, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not; I tied it and it fails. The issue is, we would want to register the LogCaptureAppender only for KafkaStreams.class but at this point in the test, the corresponding logger does not exist yet: it's only created in the constructor call, that we actually test. Does this make sense?

Copy link
Copy Markdown
Contributor

@vvcephei vvcephei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @mjsax ! It seems like we should look into the try-with-resources thing, unless you already rejected that option for some reason. I also registered one complaint below.

But I don't want to block on either one, if you really prefer the current code.

Other than those two issues, it's a really nice cleanup. Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge fan of this particular "clean up". We're trading a tightly defined scope for de-duplicating the trivial line results.add(deserialization). It doesn't seem like a good trade to me.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel "strong" about it -- it was an IntelliJ warning/suggestion to "share common code".

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 19, 2020

I did not consider using try-with-resource. Great idea!

@mjsax mjsax force-pushed the minor-imporve-log-capturing branch from 8a9d0aa to 81d042b Compare April 19, 2020 08:47
@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 19, 2020

Updated this. Call for review.

@mjsax mjsax force-pushed the minor-imporve-log-capturing branch from b594d46 to 4b65333 Compare April 20, 2020 18:17
Copy link
Copy Markdown
Contributor

@vvcephei vvcephei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @mjsax ; this is awesome!

I just had one question about renaming the test thread. Otherwise, it LGTM!

Comment on lines +90 to +92
// When executing on Jenkins, the thread name is set to an unknown value,
// hence, we need to set it explicitly to make our log-assertions pass
Thread.currentThread().setName(threadName);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this might be surprising. Won't this cause the test executor thread to be called "threadName" from now until the end of the build?

Do you think we could instead get the currentThread's name and use that in the assertions? Or otherwise make the assertions agnostic to the name of the thread?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting the name is much better! Should have done this for the beginning on...

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 20, 2020

Java 8 passed.
Java11: kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup
Java14 failed due to #8519 (already merged).

Retest this please.

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 20, 2020

Updated this PR. Will merge after Jenkins passe.

Copy link
Copy Markdown
Contributor

@vvcephei vvcephei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks.

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 21, 2020

Java 8: org.apache.kafka.streams.integration.StandbyTaskEOSIntegrationTest.surviveWithOneTaskAsStandby[exactly_once_beta]
Java 11: org.apache.kafka.connect.mirror.MirrorConnectorsIntegrationTest.testReplication
Java 14: org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQuerySpecificActivePartitionStores

Retest this please.

@mjsax
Copy link
Copy Markdown
Member Author

mjsax commented Apr 21, 2020

Java 8 and Java 11 passed.
Java 14: failed with unknown error...

@mjsax mjsax merged commit 11d8ef7 into apache:trunk Apr 21, 2020
@mjsax mjsax deleted the minor-imporve-log-capturing branch April 21, 2020 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants