MINOR: set log4j.logger.kafka and all Config logger levels to ERROR for Streams tests#11823
Merged
ableegoldman merged 2 commits intoapache:trunkfrom Mar 2, 2022
Conversation
showuon
approved these changes
Mar 1, 2022
Member
showuon
left a comment
showuon
left a comment
There was a problem hiding this comment.
+1 to reduce the logs for better debugging the flaky tests on jenkins. After all, user can always change the log level when running in local env. Thanks.
guozhangwang
approved these changes
Mar 1, 2022
Contributor
guozhangwang
left a comment
guozhangwang
left a comment
There was a problem hiding this comment.
Thanks @ableegoldman . I agree that skipping INFO at kafka server logs is a good idea.
Just one question on one log4j line, otherwise LGTM.
|
|
||
| log4j.logger.kafka=INFO | ||
| log4j.logger.kafka=ERROR | ||
| log4j.logger.state=ERROR |
Contributor
There was a problem hiding this comment.
Not sure what does "log4j.logger.state" mean here? Or did you mean to write "log4j.logger.state.change.logger"?
Member
Author
There was a problem hiding this comment.
Ah, yeah, I guess there isn't anything else under log4j.logger.state besides the state.change.logger -- I'll update this
Member
Author
|
Merged to trunk -- thanks @showuon and @guozhangwang ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pretty much any time we have an integration test failure that's flaky or only exposed when running on Jenkins through the PR builds, it's impossible to debug if it cannot be reproduced locally as the logs attached to the test results have truncated the entire useful part of the logs. This is due to the logs being flooded at the beginning of the test when the Kafka cluster is coming up, eating up all of the allotted characters before we even get to the actual Streams test. Setting
log4j.logger.kafkatoERRORgreatly improves the situation and cuts down on most of the excessive logging in my local runs. To improve things even more and have some hope of getting the part of the logs we actually need, I also set the loggers for all of the Config objects to ERROR, as these print out the value of every single config (of which there are a lot) and are not useful as we can easily figure out what the configs were if necessary by just inspecting the test locally.Hopefully this will help us debug and then fix some of the flaky tests within Streams that show up on PR builds