Skip to content

Add pre-check for heavy debug log#15706

Merged
kfaraz merged 7 commits intoapache:masterfrom
fectrain:add_prior_check_for_heavy_debug_log
Feb 29, 2024
Merged

Add pre-check for heavy debug log#15706
kfaraz merged 7 commits intoapache:masterfrom
fectrain:add_prior_check_for_heavy_debug_log

Conversation

@fectrain
Copy link
Copy Markdown
Contributor

@fectrain fectrain commented Jan 17, 2024

Fixes #15704

Description

Add pre-check for heavy debug log

Release note

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Comment thread sql/src/main/java/org/apache/druid/sql/calcite/parser/DruidSqlParserUtils.java Outdated
)
);
LOG.debug("Index status response" + response.getContent());
if (LOG.isDebugEnabled()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose the changes in this class are not really needed as it is just a test class anyway. The changes wouldn't really have any impact.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okie, already reverted this one

@fectrain fectrain requested a review from kfaraz January 18, 2024 03:46
Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes @fectrain !

@kfaraz
Copy link
Copy Markdown
Contributor

kfaraz commented Jan 18, 2024

@fectrain , there seem to be a few code coverage issues. Please see if you can address them by adding /updating unit tests.

@fectrain
Copy link
Copy Markdown
Contributor Author

fectrain commented Jan 18, 2024

Hello, @kfaraz! Thank you for your review and suggestions.

I'm also thinking about this but haven't found a straightforward solution to address this coverage issue. One option might be to enable debug-level logging for the entire unit test package through modifications to the log property files, which I think may not be quite necessary. 😄

As this change only involves adding a condition check and adjusting the indentation of the original log line, which should not able to alter the original logic or introduce exceptions. So I think merging this change is fine. What do you think?

@kfaraz
Copy link
Copy Markdown
Contributor

kfaraz commented Jan 19, 2024

@fectrain , yes, I agree that the changes here are simple enough to not cause any issue. But the problem is that it would also reduce the overall coverage of these classes because of the lines that are now inside the log.isDebugEnabled if clause. That might cause further build failures down the line.

Enabling debug logging in tests doesn't seem to be a great option either as it would create unnecessary logs when running tests.

See if you can reduce the number of lines in some cases. For example, the following code:

log.debug(                             
    "ZKNode created for server to [%s] %s [%s]",
    basePath,
    segmentHolder.getAction(),
    segmentHolder.getSegmentIdentifier()
);

can be rewritten as

log.debug(                             
    "ZKNode created for server to [%s] %s [%s]",
    basePath, segmentHolder.getAction(), segmentHolder.getSegmentIdentifier()
);

without compromising on readability.

I know it's not a great solution but should help with the coverage to some extent.

fectrain and others added 3 commits January 22, 2024 11:31
@fectrain fectrain force-pushed the add_prior_check_for_heavy_debug_log branch 2 times, most recently from 92a659f to fe25e53 Compare January 22, 2024 06:40
@fectrain fectrain force-pushed the add_prior_check_for_heavy_debug_log branch from fe25e53 to ac203d7 Compare January 22, 2024 11:31
@fectrain
Copy link
Copy Markdown
Contributor Author

Hi, @kfaraz, thanks for the advice.
I just inlined some logs as suggested to improve the line coverage rate, can see if now it's ok~

@asdf2014 asdf2014 requested a review from kfaraz January 30, 2024 03:07
Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience, @fectrain ! I have left some final comments. Please let me know if you would be available to address these. If not, I will commit the changes myself and merge the PR.

@FrankChen021
Copy link
Copy Markdown
Member

Hi @fectrain Can you pick up this PR again to resolve the conflict so that we can merge it?

@fectrain
Copy link
Copy Markdown
Contributor Author

oh, I might have missed this one, let me refine on it, thanks for reminding @FrankChen021

@fectrain
Copy link
Copy Markdown
Contributor Author

Hi, @kfaraz , big thanks for all your advice on this. I've applied the changes you suggested and hope they're good to go for merging~ 🙏

@kfaraz
Copy link
Copy Markdown
Contributor

kfaraz commented Feb 29, 2024

The failing checks are all due to coverage of the new log lines. The coverage requirement is difficult to satisfy without enabling debug logging for all the relevant tests, which seems overkill.

Going forward with the merge.

@kfaraz kfaraz merged commit e0bce0e into apache:master Feb 29, 2024
317brian pushed a commit to 317brian/druid that referenced this pull request Feb 29, 2024
Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>
Co-authored-by: Benedict Jin <asdf2014@apache.org>
asdf2014 added a commit that referenced this pull request Mar 4, 2024
* docs: add mermaid diagram support

* fix crash when parsing data in data loader that can not be parsed (#15983)

* update jetty to address CVE (#16000)

* Concurrent replace should work with supervisors using concurrent locks (#15995)

* Concurrent replace should work with supervisors using concurrent locks

* Ignore supervisors with useConcurrentLocks set to false

* Apply feedback

* Add pre-check for heavy debug logs (#15706)

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>
Co-authored-by: Benedict Jin <asdf2014@apache.org>

* Remove helm paths from CodeQL config (#16006)

* docs: mention acid-compliance for metadb

---------

Co-authored-by: Vadim Ogievetsky <vadim@ogievetsky.com>
Co-authored-by: Jan Werner <105367074+janjwerner-confluent@users.noreply.github.com>
Co-authored-by: AmatyaAvadhanula <amatya.avadhanula@imply.io>
Co-authored-by: Sensor <fectrain@outlook.com>
Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>
Co-authored-by: Benedict Jin <asdf2014@apache.org>
@adarshsanjeev adarshsanjeev added this to the 30.0.0 milestone May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhance performance by checking isDebugEnabled() before calling debug(...)

5 participants