Skip to content

Don't print statsd log messages unless debug / verbose mode is used #4441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Kami merged 8 commits into from
Nov 14, 2018
Merged

Don't print statsd log messages unless debug / verbose mode is used #4441

Kami merged 8 commits into from
Nov 14, 2018

Conversation

@Kami
Copy link
Member

@Kami Kami commented Nov 13, 2018

This pull request updates our code so we don't print log messages generated by statsd client (when statsd metrics driver is used) if debug / verbose mode is not enabled.

Statsd client logs all the operations under INFO by default and that causes a lot of noise in the log files. Those log messages are not desired unless running in debug / verbose mode.

This also fixes st2-register-content "regression".

(virtualenv) vagrant@ubuntu-xenial:/data/stanley$ python ./st2common/bin/st2-register-content --register-actions --config-file conf/st2.dev.conf 
2018-11-13 16:06:55,826 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "None".
2018-11-13 16:06:55,832 INFO [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "None".
2018-11-13 16:06:56,105 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 22.95600000ms
2018-11-13 16:06:56,105 INFO [-] st2.amqp.publish.update: 23.81000000ms
2018-11-13 16:06:56,128 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 8.55600000ms
2018-11-13 16:06:56,129 INFO [-] st2.amqp.publish.update: 9.16700000ms
2018-11-13 16:06:56,159 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 1.96500000ms
2018-11-13 16:06:56,159 INFO [-] st2.amqp.publish.update: 2.34900000ms
2018-11-13 16:06:56,185 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 7.85500000ms
2018-11-13 16:06:56,186 INFO [-] st2.amqp.publish.update: 8.47800000ms
2018-11-13 16:06:56,212 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 3.88400000ms
2018-11-13 16:06:56,213 INFO [-] st2.amqp.publish.update: 4.22300000ms
2018-11-13 16:06:56,236 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 3.19800000ms
2018-11-13 16:06:56,237 INFO [-] st2.amqp.publish.update: 3.63800000ms
2018-11-13 16:06:56,259 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 5.47000000ms
2018-11-13 16:06:56,259 INFO [-] st2.amqp.publish.update: 6.28000000ms
2018-11-13 16:06:56,283 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 3.02200000ms
2018-11-13 16:06:56,284 INFO [-] st2.amqp.publish.update: 3.39700000ms
2018-11-13 16:06:56,307 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 5.45000000ms
2018-11-13 16:06:56,307 INFO [-] st2.amqp.publish.update: 6.59100000ms
2018-11-13 16:06:56,331 INFO [-] st2.amqp.pool_publisher.publish_with_retries.st2.trigger: 4.71300000ms
2018-11-13 16:06:56,332 INFO [-] st2.amqp.publish.update: 5.28400000ms
2018-11-13 16:06:56,333 INFO [-] =========================================================
2018-11-13 16:06:56,333 INFO [-] ############## Registering runners ######################
2018-11-13 16:06:56,333 INFO [-] =========================================================
2018-11-13 16:06:56,769 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
2018-11-13 16:06:56,790 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
2018-11-13 16:06:57,700 INFO [-] st2.st2.register.runners: 1366.99800000ms
2018-11-13 16:06:57,701 INFO [-] Registered 18 runners.
2018-11-13 16:06:57,702 INFO [-] =========================================================
2018-11-13 16:06:57,702 INFO [-] ############## Registering actions ######################
2018-11-13 16:06:57,703 INFO [-] =========================================================
2018-11-13 16:07:06,978 INFO [-] st2.st2.register.actions: 9274.77600000ms
2018-11-13 16:07:06,978 INFO [-] Registered 322 actions.

After:

(virtualenv) vagrant@ubuntu-xenial:/data/stanley$ python ./st2common/bin/st2-register-content --register-actions --config-file conf/st2.dev.conf
2018-11-13 16:35:20,581 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "None".
2018-11-13 16:35:20,587 INFO [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "None".
2018-11-13 16:35:21,055 INFO [-] =========================================================
2018-11-13 16:35:21,055 INFO [-] ############## Registering runners ######################
2018-11-13 16:35:21,056 INFO [-] =========================================================
2018-11-13 16:35:21,290 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
2018-11-13 16:35:21,314 INFO [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
2018-11-13 16:35:22,208 INFO [-] Registered 18 runners.
2018-11-13 16:35:22,209 INFO [-] =========================================================
2018-11-13 16:35:22,210 INFO [-] ############## Registering actions ######################
2018-11-13 16:35:22,210 INFO [-] =========================================================
2018-11-13 16:35:29,783 INFO [-] Registered 322 actions.

NOTE: If user runs StackStorm service with --debug flag or st2-register-content script with --verbose flags those messages will still be printed.

Kami added 2 commits November 13, 2018 17:31
@Kami
Don't print statsd log messages unless debug / verbose mode is used.
701eeb2 
NOTE: By default statsd client logs all the messages under INFO which
causes a lot of noise.
@Kami
Update logger format string.
c19887e
@Kami Kami added this to the 3.0.0 milestone Nov 13, 2018
@Kami
Copy link
Member Author

Kami commented Nov 13, 2018

While at it, I also pushed a change to ignore "useless" "Generating grammar tables" log messages printed out by networkx which uses 2to3 library - 5515a9e.

arm4b
arm4b approved these changes Nov 13, 2018
View reviewed changes
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@arm4b arm4b added the bug label Nov 13, 2018
@Kami Kami merged commit 17ee57f into master Nov 14, 2018
@Kami Kami deleted the update_statsd_driver_logging branch November 14, 2018 13:26
@lakshmi-kannan
Copy link
Contributor

lakshmi-kannan commented Nov 15, 2018

👍

@Kami
Copy link
Member Author

Kami commented Jan 24, 2019

Those messages are still appearing under INFO log level on CICD, looking into it to figure out why.

@Kami Kami mentioned this pull request Jan 24, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@arm4b arm4b arm4b approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

bug

Projects

None yet

Milestone

2.10.0

Development

Successfully merging this pull request may close these issues.

4 participants

@Kami @lakshmi-kannan @arm4b