Skip to content

stdout/stderr log filtering for windows#718

Merged
sarahpeiffer merged 11 commits intoci_prodfrom
sarah/fluentbit_windows_fix
Mar 11, 2022
Merged

stdout/stderr log filtering for windows#718
sarahpeiffer merged 11 commits intoci_prodfrom
sarah/fluentbit_windows_fix

Conversation

@sarahpeiffer
Copy link
Contributor

@sarahpeiffer sarahpeiffer commented Mar 10, 2022

This PR adds the FILTER plugin for windows fluent bit container logs.

Validation/testing:
(tested on cluster with docker and containerd windows nodes. logging yaml applied outputting logs per second w/ 2k log size, 1,800 logs per minute)

  • Confirmed setting log_collection_settings.stdout enabled to false stops collection of stdout logs (but not stderr)
  • Confirmed setting log_collection_settings.stderr enabled to false stops collection of stderr logs (but not stdout)
  • Confirmed setting both stdout and stderr to false with no namespaces in the excluded list stops all logs
  • Confirmed setting log_collection_settings.stdout enabled to true and filtering out a stdout namespace works as expected
  • Switched between setting stdoutenabled to true and stdoutenabled to false multiple times with expected behavior
  • Checked that changing the omsagent-win image does not duplicate logs since fluent bit state is maintained (this was expected just confirming before release)
  • Checked that setting log_collection_settings.enrich_container_logs to true shows container image and container name as expected

This PR also includes some changes to fix build breaks for PRs that were already in ci_dev

@sarahpeiffer sarahpeiffer requested a review from a team as a code owner March 10, 2022 23:35
@ganga1980
Copy link
Contributor

This PR adds the FILTER plugin for windows fluent bit container logs.

Validation/testing: (tested on cluster with docker and containerd windows nodes. logging yaml applied outputting logs per second w/ 2k log size, 1,800 logs per minute)

  • Confirmed setting log_collection_settings.stdout enabled to false stops collection of stdout logs
  • Confirmed setting log_collection_settings.stderr enabled to false does not stop collection of stdout logs
  • Confirmed setting log_collection_settings.stdout enabled to true and filtering out a stdout namespace works as expected
  • Switched between setting stdoutenabled to true and stdoutenabled to false multiple times with expected behavior
  • Checked that changing the omsagent-win image does not duplicate logs since fluent bit state is maintained (this was expected just confirming before release)
  • Checked that setting log_collection_settings.enrich_container_logs to true shows container image and container name as expected

Thanks for thorough validation and explanation what's being tested.

ganga1980
ganga1980 previously approved these changes Mar 11, 2022
Copy link
Contributor

@ganga1980 ganga1980 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, please update release related changes as discussed.

@sarahpeiffer sarahpeiffer merged commit c71e963 into ci_prod Mar 11, 2022
jatakiajanvi12 pushed a commit that referenced this pull request Dec 2, 2022
* bug fix for stdout/stderr log filtering for windows

* fix build break (update go lang version, fix uuid vulnerability)

Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: Rashmi Chandrashekar <rashmy@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants