Expose an Aggregator to Python checks by masci · Pull Request #2 · DataDog/datadog-agent

masci · 2016-05-30T17:35:37Z

Variation of the master design: expose a Go function to Python and let checks call aggregator's functions directly, skipping the JSON serialization of the results.

Using this approach we don't need anymore the Result processing logic, so I got rid of the dedicated channel as well as the CheckResult type.

Notice: ~~tests were not updated, will do it as soon as the current design will be approved.~~ tests updated

unpack python list containing tags remove results processor logic

olivielpeau · 2016-06-09T15:55:09Z

First time I'm looking at the new collector, but this LGTM! 👍

[test/two] Update init test with new six extension interface

Fix partial evaluation and add stringer interface to evaluators

Fix rename on older kernels

#7605)

Added reference to the upgraded gohai commit (#2)

Code review related changes #2

# This is the 1st commit message: [build] Fork rules_multitool to our own extension. Import the core parts of rules_multitool but with appropriate modifications for our needs. - Re-root things so this is a local extension, rather than a distinct module. - remove the ability to use .netrc for authentication. - remove the :cwd and :workspace_root variations. - There are equivalent workarounds using `--run_under="cd <path> &&"` - Using `bazel run` is usually not the best practice. Rules should use `$(location <tool>)`. If a tool is so common that people need to call it anywhere, at any time, then it should be in their path. - The code is left commented out. Ready to be enabled if a good case is presented. - Remove the support for WORKSPACE. Next steps: - Change download structure so the exeuctable name is the tool, rather than `"executable"`. - Stop passing the attributes of each binary around as json blobs to be encoded and decoded. - Make `:path` feature that will print the full execution path. # This is the commit message #2: Update bazel/multitool/extension.bzl Yeah. That's a better name. Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #3: Update bazel/multitool/extension.bzl Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #4: Update bazel/multitool/private/templates.bzl Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #5: render

# This is the 1st commit message: [build] Fork rules_multitool to our own extension. Import the core parts of rules_multitool but with appropriate modifications for our needs. - Re-root things so this is a local extension, rather than a distinct module. - remove the ability to use .netrc for authentication. - remove the :cwd and :workspace_root variations. - There are equivalent workarounds using `--run_under="cd <path> &&"` - Using `bazel run` is usually not the best practice. Rules should use `$(location <tool>)`. If a tool is so common that people need to call it anywhere, at any time, then it should be in their path. - The code is left commented out. Ready to be enabled if a good case is presented. - Remove the support for WORKSPACE. Next steps: - Change download structure so the exeuctable name is the tool, rather than `"executable"`. - Stop passing the attributes of each binary around as json blobs to be encoded and decoded. - Make `:path` feature that will print the full execution path. # This is the commit message #2: Update bazel/multitool/extension.bzl Yeah. That's a better name. Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #3: Update bazel/multitool/extension.bzl Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #4: Update bazel/multitool/private/templates.bzl Co-authored-by: Joseph Gette <jgettepost@gmail.com> # This is the commit message #5: render # This is the commit message #6: iff # This is the commit message #7: commit # This is the commit message #8: lintyfresth

# This is the 1st commit message: Add ability to detect ship_source_offer # This is the commit message #2: get rid of merge ghost # This is the commit message #3: buildifier woes

V0.1.1

### What does this PR do? This syncs additional code from the original PAR into the datadog agent ### Motivation ### Describe how you validated your changes Code is not called yet ### Additional Notes Previous PR : #43412 More PRs to come Co-authored-by: gabriel.plassard <gabriel.plassard@datadoghq.com>

# This is the 1st commit message: Keep our own copy of cacert.pem - Replace omnibus fetch from upstream with that static copy. - Include text in the BUILD file about how we check for new upstream versions. - Add explanation of why we have this. https://datadoghq.atlassian.net/browse/ABLD-169 # This is the commit message #2: just use copy for windows # This is the commit message #3: qmarks # This is the commit message #4: omnibus is to blame

# This is the 1st commit message: Keep our own copy of cacert.pem - Replace omnibus fetch from upstream with that static copy. - Include text in the BUILD file about how we check for new upstream versions. - Add explanation of why we have this. https://datadoghq.atlassian.net/browse/ABLD-169 # This is the commit message #2: just use copy for windows # This is the commit message #3: qmarks # This is the commit message #4: omnibus is to blame # This is the commit message #5: maybe # This is the commit message #6: add back in default version # This is the commit message #7: drop livestream on debug # This is the commit message #8: You're kidding, :live_stream? # This is the commit message #9: srsly # This is the commit message #10: just copy on windows # This is the commit message #11: cwd with copy probably does not work # This is the commit message #12: just give up on pkg_install for certs # This is the commit message #13: drop unneded pkg_install targets # This is the commit message #14: - use cwd to make it a little cleaner - update cert to 2025-09-09 # This is the commit message #15: comma # This is the commit message #16: Revert use of cwd on copy. It doesn't matter if it is ugly or not. We are going to delete it this quarter anyway.

…ws profiles (#45281) **Note:** This is a reimplementation of [this PR](#44488), which was reverted [here](#45200). This fixes the failing CI test. Working CI tests: [Job 1](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1370634331), [Job 2](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1370634332) ### What does this PR do? - Adds `_dd.origin:appservice` tag to profiles for Azure App Service Windows Apps - The tag gets passed via `apm_config.additional_profile_tags`, which is set in [this PR](DataDog/datadog-aas-extension#419) in the AAS Site Extension - Refactors how extra tags get added to profiles - Turns AzureServerlessTags into a map called `AdditionalProfileTags` for clearer future use The only changes between this PR and the reverted PR is commit [ef27a8a](ef27a8a), which fixes the CI test ([failed job #1](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1356878378), [failed job #2](https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1356827817)), and dcc9ad9, which renames `azureServerlessTags` to be cloud-agnostic as `_dd.origin` gets set for profiles in Google Cloud Run Services, Jobs, and Functions, not just Azure products. ### Motivation - This will help the profiling backend filter out and track serverless profiles for billing purposes. [See related PR](#44361) ### Describe how you validated your changes Deployed Node.js and .NET Azure App Service Windows Apps with the Site Extension with `DD_PROFILING_ENABLED=true` and looked at network request to validate that `_dd.origin` is set correctly 1. Downloaded `datadog-trace-agent.exe` and `dogstatsd.exe` from the `package_build > windows_zip_agent_binaries_x64-a7` job 2. Stopped an existing AAS Windows app using the site extension 3. In Kudu, replaced `datadog-trace-agent.exe` and `dogstatsd.exe` with the dev version 4. Copied and pasted the changes in `datadog.yaml` from the related [AAS Extension PR](DataDog/datadog-aas-extension#419) over 5. Started the app 6. Saw `_dd.origin` in the profile tags! <img width="204" height="127" alt="image" src="https://github.com/user-attachments/assets/6a08a29c-be18-4604-9d6f-af7b43c87fa7" /> - Also tested with an Azure Container App to make sure the refactoring didn't affect anything there, and tested in GCP environments to verify the `_dd.origin` tag gets set in profiles Co-authored-by: kathie.huang <kathie.huang@datadoghq.com>

### What does this PR do? Skip the SSH session patcher and add a test to illustrate the current issue. In addition, adds the possibility to check specific fields in the json returned for ssh_session events. ### Motivation The retry mechanism could cause the agent to send no more than one event per minute if an SSH session was not properly resolved. Previously, the event was not sent and the agent would wait one minute before sending it with the `unknown` type. However, this `authtype` would never be resolved because the session was initialized before the agent started processing events. As a result, every subsequent SSH event would wait one minute for nothing, causing a significant delay in agent events, potentially blocking all the other events. ### Describe how you validated your changes Added a test that illustrate the issue : `TestSSHUserSessionBlocking` With this change, the ssh_session event is now sent with `authtype` set to `unknown` and directly sent. Error without commenting the patcher : ``` Error: Received unexpected error: All attempts fail: #1: not found #2: not found #3: not found #4: not found #5: not found #6: not found #7: not found #8: not found #9: not found #10: not found #11: not found #12: not found #13: not found #14: not found #15: not found #16: not found #17: not found #18: not found #19: not found #20: not found #21: not found #22: not found #23: not found #24: not found #25: not found #26: not found #27: not found #28: not found #29: not found #30: not found Test: TestSSHUserSessionBlocking/second_ssh_no_auth ``` Co-authored-by: theo.putegnat <theo.putegnat@datadoghq.com>

Skip the SSH session patcher and add a test to illustrate the current issue. In addition, adds the possibility to check specific fields in the json returned for ssh_session events. ### Motivation The retry mechanism could cause the agent to send no more than one event per minute if an SSH session was not properly resolved. Previously, the event was not sent and the agent would wait one minute before sending it with the `unknown` type. However, this `authtype` would never be resolved because the session was initialized before the agent started processing events. As a result, every subsequent SSH event would wait one minute for nothing, causing a significant delay in agent events, potentially blocking all the other events. ### Describe how you validated your changes Added a test that illustrate the issue : `TestSSHUserSessionBlocking` With this change, the ssh_session event is now sent with `authtype` set to `unknown` and directly sent. Error without commenting the patcher : ``` Error: Received unexpected error: All attempts fail: #1: not found #2: not found #3: not found #4: not found #5: not found #6: not found #7: not found #8: not found #9: not found #10: not found #11: not found #12: not found #13: not found #14: not found #15: not found #16: not found #17: not found #18: not found #19: not found #20: not found #21: not found #22: not found #23: not found #24: not found #25: not found #26: not found #27: not found #28: not found #29: not found #30: not found Test: TestSSHUserSessionBlocking/second_ssh_no_auth ``` Co-authored-by: theo.putegnat <theo.putegnat@datadoghq.com> (cherry picked from commit 40d1f09) ___ Co-authored-by: Théo Putegnat <theo.putegnat@datadoghq.com>

expose aggregator to the Python world

79c29a5

unpack python list containing tags remove results processor logic

masci force-pushed the massi/expose_aggregator branch from 5f1c755 to 79c29a5 Compare May 31, 2016 21:57

Massimiliano Pippi added 6 commits June 2, 2016 10:38

ignore coverage config file

36fd4f3

added test launcher

6918538

fixed checks tests

82e1abf

fixed TestRun function

f0ec7cd

initialize the aggregator in the test runner

78d2878

avoid talking to the aggregator during unit tests

71fe592

masci merged commit 18bbf41 into master Jun 9, 2016

masci deleted the massi/expose_aggregator branch June 9, 2016 17:40

masci added a commit that referenced this pull request Feb 12, 2017

experiment #2

5e9ab60

masci added a commit that referenced this pull request Apr 1, 2019

Merge pull request #2 from DataDog/olivielpeau/fix-py2-test

a5ca0b7

[test/two] Update init test with new six extension interface

hush-hush pushed a commit that referenced this pull request Apr 17, 2019

Merge pull request #2 from DataDog/olivielpeau/fix-py2-test

9c515f1

[test/two] Update init test with new six extension interface

safchain pushed a commit to safchain/datadog-agent that referenced this pull request May 11, 2020

Merge pull request DataDog#2 from safchain/fix-partial-discriminator

cb4aff6

Fix partial evaluation and add stringer interface to evaluators

safchain added a commit to safchain/datadog-agent that referenced this pull request Jun 4, 2020

Merge pull request DataDog#2 from lebauce/fix-rename

0fc3579

Fix rename on older kernels

prognant added a commit that referenced this pull request Sep 18, 2020

[logs] multi line aggregation part #2

6f7d49a

prognant added a commit that referenced this pull request Sep 18, 2020

[logs] Aggregates long lines when tailing in a k8s env - part #2 (#6266)

48a7127

prognant mentioned this pull request Dec 9, 2020

[logs/AD] Logs tailing from docker label #6883

Merged

L3n41c added a commit that referenced this pull request Jun 23, 2021

Do not make the liveness probe fail in case of network issue — Part #2 (

be59410

#7605)

brycekahle mentioned this pull request Sep 21, 2021

[networks] Improve memory allocation pattern #9206

Merged

5 tasks

dujuku mentioned this pull request Nov 2, 2021

pkg/trace/obfuscate: fix bug where obfuscation fails for autovacuum sql text #9649

Merged

5 tasks

gh123man mentioned this pull request Dec 13, 2021

Dual shipping v2 + Refactor and optimize HTTP compression #10162

Merged

8 tasks

iglendd added a commit that referenced this pull request Mar 18, 2022

Switched to github.com/shirou/gopsutil/v3

e353168

Added reference to the upgraded gohai commit (#2)

iglendd added a commit that referenced this pull request Mar 23, 2022

Switched to github.com/shirou/gopsutil/v3

8242482

Code review related changes #2

just-chillin added a commit that referenced this pull request Sep 8, 2022

empty commit #2

3401888

nhulston mentioned this pull request Jun 2, 2025

[serverless-init] Fix multiline stacktraces being split up into multiple logs #37551

Merged

JSGette added a commit that referenced this pull request Aug 14, 2025

Remove non-existing SQG #2

6d2ab01

JSGette added a commit that referenced this pull request Sep 8, 2025

Revert changes in agent.py #2

f989f68

JSGette added a commit that referenced this pull request Sep 11, 2025

Remove non-existing parameter #2

959e5b4

JSGette added a commit that referenced this pull request Sep 17, 2025

Protect $ #2

24238f3

guohdd added a commit that referenced this pull request Sep 23, 2025

fix linter issue #2

32f7a66

JSGette added a commit that referenced this pull request Sep 26, 2025

Protect $ #2

87015ef

guohdd added a commit that referenced this pull request Oct 27, 2025

changes per review #2

ad50f4d

DanielLavie added a commit that referenced this pull request Nov 4, 2025

Fixed @paulcacheux comments try #2

d698c6a

JSGette added a commit that referenced this pull request Nov 12, 2025

Fix the comment #2

0cb3cc5

aiuto added a commit that referenced this pull request Nov 18, 2025

# This is a combination of 3 commits.

8edfef9

# This is the 1st commit message: Add ability to detect ship_source_offer # This is the commit message #2: get rid of merge ghost # This is the commit message #3: buildifier woes

s-alad pushed a commit that referenced this pull request Nov 21, 2025

Merge pull request #2 from rapdev-io/v0.1.1

277ab95

V0.1.1

guohdd added a commit that referenced this pull request Jan 27, 2026

fix CI failures #2

2a05318

louis-cqrl mentioned this pull request Feb 9, 2026

feat(health-platform): persist issues to disk across restarts #46069

Merged

BaptisteFoy mentioned this pull request Feb 24, 2026

fix(fleet/installer): fix ~90s daemon stop delay during installer setup #46757

Closed

matt-dz mentioned this pull request Feb 26, 2026

Implement Agent Safe Shell — POSIX commands as safe builtins #46945

Closed

4 tasks

chatgpt-codex-connector Bot mentioned this pull request Apr 1, 2026

Improving Disk Metrics: Distinguishing Real Disks from Pseudo-Filesystems #48766

Merged

louis-cqrl mentioned this pull request Apr 24, 2026

[AGTHEAL-115] refactor(core): migrate comp/core/flare to V2 component architecture #49769

Draft

This was referenced Apr 24, 2026

Coordinator run log — observer AD iteration #49678

Draft

coord run-log (full) — 2026-04-27 16:44 #49939

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose an Aggregator to Python checks#2

Expose an Aggregator to Python checks#2
masci merged 7 commits intomasterfrom
massi/expose_aggregator

masci commented May 30, 2016 •

edited

Loading

Uh oh!

olivielpeau commented Jun 9, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

masci commented May 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

olivielpeau commented Jun 9, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

masci commented May 30, 2016 •

edited

Loading