[system_tests] Fix flaky debugger tests

[vlad-scherbich]

https://datadoghq.atlassian.net/browse/PROF-13796

Motivation

While iterating on PR #6377 (IAST standalone fix), CI repeatedly failed on debugger scenarios (DEBUGGER_TELEMETRY, DEBUGGER_PII_REDACTION, DEBUGGER_PROBES_SNAPSHOT_WITH_SCM, DEBUGGER_SYMDB) across dotnet, java, golang, and python jobs.

Fixing them now to reduces CI noise.

Changes

• DebuggerScenario (utils/_context/_scenarios/debugger.py): library_interface_timeout 5 → 15 seconds for all debugger scenarios.
• debugger_telemetry scenario (utils/_context/_scenarios/__init__.py): library_interface_timeout 5 → 20 seconds (dotnet needs more time for RC + telemetry).
• test_debugger_telemetry.py: Add explicit wait_for_telemetry timeouts (5s default, 25s for dotnet).
• test_debugger_pii.py:
  • Add wait_for_all_probes(statuses=["EMITTING"]) in the retry loop.
  • Add defensive check for missing probe snapshots before iterating (clear AssertionError instead of KeyError).

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

tests/debugger/test_debugger_pii.py                                     @DataDog/debugger @DataDog/system-tests-core
tests/debugger/test_debugger_telemetry.py                               @DataDog/debugger @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/debugger.py                                   @DataDog/system-tests-core

[datadog-datadog-prod-us1-2]

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 18 Tests failed

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip.test_blocking_before[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip object at 0x7f97d87f7a40>

    def test_blocking_before(self):
        """Test that blocked requests are blocked before being processed"""
        # second request should block and must not set the tag in span
        assert self.block_req2.status_code == 403
>       interfaces.library.assert_waf_attack(self.block_req2, rule="blk-001-001")

...

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip.test_blocking[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip object at 0x7f97d87f7980>

    def test_blocking(self):
        """Can block the request forwarded for the ip"""
    
        assert self.rm_req_block.status_code == 403
>       interfaces.library.assert_waf_attack(self.rm_req_block, rule="blk-001-001")

...

tests.appsec.test_blocking_addresses.Test_Blocking_client_ip_with_forwarded.test_blocking_before[envoy] from system_tests_suite (Datadog) (Fix with Cursor)

ValueError: No appsec event validate this condition

self = <tests.appsec.test_blocking_addresses.Test_Blocking_client_ip_with_forwarded object at 0x7f97d87f7e00>

    def test_blocking_before(self):
        """Test that blocked requests are blocked before being processed"""
        # second request should block and must not set the tag in span
        assert self.block_req2.status_code == 403
>       interfaces.library.assert_waf_attack(self.block_req2, rule="blk-001-001")

...

View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cfe1218 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}