[test optimization] Fix playwright's quarantine and retry logic

[juan-fernandez]

What does this PR do?

• Fix the logic to prevent a test session from failing.
• Report a test session as passed if it's prevented from failing.
• Additionally not prevent ATR if test management is enabled and only disable retries for tests that are attempted to fix or retried by EFD.

Motivation

Quarantine logic fix

When deciding whether the overall test session status should be flipped to passed because all the failures were quarantined, we were doing it incorrectly, as eventually passing tests (flaky tests) were counted towards totalFailedTestCount. This made totalFailedTestCount > totalIgnorableFailures, and our logic to switch the status to passed failed.

Additionally, if the session is actually prevented from failing we'll report a test.status='pass' instead of 'fail'

Retry logic fix

We were disabling ATR altogether whenever we detected that test management was enabled. This is not necessary, as only attempt to fix tests should not be retried (they're "rerun" already).

We were not disabling --retries or ATR for tests being retried by EFD. This caused a potential explosion of retries.

Plugin Checklist

• Integration tests.

[github-actions]

Overall package size

Self size: 13.43 MB
Deduped: 113.62 MB
No deduping: 128.64 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.3.0 | 20.73 MB | 20.74 MB | | @datadog/pprof | 5.12.0 | 11.19 MB | 11.57 MB | | @datadog/native-iast-taint-tracking | 4.1.0 | 9.01 MB | 9.02 MB | | @opentelemetry/resources | 1.30.1 | 557.67 kB | 7.71 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.4 | 2.95 MB | 5.83 MB | | @datadog/wasm-js-rewriter | 5.0.1 | 2.82 MB | 3.53 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api-logs | 0.208.0 | 199.48 kB | 1.42 MB | | @opentelemetry/api | 1.9.0 | 1.22 MB | 1.22 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.15.0 | 127.66 kB | 856.24 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | @datadog/openfeature-node-server | 0.2.0 | 118.51 kB | 437.19 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | source-map | 0.7.6 | 185.63 kB | 185.63 kB | | pprof-format | 2.2.1 | 163.06 kB | 163.06 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | @isaacs/ttlcache | 2.1.3 | 90.79 kB | 90.79 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB | | escape-string-regexp | 5.0.0 | 3.66 kB | 3.66 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.59%. Comparing base (0bb1f17) to head (15787e9).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #7024       +/-   ##
===========================================
- Coverage   84.94%   72.59%   -12.35%     
===========================================
  Files         514      173      -341     
  Lines       21754     6046    -15708     
===========================================
- Hits        18478     4389    -14089     
+ Misses       3276     1657     -1619     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-12-04 13:49:44

Comparing candidate commit 15787e9 in PR branch juan-fernandez/fix-quarantine-for-tests-with-other-failures with baseline commit 0bb1f17 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 289 metrics, 31 unstable metrics.