Skip to content

[test optimization] [SDTEST-1272] Detect new tests regardless of Early Flake Detection#5138

Merged
juan-fernandez merged 10 commits intomasterfrom
juan-fernandez/request-known-tests
Jan 22, 2025
Merged

[test optimization] [SDTEST-1272] Detect new tests regardless of Early Flake Detection#5138
juan-fernandez merged 10 commits intomasterfrom
juan-fernandez/request-known-tests

Conversation

@juan-fernandez
Copy link
Copy Markdown
Collaborator

@juan-fernandez juan-fernandez commented Jan 21, 2025

What does this PR do?

  • Always request known tests if known_tests_enabled is responded by the backend.
  • Always set test.is_new: true tag to new tests.
  • Set test.retry_reason: 'efd' if tests are retried with early flake detection.

Motivation

  • Detecting new tests adds value regardless of whether you want to detect flakiness on them or not.

Plugin Checklist

  • Unit tests.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 21, 2025

Overall package size

Self size: 8.52 MB
Deduped: 94.88 MB
No deduping: 95.4 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@juan-fernandez juan-fernandez changed the title [test optimization] Detect new tests regardless of Early Flake Detection [test optimization] [SDTEST-1272] Detect new tests regardless of Early Flake Detection Jan 21, 2025
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jan 21, 2025

Benchmarks

Benchmark execution time: 2025-01-22 14:42:04

Comparing candidate commit ff95b2d in PR branch juan-fernandez/request-known-tests with baseline commit 504b460 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 908 metrics, 20 unstable metrics.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.15%. Comparing base (d9ffb78) to head (f047d3a).
Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #5138       +/-   ##
===========================================
+ Coverage   64.56%   91.15%   +26.58%     
===========================================
  Files         313      121      -192     
  Lines       14636     4195    -10441     
===========================================
- Hits         9450     3824     -5626     
+ Misses       5186      371     -4815     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@juan-fernandez juan-fernandez marked this pull request as ready for review January 22, 2025 12:06
@juan-fernandez juan-fernandez requested review from a team as code owners January 22, 2025 12:06
@juan-fernandez juan-fernandez merged commit c13d368 into master Jan 22, 2025
@juan-fernandez juan-fernandez deleted the juan-fernandez/request-known-tests branch January 22, 2025 14:51
szegedi pushed a commit that referenced this pull request Jan 28, 2025
@szegedi szegedi mentioned this pull request Jan 28, 2025
szegedi pushed a commit that referenced this pull request Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants