chore(ci): use dd-sts for system-tests test optimization by nccatoni · Pull Request #11070 · DataDog/dd-trace-java

nccatoni · 2026-04-09T13:57:43Z

Summary

Migrates system-tests CI to use dd-sts for Datadog Test Optimization instead of long-lived API keys.

All repositories now share a single system-tests policy (see dd-source#408172) — no per-repo policy is needed.

Depends on DataDog/system-tests#6726.

Changes

Pin system-tests to 1e5d6b709 (current main, pre-migration) to allow a controlled rollout: repos stay on the pre-migration workflow until their pin is explicitly updated to the post-merge SHA

pr-commenter · 2026-04-09T14:44:56Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nccatoni/system-tests-dd-sts
git_commit_date	1776242720	1776242123
git_commit_sha	93449b6f0f	`de54455`
release_version	1.62.0-SNAPSHOT~693449b6f0f	1.62.0-SNAPSHOT~de54455d98

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1776244012	1776244012
ci_job_id	1596418804	1596418804
ci_pipeline_id	107756350	107756350
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-19me8sam 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-19me8sam 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.055 s) : 0, 1055054
Total [baseline] (8.83 s) : 0, 8830460
Agent [candidate] (1.054 s) : 0, 1053939
Total [candidate] (8.804 s) : 0, 8803889
section iast
Agent [baseline] (1.219 s) : 0, 1219484
Total [baseline] (9.544 s) : 0, 9543555
Agent [candidate] (1.231 s) : 0, 1231413
Total [candidate] (9.642 s) : 0, 9641619

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.055 s	-
Agent	iast	1.219 s	164.43 ms (15.6%)
Total	tracing	8.83 s	-
Total	iast	9.544 s	713.095 ms (8.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.054 s	-
Agent	iast	1.231 s	177.474 ms (16.8%)
Total	tracing	8.804 s	-
Total	iast	9.642 s	837.729 ms (9.5%)

gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (632.495 ms) : 0, 632495
BytebuddyAgent [candidate] (631.07 ms) : 0, 631070
AgentMeter [baseline] (29.399 ms) : 0, 29399
AgentMeter [candidate] (29.42 ms) : 0, 29420
GlobalTracer [baseline] (248.191 ms) : 0, 248191
GlobalTracer [candidate] (248.161 ms) : 0, 248161
AppSec [baseline] (32.318 ms) : 0, 32318
AppSec [candidate] (32.321 ms) : 0, 32321
Debugger [baseline] (59.121 ms) : 0, 59121
Debugger [candidate] (59.014 ms) : 0, 59014
Remote Config [baseline] (594.396 µs) : 0, 594
Remote Config [candidate] (587.124 µs) : 0, 587
Telemetry [baseline] (8.028 ms) : 0, 8028
Telemetry [candidate] (8.006 ms) : 0, 8006
Flare Poller [baseline] (7.488 ms) : 0, 7488
Flare Poller [candidate] (8.065 ms) : 0, 8065
section iast
crashtracking [baseline] (1.217 ms) : 0, 1217
crashtracking [candidate] (1.236 ms) : 0, 1236
BytebuddyAgent [baseline] (798.182 ms) : 0, 798182
BytebuddyAgent [candidate] (806.531 ms) : 0, 806531
AgentMeter [baseline] (11.345 ms) : 0, 11345
AgentMeter [candidate] (11.467 ms) : 0, 11467
GlobalTracer [baseline] (238.559 ms) : 0, 238559
GlobalTracer [candidate] (240.002 ms) : 0, 240002
IAST [baseline] (25.758 ms) : 0, 25758
IAST [candidate] (26.773 ms) : 0, 26773
AppSec [baseline] (31.856 ms) : 0, 31856
AppSec [candidate] (30.81 ms) : 0, 30810
Debugger [baseline] (60.133 ms) : 0, 60133
Debugger [candidate] (61.556 ms) : 0, 61556
Remote Config [baseline] (544.823 µs) : 0, 545
Remote Config [candidate] (534.953 µs) : 0, 535
Telemetry [baseline] (12.013 ms) : 0, 12013
Telemetry [candidate] (12.608 ms) : 0, 12608
Flare Poller [baseline] (3.427 ms) : 0, 3427
Flare Poller [candidate] (3.501 ms) : 0, 3501

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1061933
Total [baseline] (11.082 s) : 0, 11081513
Agent [candidate] (1.058 s) : 0, 1058184
Total [candidate] (11.025 s) : 0, 11025072
section appsec
Agent [baseline] (1.248 s) : 0, 1247678
Total [baseline] (11.189 s) : 0, 11189396
Agent [candidate] (1.249 s) : 0, 1249304
Total [candidate] (11.069 s) : 0, 11068528
section iast
Agent [baseline] (1.234 s) : 0, 1233778
Total [baseline] (11.301 s) : 0, 11301199
Agent [candidate] (1.232 s) : 0, 1231563
Total [candidate] (11.339 s) : 0, 11339197
section profiling
Agent [baseline] (1.184 s) : 0, 1183817
Total [baseline] (11.079 s) : 0, 11078536
Agent [candidate] (1.185 s) : 0, 1185262
Total [candidate] (11.006 s) : 0, 11005621

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.062 s	-
Agent	appsec	1.248 s	185.745 ms (17.5%)
Agent	iast	1.234 s	171.845 ms (16.2%)
Agent	profiling	1.184 s	121.884 ms (11.5%)
Total	tracing	11.082 s	-
Total	appsec	11.189 s	107.883 ms (1.0%)
Total	iast	11.301 s	219.685 ms (2.0%)
Total	profiling	11.079 s	-2.977 ms (-0.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	appsec	1.249 s	191.121 ms (18.1%)
Agent	iast	1.232 s	173.379 ms (16.4%)
Agent	profiling	1.185 s	127.078 ms (12.0%)
Total	tracing	11.025 s	-
Total	appsec	11.069 s	43.456 ms (0.4%)
Total	iast	11.339 s	314.125 ms (2.8%)
Total	profiling	11.006 s	-19.451 ms (-0.2%)

gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.243 ms) : 0, 1243
crashtracking [candidate] (1.22 ms) : 0, 1220
BytebuddyAgent [baseline] (636.595 ms) : 0, 636595
BytebuddyAgent [candidate] (632.611 ms) : 0, 632611
AgentMeter [baseline] (29.659 ms) : 0, 29659
AgentMeter [candidate] (29.385 ms) : 0, 29385
GlobalTracer [baseline] (249.88 ms) : 0, 249880
GlobalTracer [candidate] (248.852 ms) : 0, 248852
AppSec [baseline] (32.488 ms) : 0, 32488
AppSec [candidate] (32.328 ms) : 0, 32328
Debugger [baseline] (60.344 ms) : 0, 60344
Debugger [candidate] (60.005 ms) : 0, 60005
Remote Config [baseline] (601.086 µs) : 0, 601
Remote Config [candidate] (594.282 µs) : 0, 594
Telemetry [baseline] (8.12 ms) : 0, 8120
Telemetry [candidate] (8.09 ms) : 0, 8090
Flare Poller [baseline] (6.753 ms) : 0, 6753
Flare Poller [candidate] (9.002 ms) : 0, 9002
section appsec
crashtracking [baseline] (1.217 ms) : 0, 1217
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (661.073 ms) : 0, 661073
BytebuddyAgent [candidate] (662.866 ms) : 0, 662866
AgentMeter [baseline] (12.088 ms) : 0, 12088
AgentMeter [candidate] (12.039 ms) : 0, 12039
GlobalTracer [baseline] (249.143 ms) : 0, 249143
GlobalTracer [candidate] (249.347 ms) : 0, 249347
AppSec [baseline] (184.954 ms) : 0, 184954
AppSec [candidate] (184.845 ms) : 0, 184845
Debugger [baseline] (65.651 ms) : 0, 65651
Debugger [candidate] (65.628 ms) : 0, 65628
Remote Config [baseline] (607.896 µs) : 0, 608
Remote Config [candidate] (600.974 µs) : 0, 601
Telemetry [baseline] (8.472 ms) : 0, 8472
Telemetry [candidate] (8.396 ms) : 0, 8396
Flare Poller [baseline] (3.53 ms) : 0, 3530
Flare Poller [candidate] (3.501 ms) : 0, 3501
IAST [baseline] (24.588 ms) : 0, 24588
IAST [candidate] (24.465 ms) : 0, 24465
section iast
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.227 ms) : 0, 1227
BytebuddyAgent [baseline] (808.734 ms) : 0, 808734
BytebuddyAgent [candidate] (807.862 ms) : 0, 807862
AgentMeter [baseline] (11.49 ms) : 0, 11490
AgentMeter [candidate] (11.61 ms) : 0, 11610
GlobalTracer [baseline] (240.018 ms) : 0, 240018
GlobalTracer [candidate] (239.465 ms) : 0, 239465
AppSec [baseline] (31.325 ms) : 0, 31325
AppSec [candidate] (29.642 ms) : 0, 29642
Debugger [baseline] (62.012 ms) : 0, 62012
Debugger [candidate] (65.19 ms) : 0, 65190
Remote Config [baseline] (1.75 ms) : 0, 1750
Remote Config [candidate] (532.383 µs) : 0, 532
Telemetry [baseline] (11.479 ms) : 0, 11479
Telemetry [candidate] (10.552 ms) : 0, 10552
Flare Poller [baseline] (3.48 ms) : 0, 3480
Flare Poller [candidate] (3.477 ms) : 0, 3477
IAST [baseline] (25.779 ms) : 0, 25779
IAST [candidate] (25.764 ms) : 0, 25764
section profiling
ProfilingAgent [baseline] (93.939 ms) : 0, 93939
ProfilingAgent [candidate] (93.568 ms) : 0, 93568
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.174 ms) : 0, 1174
BytebuddyAgent [baseline] (690.719 ms) : 0, 690719
BytebuddyAgent [candidate] (692.246 ms) : 0, 692246
AgentMeter [baseline] (9.158 ms) : 0, 9158
AgentMeter [candidate] (9.136 ms) : 0, 9136
GlobalTracer [baseline] (207.445 ms) : 0, 207445
GlobalTracer [candidate] (207.768 ms) : 0, 207768
AppSec [baseline] (32.719 ms) : 0, 32719
AppSec [candidate] (32.706 ms) : 0, 32706
Debugger [baseline] (65.488 ms) : 0, 65488
Debugger [candidate] (65.39 ms) : 0, 65390
Remote Config [baseline] (572.983 µs) : 0, 573
Remote Config [candidate] (582.264 µs) : 0, 582
Telemetry [baseline] (7.772 ms) : 0, 7772
Telemetry [candidate] (7.773 ms) : 0, 7773
Flare Poller [baseline] (3.516 ms) : 0, 3516
Flare Poller [candidate] (3.562 ms) : 0, 3562
Profiling [baseline] (94.528 ms) : 0, 94528
Profiling [candidate] (94.13 ms) : 0, 94130

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nccatoni/system-tests-dd-sts
git_commit_date	1776242859	1776242123
git_commit_sha	93449b6f0f	`de54455`
release_version	1.62.0-SNAPSHOT~693449b6f0f	1.62.0-SNAPSHOT~de54455d98

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1776244632	1776244632
ci_job_id	1596418807	1596418807
ci_pipeline_id	107756350	107756350
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-aeeh04pp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-aeeh04pp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 17 metrics, 18 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:tracing:high_load	better [-964.609µs; -467.974µs] or [-5.339%; -2.590%]	unsure [-1685.741µs; -139.168µs] or [-5.788%; -0.478%]	unstable [-22.854op/s; +41.354op/s] or [-9.015%; +16.313%]	17.351ms	28.214ms	262.750op/s	18.067ms	29.126ms	253.500op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.252 ms) : 1240, 1264
.   : milestone, 1252,
iast (3.293 ms) : 3249, 3337
.   : milestone, 3293,
iast_FULL (5.927 ms) : 5867, 5986
.   : milestone, 5927,
iast_GLOBAL (3.697 ms) : 3638, 3756
.   : milestone, 3697,
profiling (2.191 ms) : 2169, 2213
.   : milestone, 2191,
tracing (1.886 ms) : 1871, 1902
.   : milestone, 1886,
section candidate
no_agent (1.255 ms) : 1243, 1267
.   : milestone, 1255,
iast (3.414 ms) : 3363, 3466
.   : milestone, 3414,
iast_FULL (6.108 ms) : 6046, 6171
.   : milestone, 6108,
iast_GLOBAL (3.756 ms) : 3692, 3820
.   : milestone, 3756,
profiling (2.048 ms) : 2029, 2067
.   : milestone, 2048,
tracing (1.857 ms) : 1840, 1874
.   : milestone, 1857,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.252 ms [1.24 ms, 1.264 ms]	-
iast	3.293 ms [3.249 ms, 3.337 ms]	2.042 ms (163.1%)
iast_FULL	5.927 ms [5.867 ms, 5.986 ms]	4.675 ms (373.4%)
iast_GLOBAL	3.697 ms [3.638 ms, 3.756 ms]	2.445 ms (195.3%)
profiling	2.191 ms [2.169 ms, 2.213 ms]	938.896 µs (75.0%)
tracing	1.886 ms [1.871 ms, 1.902 ms]	634.526 µs (50.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.255 ms [1.243 ms, 1.267 ms]	-
iast	3.414 ms [3.363 ms, 3.466 ms]	2.16 ms (172.1%)
iast_FULL	6.108 ms [6.046 ms, 6.171 ms]	4.854 ms (386.8%)
iast_GLOBAL	3.756 ms [3.692 ms, 3.82 ms]	2.501 ms (199.3%)
profiling	2.048 ms [2.029 ms, 2.067 ms]	793.336 µs (63.2%)
tracing	1.857 ms [1.84 ms, 1.874 ms]	602.003 µs (48.0%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.716 ms) : 18528, 18904
.   : milestone, 18716,
appsec (18.986 ms) : 18794, 19178
.   : milestone, 18986,
code_origins (17.734 ms) : 17563, 17905
.   : milestone, 17734,
iast (18.019 ms) : 17840, 18197
.   : milestone, 18019,
profiling (17.997 ms) : 17819, 18176
.   : milestone, 17997,
tracing (18.406 ms) : 18222, 18589
.   : milestone, 18406,
section candidate
no_agent (18.278 ms) : 18091, 18465
.   : milestone, 18278,
appsec (18.677 ms) : 18486, 18867
.   : milestone, 18677,
code_origins (17.746 ms) : 17575, 17917
.   : milestone, 17746,
iast (17.859 ms) : 17684, 18034
.   : milestone, 17859,
profiling (18.022 ms) : 17844, 18200
.   : milestone, 18022,
tracing (17.758 ms) : 17583, 17934
.   : milestone, 17758,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.716 ms [18.528 ms, 18.904 ms]	-
appsec	18.986 ms [18.794 ms, 19.178 ms]	269.546 µs (1.4%)
code_origins	17.734 ms [17.563 ms, 17.905 ms]	-982.601 µs (-5.2%)
iast	18.019 ms [17.84 ms, 18.197 ms]	-697.548 µs (-3.7%)
profiling	17.997 ms [17.819 ms, 18.176 ms]	-719.1 µs (-3.8%)
tracing	18.406 ms [18.222 ms, 18.589 ms]	-310.719 µs (-1.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.278 ms [18.091 ms, 18.465 ms]	-
appsec	18.677 ms [18.486 ms, 18.867 ms]	398.665 µs (2.2%)
code_origins	17.746 ms [17.575 ms, 17.917 ms]	-531.882 µs (-2.9%)
iast	17.859 ms [17.684 ms, 18.034 ms]	-418.802 µs (-2.3%)
profiling	18.022 ms [17.844 ms, 18.2 ms]	-255.754 µs (-1.4%)
tracing	17.758 ms [17.583 ms, 17.934 ms]	-519.44 µs (-2.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nccatoni/system-tests-dd-sts
git_commit_date	1776242849	1776242123
git_commit_sha	93449b6f0f	`de54455`
release_version	1.62.0-SNAPSHOT~693449b6f0f	1.62.0-SNAPSHOT~de54455d98

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1776244324	1776244324
ci_job_id	1596418809	1596418809
ci_pipeline_id	107756350	107756350
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-1b3buqrb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-1b3buqrb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.466ms; -1.121ms] or [-38.322%; -29.284%]	2.533ms	3.826ms

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1471, 1494
.   : milestone, 1482,
appsec (3.826 ms) : 3606, 4047
.   : milestone, 3826,
iast (2.26 ms) : 2191, 2329
.   : milestone, 2260,
iast_GLOBAL (2.307 ms) : 2238, 2377
.   : milestone, 2307,
profiling (2.097 ms) : 2042, 2152
.   : milestone, 2097,
tracing (2.081 ms) : 2027, 2135
.   : milestone, 2081,
section candidate
no_agent (1.483 ms) : 1472, 1495
.   : milestone, 1483,
appsec (2.533 ms) : 2478, 2588
.   : milestone, 2533,
iast (2.274 ms) : 2204, 2344
.   : milestone, 2274,
iast_GLOBAL (2.318 ms) : 2248, 2387
.   : milestone, 2318,
profiling (2.51 ms) : 2346, 2673
.   : milestone, 2510,
tracing (2.068 ms) : 2014, 2122
.   : milestone, 2068,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.471 ms, 1.494 ms]	-
appsec	3.826 ms [3.606 ms, 4.047 ms]	2.344 ms (158.1%)
iast	2.26 ms [2.191 ms, 2.329 ms]	777.543 µs (52.4%)
iast_GLOBAL	2.307 ms [2.238 ms, 2.377 ms]	824.701 µs (55.6%)
profiling	2.097 ms [2.042 ms, 2.152 ms]	614.153 µs (41.4%)
tracing	2.081 ms [2.027 ms, 2.135 ms]	598.149 µs (40.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.483 ms [1.472 ms, 1.495 ms]	-
appsec	2.533 ms [2.478 ms, 2.588 ms]	1.05 ms (70.8%)
iast	2.274 ms [2.204 ms, 2.344 ms]	791.129 µs (53.3%)
iast_GLOBAL	2.318 ms [2.248 ms, 2.387 ms]	834.487 µs (56.3%)
profiling	2.51 ms [2.346 ms, 2.673 ms]	1.026 ms (69.2%)
tracing	2.068 ms [2.014 ms, 2.122 ms]	584.994 µs (39.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~de54455d98, baseline=1.62.0-SNAPSHOT~693449b6f0f
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.925 s) : 14925000, 14925000
.   : milestone, 14925000,
appsec (14.723 s) : 14723000, 14723000
.   : milestone, 14723000,
iast (18.415 s) : 18415000, 18415000
.   : milestone, 18415000,
iast_GLOBAL (18.244 s) : 18244000, 18244000
.   : milestone, 18244000,
profiling (14.79 s) : 14790000, 14790000
.   : milestone, 14790000,
tracing (14.909 s) : 14909000, 14909000
.   : milestone, 14909000,
section candidate
no_agent (15.167 s) : 15167000, 15167000
.   : milestone, 15167000,
appsec (15.102 s) : 15102000, 15102000
.   : milestone, 15102000,
iast (18.178 s) : 18178000, 18178000
.   : milestone, 18178000,
iast_GLOBAL (17.826 s) : 17826000, 17826000
.   : milestone, 17826000,
profiling (15.571 s) : 15571000, 15571000
.   : milestone, 15571000,
tracing (14.918 s) : 14918000, 14918000
.   : milestone, 14918000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.925 s [14.925 s, 14.925 s]	-
appsec	14.723 s [14.723 s, 14.723 s]	-202.0 ms (-1.4%)
iast	18.415 s [18.415 s, 18.415 s]	3.49 s (23.4%)
iast_GLOBAL	18.244 s [18.244 s, 18.244 s]	3.319 s (22.2%)
profiling	14.79 s [14.79 s, 14.79 s]	-135.0 ms (-0.9%)
tracing	14.909 s [14.909 s, 14.909 s]	-16.0 ms (-0.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.167 s [15.167 s, 15.167 s]	-
appsec	15.102 s [15.102 s, 15.102 s]	-65.0 ms (-0.4%)
iast	18.178 s [18.178 s, 18.178 s]	3.011 s (19.9%)
iast_GLOBAL	17.826 s [17.826 s, 17.826 s]	2.659 s (17.5%)
profiling	15.571 s [15.571 s, 15.571 s]	404.0 ms (2.7%)
tracing	14.918 s [14.918 s, 14.918 s]	-249.0 ms (-1.6%)

github-actions · 2026-04-13T13:46:51Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

PerfectSlayer

🔨 issue: ‏You should not be pinning system tests version on master branch.
Is there a reason for?

nccatoni · 2026-04-14T08:40:14Z

@PerfectSlayer The pin is just temporary to be able to control the rollout of the dd-sts migration. Otherwise when I merge in system-tests all the repos will be moved to the new method at once which could cause the CI to break.

bric3 · 2026-04-14T12:50:27Z

@nccatoni How long is the rollout ?

THe current ref do not include much difference, but it should be updated nonetheless to the latest version.

DataDog/system-tests@1e5d6b7...main

nccatoni · 2026-04-14T13:07:20Z

@bric3 It shouldn't take more than one or two days. You can check the thread in #apm-shared-testing for more details and updates. I just updated the ref

chore(ci): use dd-sts for system-tests test optimization

d1f4080

nccatoni force-pushed the nccatoni/system-tests-dd-sts branch from 5030569 to d1f4080 Compare April 9, 2026 14:44

nccatoni added 4 commits April 13, 2026 09:17

chore(ci): remove dd_sts_policy from system-tests subworkflow

1dd9568

chore(ci): remove unneeded permissions from system-tests subworkflow

a9227be

chore(ci): pin system-tests workflow to 1e5d6b709

7c21267

fix: add contents:read permission to system-tests workflow call

72f4562

nccatoni marked this pull request as ready for review April 13, 2026 13:46

nccatoni requested a review from a team as a code owner April 13, 2026 13:46

nccatoni requested review from bric3 and removed request for a team April 13, 2026 13:46

nccatoni added comp: tooling Build & Tooling type: enhancement Enhancements and improvements labels Apr 13, 2026

chore(ci): pass ref to system-tests reusable workflow

8e3a1de

PerfectSlayer reviewed Apr 14, 2026

View reviewed changes

nccatoni requested a review from PerfectSlayer April 14, 2026 11:15

Update ref

9706336

Merge branch 'master' into nccatoni/system-tests-dd-sts

de54455

nccatoni closed this Apr 17, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(ci): use dd-sts for system-tests test optimization#11070

chore(ci): use dd-sts for system-tests test optimization#11070
nccatoni wants to merge 8 commits intomasterfrom
nccatoni/system-tests-dd-sts

nccatoni commented Apr 9, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Apr 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 13, 2026 •

edited

Loading

Uh oh!

PerfectSlayer left a comment •

edited

Loading

Uh oh!

nccatoni commented Apr 14, 2026

Uh oh!

bric3 commented Apr 14, 2026

Uh oh!

nccatoni commented Apr 14, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

nccatoni commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

pr-commenter Bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

github-actions Bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PerfectSlayer left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nccatoni commented Apr 14, 2026

Uh oh!

bric3 commented Apr 14, 2026

Uh oh!

nccatoni commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nccatoni commented Apr 9, 2026 •

edited

Loading

pr-commenter Bot commented Apr 9, 2026 •

edited

Loading

github-actions Bot commented Apr 13, 2026 •

edited

Loading

PerfectSlayer left a comment •

edited

Loading

nccatoni commented Apr 14, 2026 •

edited

Loading