Update system-tests to 74de81e97fd6786854c2fe254764cfb75610c676 by smola · Pull Request #5964 · DataDog/dd-trace-java

smola · 2023-09-29T11:02:50Z

What Does This Do

Comparison:
https://github.com/datadog/system-tests/compare/a5c7ef0fa0f988b4c09e3734b99a37dd436a3eef..74de81e97fd6786854c2fe254764cfb75610c676

Motivation

Additional Notes

Previous update: #5915

Relevant changes:

pr-commenter · 2023-09-29T11:44:58Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.22.0-SNAPSHOT~8b135c1bcc	1.22.0-SNAPSHOT~ae8b3e0a28
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.014 s) : 0, 1014051
Total [baseline] (9.256 s) : 0, 9256482
Agent [candidate] (1.012 s) : 0, 1012284
Total [candidate] (9.181 s) : 0, 9181019
section appsec
Agent [baseline] (1.096 s) : 0, 1096169
Total [baseline] (9.255 s) : 0, 9254616
Agent [candidate] (1.102 s) : 0, 1102344
Total [candidate] (9.343 s) : 0, 9343186
section iast
Agent [baseline] (1.119 s) : 0, 1119163
Total [baseline] (9.377 s) : 0, 9376518
Agent [candidate] (1.119 s) : 0, 1118783
Total [candidate] (9.445 s) : 0, 9444720
section profiling
Agent [baseline] (1.178 s) : 0, 1178286
Total [baseline] (9.388 s) : 0, 9388173
Agent [candidate] (1.186 s) : 0, 1186003
Total [candidate] (9.407 s) : 0, 9407400

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.014 s	-
Agent	appsec	1.096 s	82.118 ms (8.1%)
Agent	iast	1.119 s	105.112 ms (10.4%)
Agent	profiling	1.178 s	164.235 ms (16.2%)
Total	tracing	9.256 s	-
Total	appsec	9.255 s	-1.866 ms (-0.0%)
Total	iast	9.377 s	120.035 ms (1.3%)
Total	profiling	9.388 s	131.691 ms (1.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.012 s	-
Agent	appsec	1.102 s	90.06 ms (8.9%)
Agent	iast	1.119 s	106.5 ms (10.5%)
Agent	profiling	1.186 s	173.719 ms (17.2%)
Total	tracing	9.181 s	-
Total	appsec	9.343 s	162.166 ms (1.8%)
Total	iast	9.445 s	263.7 ms (2.9%)
Total	profiling	9.407 s	226.381 ms (2.5%)

gantt
    title petclinic - break down per module: candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (628.968 ms) : 0, 628968
BytebuddyAgent [candidate] (628.737 ms) : 0, 628737
GlobalTracer [baseline] (294.687 ms) : 0, 294687
GlobalTracer [candidate] (293.826 ms) : 0, 293826
AppSec [baseline] (49.101 ms) : 0, 49101
AppSec [candidate] (48.619 ms) : 0, 48619
Remote Config [baseline] (666.732 µs) : 0, 667
Remote Config [candidate] (650.741 µs) : 0, 651
Telemetry [baseline] (6.071 ms) : 0, 6071
Telemetry [candidate] (5.952 ms) : 0, 5952
section appsec
BytebuddyAgent [baseline] (625.585 ms) : 0, 625585
BytebuddyAgent [candidate] (628.113 ms) : 0, 628113
GlobalTracer [baseline] (292.575 ms) : 0, 292575
GlobalTracer [candidate] (295.65 ms) : 0, 295650
AppSec [baseline] (137.275 ms) : 0, 137275
AppSec [candidate] (137.83 ms) : 0, 137830
Remote Config [baseline] (645.452 µs) : 0, 645
Remote Config [candidate] (648.501 µs) : 0, 649
Telemetry [baseline] (5.794 ms) : 0, 5794
Telemetry [candidate] (5.833 ms) : 0, 5833
section iast
BytebuddyAgent [baseline] (740.891 ms) : 0, 740891
BytebuddyAgent [candidate] (740.386 ms) : 0, 740386
GlobalTracer [baseline] (276.983 ms) : 0, 276983
GlobalTracer [candidate] (277.147 ms) : 0, 277147
AppSec [baseline] (46.077 ms) : 0, 46077
AppSec [candidate] (46.259 ms) : 0, 46259
Remote Config [baseline] (565.909 µs) : 0, 566
Remote Config [candidate] (555.385 µs) : 0, 555
Telemetry [baseline] (5.775 ms) : 0, 5775
Telemetry [candidate] (5.75 ms) : 0, 5750
IAST [baseline] (14.468 ms) : 0, 14468
IAST [candidate] (14.376 ms) : 0, 14376
section profiling
BytebuddyAgent [baseline] (636.513 ms) : 0, 636513
BytebuddyAgent [candidate] (641.497 ms) : 0, 641497
GlobalTracer [baseline] (352.782 ms) : 0, 352782
GlobalTracer [candidate] (354.44 ms) : 0, 354440
AppSec [baseline] (48.896 ms) : 0, 48896
AppSec [candidate] (49.105 ms) : 0, 49105
Remote Config [baseline] (661.7 µs) : 0, 662
Remote Config [candidate] (658.428 µs) : 0, 658
Telemetry [baseline] (6.115 ms) : 0, 6115
Telemetry [candidate] (6.113 ms) : 0, 6113
ProfilingAgent [baseline] (80.312 ms) : 0, 80312
ProfilingAgent [candidate] (80.821 ms) : 0, 80821
Profiling [baseline] (80.336 ms) : 0, 80336
Profiling [candidate] (80.845 ms) : 0, 80845

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.006 s) : 0, 1005532
Total [baseline] (8.613 s) : 0, 8613471
Agent [candidate] (1.008 s) : 0, 1008225
Total [candidate] (8.635 s) : 0, 8635132
section appsec
Agent [baseline] (1.101 s) : 0, 1100899
Total [baseline] (8.792 s) : 0, 8792344
Agent [candidate] (1.104 s) : 0, 1103826
Total [candidate] (8.718 s) : 0, 8717761
section iast
Agent [baseline] (1.118 s) : 0, 1117689
Total [baseline] (9.235 s) : 0, 9235086
Agent [candidate] (1.117 s) : 0, 1116926
Total [candidate] (9.21 s) : 0, 9209988
section profiling
Agent [baseline] (1.185 s) : 0, 1185451
Total [baseline] (8.915 s) : 0, 8915413
Agent [candidate] (1.18 s) : 0, 1180456
Total [candidate] (8.876 s) : 0, 8875808

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.006 s	-
Agent	appsec	1.101 s	95.367 ms (9.5%)
Agent	iast	1.118 s	112.158 ms (11.2%)
Agent	profiling	1.185 s	179.919 ms (17.9%)
Total	tracing	8.613 s	-
Total	appsec	8.792 s	178.873 ms (2.1%)
Total	iast	9.235 s	621.615 ms (7.2%)
Total	profiling	8.915 s	301.942 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.008 s	-
Agent	appsec	1.104 s	95.6 ms (9.5%)
Agent	iast	1.117 s	108.7 ms (10.8%)
Agent	profiling	1.18 s	172.23 ms (17.1%)
Total	tracing	8.635 s	-
Total	appsec	8.718 s	82.629 ms (1.0%)
Total	iast	9.21 s	574.856 ms (6.7%)
Total	profiling	8.876 s	240.676 ms (2.8%)

gantt
    title insecure-bank - break down per module: candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (623.718 ms) : 0, 623718
BytebuddyAgent [candidate] (625.055 ms) : 0, 625055
GlobalTracer [baseline] (291.895 ms) : 0, 291895
GlobalTracer [candidate] (293.106 ms) : 0, 293106
AppSec [baseline] (49.084 ms) : 0, 49084
AppSec [candidate] (49.132 ms) : 0, 49132
Remote Config [baseline] (651.095 µs) : 0, 651
Remote Config [candidate] (670.266 µs) : 0, 670
Telemetry [baseline] (5.973 ms) : 0, 5973
Telemetry [candidate] (6.035 ms) : 0, 6035
section appsec
BytebuddyAgent [baseline] (626.562 ms) : 0, 626562
BytebuddyAgent [candidate] (629.579 ms) : 0, 629579
GlobalTracer [baseline] (295.536 ms) : 0, 295536
GlobalTracer [candidate] (295.425 ms) : 0, 295425
AppSec [baseline] (138.034 ms) : 0, 138034
AppSec [candidate] (137.848 ms) : 0, 137848
Remote Config [baseline] (645.638 µs) : 0, 646
Remote Config [candidate] (642.504 µs) : 0, 643
Telemetry [baseline] (5.854 ms) : 0, 5854
Telemetry [candidate] (5.822 ms) : 0, 5822
section iast
BytebuddyAgent [baseline] (738.164 ms) : 0, 738164
BytebuddyAgent [candidate] (738.733 ms) : 0, 738733
GlobalTracer [baseline] (278.16 ms) : 0, 278160
GlobalTracer [candidate] (276.858 ms) : 0, 276858
AppSec [baseline] (46.583 ms) : 0, 46583
AppSec [candidate] (46.172 ms) : 0, 46172
Remote Config [baseline] (555.18 µs) : 0, 555
Remote Config [candidate] (561.268 µs) : 0, 561
Telemetry [baseline] (5.725 ms) : 0, 5725
Telemetry [candidate] (5.787 ms) : 0, 5787
IAST [baseline] (14.341 ms) : 0, 14341
IAST [candidate] (14.487 ms) : 0, 14487
section profiling
BytebuddyAgent [baseline] (640.73 ms) : 0, 640730
BytebuddyAgent [candidate] (638.83 ms) : 0, 638830
GlobalTracer [baseline] (354.325 ms) : 0, 354325
GlobalTracer [candidate] (353.259 ms) : 0, 353259
AppSec [baseline] (49.245 ms) : 0, 49245
AppSec [candidate] (48.854 ms) : 0, 48854
Remote Config [baseline] (675.152 µs) : 0, 675
Remote Config [candidate] (651.929 µs) : 0, 652
Telemetry [baseline] (6.173 ms) : 0, 6173
Telemetry [candidate] (6.192 ms) : 0, 6192
ProfilingAgent [baseline] (81.061 ms) : 0, 81061
ProfilingAgent [candidate] (79.629 ms) : 0, 79629
Profiling [baseline] (81.085 ms) : 0, 81085
Profiling [candidate] (79.654 ms) : 0, 79654

Load

Parameters

	Baseline	Candidate
commit	1.22.0-SNAPSHOT~8b135c1bcc	1.22.0-SNAPSHOT~ae8b3e0a28
config	baseline	candidate
end_time	2023-10-02T07:56:21	2023-10-02T08:14:28
start_time	2023-10-02T07:56:03	2023-10-02T08:14:09

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.308 ms) : 1288, 1327
.   : milestone, 1308,
appsec (1.7 ms) : 1676, 1725
.   : milestone, 1700,
iast (1.463 ms) : 1438, 1487
.   : milestone, 1463,
profiling (1.473 ms) : 1448, 1499
.   : milestone, 1473,
tracing (1.415 ms) : 1390, 1439
.   : milestone, 1415,
section candidate
no_agent (1.32 ms) : 1302, 1339
.   : milestone, 1320,
appsec (1.687 ms) : 1662, 1711
.   : milestone, 1687,
iast (1.451 ms) : 1428, 1475
.   : milestone, 1451,
profiling (1.458 ms) : 1434, 1482
.   : milestone, 1458,
tracing (1.436 ms) : 1412, 1460
.   : milestone, 1436,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.308 ms [1.288 ms, 1.327 ms]	-
appsec	1.7 ms [1.676 ms, 1.725 ms]	392.669 µs (30.0%)
iast	1.463 ms [1.438 ms, 1.487 ms]	154.897 µs (11.8%)
profiling	1.473 ms [1.448 ms, 1.499 ms]	165.738 µs (12.7%)
tracing	1.415 ms [1.39 ms, 1.439 ms]	107.156 µs (8.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.32 ms [1.302 ms, 1.339 ms]	-
appsec	1.687 ms [1.662 ms, 1.711 ms]	366.393 µs (27.8%)
iast	1.451 ms [1.428 ms, 1.475 ms]	131.199 µs (9.9%)
profiling	1.458 ms [1.434 ms, 1.482 ms]	137.624 µs (10.4%)
tracing	1.436 ms [1.412 ms, 1.46 ms]	115.633 µs (8.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~ae8b3e0a28, baseline=1.22.0-SNAPSHOT~8b135c1bcc
    dateFormat X
    axisFormat %s
section baseline
no_agent (358.504 µs) : 339, 378
.   : milestone, 359,
appsec (684.767 µs) : 664, 706
.   : milestone, 685,
iast (452.124 µs) : 431, 473
.   : milestone, 452,
iast_FULL (512.414 µs) : 492, 533
.   : milestone, 512,
iast_INACTIVE (426.419 µs) : 405, 447
.   : milestone, 426,
profiling (427.272 µs) : 406, 449
.   : milestone, 427,
tracing (436.426 µs) : 415, 458
.   : milestone, 436,
section candidate
no_agent (359.302 µs) : 338, 381
.   : milestone, 359,
appsec (687.328 µs) : 667, 708
.   : milestone, 687,
iast (455.379 µs) : 435, 476
.   : milestone, 455,
iast_FULL (508.616 µs) : 488, 529
.   : milestone, 509,
iast_INACTIVE (428.053 µs) : 407, 449
.   : milestone, 428,
profiling (430.234 µs) : 408, 452
.   : milestone, 430,
tracing (430.483 µs) : 410, 451
.   : milestone, 430,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	358.504 µs [338.815 µs, 378.194 µs]	-
appsec	684.767 µs [664.026 µs, 705.508 µs]	326.263 µs (91.0%)
iast	452.124 µs [431.151 µs, 473.097 µs]	93.62 µs (26.1%)
iast_FULL	512.414 µs [491.704 µs, 533.124 µs]	153.91 µs (42.9%)
iast_INACTIVE	426.419 µs [405.426 µs, 447.412 µs]	67.914 µs (18.9%)
profiling	427.272 µs [405.878 µs, 448.666 µs]	68.768 µs (19.2%)
tracing	436.426 µs [415.196 µs, 457.657 µs]	77.922 µs (21.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	359.302 µs [338.014 µs, 380.59 µs]	-
appsec	687.328 µs [667.123 µs, 707.533 µs]	328.026 µs (91.3%)
iast	455.379 µs [434.674 µs, 476.083 µs]	96.077 µs (26.7%)
iast_FULL	508.616 µs [488.006 µs, 529.227 µs]	149.314 µs (41.6%)
iast_INACTIVE	428.053 µs [407.149 µs, 448.956 µs]	68.75 µs (19.1%)
profiling	430.234 µs [408.309 µs, 452.158 µs]	70.931 µs (19.7%)
tracing	430.483 µs [409.623 µs, 451.343 µs]	71.18 µs (19.8%)

Update system-tests to 74de81e97fd6786854c2fe254764cfb75610c676

0be3cbb

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Sep 29, 2023

smola requested a review from a team as a code owner September 29, 2023 11:02

smola requested review from am312 and bantonsson September 29, 2023 11:02

Merge branch 'master' into smola/update-system-tests-20230929

ae8b3e0

smola enabled auto-merge (squash) October 2, 2023 08:43

jandro996 approved these changes Oct 2, 2023

View reviewed changes

smola merged commit 3d6b259 into master Oct 2, 2023

smola deleted the smola/update-system-tests-20230929 branch October 2, 2023 08:44

github-actions Bot added this to the 1.22.0 milestone Oct 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update system-tests to 74de81e97fd6786854c2fe254764cfb75610c676#5964

Update system-tests to 74de81e97fd6786854c2fe254764cfb75610c676#5964
smola merged 2 commits intomasterfrom
smola/update-system-tests-20230929

smola commented Sep 29, 2023

Uh oh!

pr-commenter Bot commented Sep 29, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

smola commented Sep 29, 2023

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Sep 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pr-commenter Bot commented Sep 29, 2023 •

edited

Loading