Update system-tests to 3598f2583cd5c25326cd7a5c01097374d94c1993 by smola · Pull Request #5848 · DataDog/dd-trace-java

smola · 2023-09-06T15:08:15Z

What Does This Do

Comparison:
https://github.com/datadog/system-tests/compare/1a721f777d95a50144b8d9a195c20684a9a2b265..3598f2583cd5c25326cd7a5c01097374d94c1993

Motivation

Additional Notes

Previous update: #5794

Relevant changes:

pr-commenter · 2023-09-06T15:51:27Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.21.0-SNAPSHOT~782278584e	1.21.0-SNAPSHOT~ed613885c9
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (996.486 ms) : 0, 996486
Total [baseline] (9.214 s) : 0, 9213697
Agent [candidate] (995.264 ms) : 0, 995264
Total [candidate] (9.204 s) : 0, 9204274
section appsec
Agent [baseline] (1.087 s) : 0, 1086778
Total [baseline] (9.339 s) : 0, 9338945
Agent [candidate] (1.079 s) : 0, 1078553
Total [candidate] (9.255 s) : 0, 9254829
section iast
Agent [baseline] (1.098 s) : 0, 1097683
Total [baseline] (9.518 s) : 0, 9518281
Agent [candidate] (1.098 s) : 0, 1097962
Total [candidate] (9.425 s) : 0, 9424886
section profiling
Agent [baseline] (1.166 s) : 0, 1166244
Total [baseline] (9.436 s) : 0, 9435790
Agent [candidate] (1.171 s) : 0, 1171461
Total [candidate] (9.382 s) : 0, 9382097

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	996.486 ms	-
Agent	appsec	1.087 s	90.292 ms (9.1%)
Agent	iast	1.098 s	101.197 ms (10.2%)
Agent	profiling	1.166 s	169.758 ms (17.0%)
Total	tracing	9.214 s	-
Total	appsec	9.339 s	125.248 ms (1.4%)
Total	iast	9.518 s	304.584 ms (3.3%)
Total	profiling	9.436 s	222.093 ms (2.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	995.264 ms	-
Agent	appsec	1.079 s	83.289 ms (8.4%)
Agent	iast	1.098 s	102.697 ms (10.3%)
Agent	profiling	1.171 s	176.197 ms (17.7%)
Total	tracing	9.204 s	-
Total	appsec	9.255 s	50.554 ms (0.5%)
Total	iast	9.425 s	220.611 ms (2.4%)
Total	profiling	9.382 s	177.823 ms (1.9%)

gantt
    title petclinic - break down per module: candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (614.452 ms) : 0, 614452
BytebuddyAgent [candidate] (612.939 ms) : 0, 612939
GlobalTracer [baseline] (292.837 ms) : 0, 292837
GlobalTracer [candidate] (293.294 ms) : 0, 293294
AppSec [baseline] (48.182 ms) : 0, 48182
AppSec [candidate] (48.077 ms) : 0, 48077
Remote Config [baseline] (689.541 µs) : 0, 690
Remote Config [candidate] (686.447 µs) : 0, 686
Telemetry [baseline] (5.966 ms) : 0, 5966
Telemetry [candidate] (5.944 ms) : 0, 5944
section appsec
BytebuddyAgent [baseline] (617.429 ms) : 0, 617429
BytebuddyAgent [candidate] (611.871 ms) : 0, 611871
GlobalTracer [baseline] (294.388 ms) : 0, 294388
GlobalTracer [candidate] (292.117 ms) : 0, 292117
AppSec [baseline] (133.867 ms) : 0, 133867
AppSec [candidate] (133.986 ms) : 0, 133986
Remote Config [baseline] (710.493 µs) : 0, 710
Remote Config [candidate] (709.356 µs) : 0, 709
Telemetry [baseline] (5.786 ms) : 0, 5786
Telemetry [candidate] (5.709 ms) : 0, 5709
section iast
BytebuddyAgent [baseline] (719.211 ms) : 0, 719211
BytebuddyAgent [candidate] (719.471 ms) : 0, 719471
GlobalTracer [baseline] (277.566 ms) : 0, 277566
GlobalTracer [candidate] (277.81 ms) : 0, 277810
AppSec [baseline] (46.122 ms) : 0, 46122
AppSec [candidate] (45.773 ms) : 0, 45773
IAST [baseline] (14.342 ms) : 0, 14342
IAST [candidate] (14.459 ms) : 0, 14459
Remote Config [baseline] (562.829 µs) : 0, 563
Remote Config [candidate] (564.928 µs) : 0, 565
Telemetry [baseline] (5.657 ms) : 0, 5657
Telemetry [candidate] (5.684 ms) : 0, 5684
section profiling
BytebuddyAgent [baseline] (625.907 ms) : 0, 625907
BytebuddyAgent [candidate] (630.293 ms) : 0, 630293
GlobalTracer [baseline] (354.816 ms) : 0, 354816
GlobalTracer [candidate] (355.559 ms) : 0, 355559
AppSec [baseline] (47.976 ms) : 0, 47976
AppSec [candidate] (48.238 ms) : 0, 48238
Remote Config [baseline] (642.541 µs) : 0, 643
Remote Config [candidate] (646.83 µs) : 0, 647
Telemetry [baseline] (6.035 ms) : 0, 6035
Telemetry [candidate] (5.963 ms) : 0, 5963
ProfilingAgent [baseline] (78.022 ms) : 0, 78022
ProfilingAgent [candidate] (77.484 ms) : 0, 77484
Profiling [baseline] (78.049 ms) : 0, 78049
Profiling [candidate] (77.51 ms) : 0, 77510

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.009 s) : 0, 1008925
Total [baseline] (8.7 s) : 0, 8700368
Agent [candidate] (996.744 ms) : 0, 996744
Total [candidate] (8.657 s) : 0, 8656819
section appsec
Agent [baseline] (1.086 s) : 0, 1086322
Total [baseline] (8.795 s) : 0, 8795155
Agent [candidate] (1.087 s) : 0, 1087043
Total [candidate] (8.773 s) : 0, 8772739
section iast
Agent [baseline] (1.101 s) : 0, 1100954
Total [baseline] (9.202 s) : 0, 9201940
Agent [candidate] (1.098 s) : 0, 1098225
Total [candidate] (9.207 s) : 0, 9207007
section profiling
Agent [baseline] (1.174 s) : 0, 1174046
Total [baseline] (8.894 s) : 0, 8893572
Agent [candidate] (1.168 s) : 0, 1167595
Total [candidate] (8.859 s) : 0, 8858843

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.009 s	-
Agent	appsec	1.086 s	77.397 ms (7.7%)
Agent	iast	1.101 s	92.028 ms (9.1%)
Agent	profiling	1.174 s	165.12 ms (16.4%)
Total	tracing	8.7 s	-
Total	appsec	8.795 s	94.786 ms (1.1%)
Total	iast	9.202 s	501.572 ms (5.8%)
Total	profiling	8.894 s	193.203 ms (2.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	996.744 ms	-
Agent	appsec	1.087 s	90.299 ms (9.1%)
Agent	iast	1.098 s	101.48 ms (10.2%)
Agent	profiling	1.168 s	170.851 ms (17.1%)
Total	tracing	8.657 s	-
Total	appsec	8.773 s	115.92 ms (1.3%)
Total	iast	9.207 s	550.188 ms (6.4%)
Total	profiling	8.859 s	202.023 ms (2.3%)

gantt
    title insecure-bank - break down per module: candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (622.197 ms) : 0, 622197
BytebuddyAgent [candidate] (613.516 ms) : 0, 613516
GlobalTracer [baseline] (296.422 ms) : 0, 296422
GlobalTracer [candidate] (293.653 ms) : 0, 293653
AppSec [baseline] (48.759 ms) : 0, 48759
AppSec [candidate] (48.593 ms) : 0, 48593
Remote Config [baseline] (704.873 µs) : 0, 705
Remote Config [candidate] (675.204 µs) : 0, 675
Telemetry [baseline] (5.986 ms) : 0, 5986
Telemetry [candidate] (5.935 ms) : 0, 5935
section appsec
BytebuddyAgent [baseline] (617.662 ms) : 0, 617662
BytebuddyAgent [candidate] (617.155 ms) : 0, 617155
GlobalTracer [baseline] (294.03 ms) : 0, 294030
GlobalTracer [candidate] (295.428 ms) : 0, 295428
AppSec [baseline] (133.604 ms) : 0, 133604
AppSec [candidate] (133.499 ms) : 0, 133499
Remote Config [baseline] (700.942 µs) : 0, 701
Remote Config [candidate] (712.298 µs) : 0, 712
Telemetry [baseline] (5.796 ms) : 0, 5796
Telemetry [candidate] (5.756 ms) : 0, 5756
section iast
BytebuddyAgent [baseline] (720.703 ms) : 0, 720703
BytebuddyAgent [candidate] (719.523 ms) : 0, 719523
GlobalTracer [baseline] (278.844 ms) : 0, 278844
GlobalTracer [candidate] (277.936 ms) : 0, 277936
AppSec [baseline] (46.281 ms) : 0, 46281
AppSec [candidate] (45.829 ms) : 0, 45829
IAST [baseline] (14.464 ms) : 0, 14464
IAST [candidate] (14.332 ms) : 0, 14332
Remote Config [baseline] (579.981 µs) : 0, 580
Remote Config [candidate] (569.338 µs) : 0, 569
Telemetry [baseline] (5.769 ms) : 0, 5769
Telemetry [candidate] (5.702 ms) : 0, 5702
section profiling
BytebuddyAgent [baseline] (630.08 ms) : 0, 630080
BytebuddyAgent [candidate] (625.994 ms) : 0, 625994
GlobalTracer [baseline] (356.862 ms) : 0, 356862
GlobalTracer [candidate] (355.263 ms) : 0, 355263
AppSec [baseline] (48.408 ms) : 0, 48408
AppSec [candidate] (48.176 ms) : 0, 48176
Remote Config [baseline] (655.272 µs) : 0, 655
Remote Config [candidate] (648.259 µs) : 0, 648
Telemetry [baseline] (6.088 ms) : 0, 6088
Telemetry [candidate] (6.059 ms) : 0, 6059
ProfilingAgent [baseline] (78.625 ms) : 0, 78625
ProfilingAgent [candidate] (78.693 ms) : 0, 78693
Profiling [baseline] (78.652 ms) : 0, 78652
Profiling [candidate] (78.719 ms) : 0, 78719

Load

Parameters

	Baseline	Candidate
commit	1.21.0-SNAPSHOT~782278584e	1.21.0-SNAPSHOT~ed613885c9
config	baseline	candidate
end_time	2023-09-06T15:28:07	2023-09-06T15:46:12
start_time	2023-09-06T15:27:49	2023-09-06T15:45:54

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.327 ms) : 1309, 1346
.   : milestone, 1327,
appsec (1.687 ms) : 1662, 1712
.   : milestone, 1687,
iast (1.474 ms) : 1450, 1498
.   : milestone, 1474,
profiling (1.464 ms) : 1438, 1489
.   : milestone, 1464,
tracing (1.461 ms) : 1437, 1485
.   : milestone, 1461,
section candidate
no_agent (1.357 ms) : 1338, 1375
.   : milestone, 1357,
appsec (1.667 ms) : 1642, 1692
.   : milestone, 1667,
iast (1.484 ms) : 1459, 1508
.   : milestone, 1484,
profiling (1.481 ms) : 1455, 1507
.   : milestone, 1481,
tracing (1.447 ms) : 1422, 1471
.   : milestone, 1447,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.327 ms [1.309 ms, 1.346 ms]	-
appsec	1.687 ms [1.662 ms, 1.712 ms]	359.624 µs (27.1%)
iast	1.474 ms [1.45 ms, 1.498 ms]	147.322 µs (11.1%)
profiling	1.464 ms [1.438 ms, 1.489 ms]	136.663 µs (10.3%)
tracing	1.461 ms [1.437 ms, 1.485 ms]	134.344 µs (10.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.357 ms [1.338 ms, 1.375 ms]	-
appsec	1.667 ms [1.642 ms, 1.692 ms]	310.324 µs (22.9%)
iast	1.484 ms [1.459 ms, 1.508 ms]	127.299 µs (9.4%)
profiling	1.481 ms [1.455 ms, 1.507 ms]	124.389 µs (9.2%)
tracing	1.447 ms [1.422 ms, 1.471 ms]	89.994 µs (6.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.21.0-SNAPSHOT~ed613885c9, baseline=1.21.0-SNAPSHOT~782278584e
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.785 µs) : 352, 394
.   : milestone, 373,
appsec (668.366 µs) : 648, 689
.   : milestone, 668,
iast (471.415 µs) : 450, 493
.   : milestone, 471,
iast_FULL (526.409 µs) : 506, 547
.   : milestone, 526,
iast_INACTIVE (434.775 µs) : 414, 456
.   : milestone, 435,
profiling (441.987 µs) : 421, 463
.   : milestone, 442,
tracing (437.215 µs) : 416, 458
.   : milestone, 437,
section candidate
no_agent (364.913 µs) : 345, 385
.   : milestone, 365,
appsec (677.872 µs) : 657, 699
.   : milestone, 678,
iast (462.159 µs) : 441, 483
.   : milestone, 462,
iast_FULL (527.8 µs) : 507, 548
.   : milestone, 528,
iast_INACTIVE (440.529 µs) : 420, 461
.   : milestone, 441,
profiling (440.373 µs) : 419, 462
.   : milestone, 440,
tracing (439.229 µs) : 419, 460
.   : milestone, 439,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	372.785 µs [351.654 µs, 393.916 µs]	-
appsec	668.366 µs [647.767 µs, 688.965 µs]	295.581 µs (79.3%)
iast	471.415 µs [450.153 µs, 492.678 µs]	98.631 µs (26.5%)
iast_FULL	526.409 µs [505.663 µs, 547.155 µs]	153.625 µs (41.2%)
iast_INACTIVE	434.775 µs [413.784 µs, 455.766 µs]	61.99 µs (16.6%)
profiling	441.987 µs [421.258 µs, 462.716 µs]	69.202 µs (18.6%)
tracing	437.215 µs [416.189 µs, 458.24 µs]	64.43 µs (17.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	364.913 µs [345.147 µs, 384.679 µs]	-
appsec	677.872 µs [657.121 µs, 698.623 µs]	312.96 µs (85.8%)
iast	462.159 µs [441.315 µs, 483.003 µs]	97.246 µs (26.6%)
iast_FULL	527.8 µs [507.195 µs, 548.406 µs]	162.888 µs (44.6%)
iast_INACTIVE	440.529 µs [420.138 µs, 460.92 µs]	75.616 µs (20.7%)
profiling	440.373 µs [418.717 µs, 462.029 µs]	75.46 µs (20.7%)
tracing	439.229 µs [418.787 µs, 459.671 µs]	74.317 µs (20.4%)

Update system-tests to 9326debf5b273d5905bd856841c7337ba4cd5fe4

ed61388

smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Sep 6, 2023

smola requested a review from a team as a code owner September 6, 2023 15:08

smola requested review from a team, PerfectSlayer, manuel-alvarez-alvarez and ygree September 6, 2023 15:08

Update system-tests to 3598f2583cd5c25326cd7a5c01097374d94c1993

1271640

smola changed the title ~~Update system-tests to 9326debf5b273d5905bd856841c7337ba4cd5fe4~~ Update system-tests to 3598f2583cd5c25326cd7a5c01097374d94c1993 Sep 7, 2023

jandro996 approved these changes Sep 7, 2023

View reviewed changes

smola enabled auto-merge (squash) September 7, 2023 08:57

smola merged commit 706075d into master Sep 7, 2023

smola deleted the smola/system-tests-20230906 branch September 7, 2023 09:01

github-actions Bot added this to the 1.21.0 milestone Sep 7, 2023

smola mentioned this pull request Sep 19, 2023

Update system-tests to a5c7ef0fa0f988b4c09e3734b99a37dd436a3eef #5915

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update system-tests to 3598f2583cd5c25326cd7a5c01097374d94c1993#5848

Update system-tests to 3598f2583cd5c25326cd7a5c01097374d94c1993#5848
smola merged 2 commits intomasterfrom
smola/system-tests-20230906

smola commented Sep 6, 2023 •

edited

Loading

Uh oh!

pr-commenter Bot commented Sep 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

smola commented Sep 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Uh oh!

pr-commenter Bot commented Sep 6, 2023

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

smola commented Sep 6, 2023 •

edited

Loading