Skip to content

Don't create a process span for agent internal threads #6478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bantonsson merged 1 commit into from
Jan 12, 2024
Merged

Don't create a process span for agent internal threads #6478

bantonsson merged 1 commit into from
Jan 12, 2024

Conversation

@bantonsson
Copy link
Contributor

@bantonsson bantonsson commented Jan 12, 2024 •
edited
Loading

What Does This Do

Skips creating a process span for process execution that is initialized from one of the agent internal threads.

Motivation

Internal calls to hostname and other commands from the agent during setup runs at various times and pollute smoke tests with unrelated traces.

@bantonsson bantonsson added tag: no release notes Changes to exclude from release notes run-tests: all Run all tests labels Jan 12, 2024
@bantonsson bantonsson changed the title Don't create a process span for agent intternal threads Don't create a process span for agent internal threads Jan 12, 2024
@bantonsson bantonsson force-pushed the ban/no-process-spans-for-agent branch from ebb280c to 39a26b8 Compare January 12, 2024 12:55
@pr-commenter
Copy link

pr-commenter bot commented Jan 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ban/no-process-spans-for-agent
git_commit_date 1705056379 1705064087
git_commit_sha 656c660 39a26b8
release_version 1.28.0-SNAPSHOT~656c6600e0 1.28.0-SNAPSHOT~39a26b859c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705066720 1705066720
ci_job_id 407067830 407067830
ci_pipeline_id 26392532 26392532
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051394
Total [baseline] (8.713 s) : 0, 8712514
Agent [candidate] (1.053 s) : 0, 1053054
Total [candidate] (8.728 s) : 0, 8728300
section iast
Agent [baseline] (1.178 s) : 0, 1178007
Total [baseline] (9.242 s) : 0, 9242044
Agent [candidate] (1.181 s) : 0, 1180692
Total [candidate] (9.239 s) : 0, 9238761
section iast_TELEMETRY_OFF
Agent [baseline] (1.178 s) : 0, 1177601
Total [baseline] (9.278 s) : 0, 9278180
Agent [candidate] (1.169 s) : 0, 1169078
Total [candidate] (9.282 s) : 0, 9282364
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent iast 1.178 s 126.613 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.178 s 126.208 ms (12.0%)
Total tracing 8.713 s -
Total iast 9.242 s 529.529 ms (6.1%)
Total iast_TELEMETRY_OFF 9.278 s 565.665 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.181 s 127.639 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.169 s 116.024 ms (11.0%)
Total tracing 8.728 s -
Total iast 9.239 s 510.46 ms (5.8%)
Total iast_TELEMETRY_OFF 9.282 s 554.063 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (664.187 ms) : 0, 664187
BytebuddyAgent [candidate] (665.401 ms) : 0, 665401
GlobalTracer [baseline] (294.748 ms) : 0, 294748
GlobalTracer [candidate] (295.051 ms) : 0, 295051
AppSec [baseline] (50.467 ms) : 0, 50467
AppSec [candidate] (50.628 ms) : 0, 50628
Remote Config [baseline] (657.349 µs) : 0, 657
Remote Config [candidate] (645.223 µs) : 0, 645
Telemetry [baseline] (7.184 ms) : 0, 7184
Telemetry [candidate] (7.156 ms) : 0, 7156
section iast
BytebuddyAgent [baseline] (776.73 ms) : 0, 776730
BytebuddyAgent [candidate] (778.358 ms) : 0, 778358
GlobalTracer [baseline] (285.494 ms) : 0, 285494
GlobalTracer [candidate] (287.286 ms) : 0, 287286
AppSec [baseline] (52.807 ms) : 0, 52807
AppSec [candidate] (53.428 ms) : 0, 53428
IAST [baseline] (20.714 ms) : 0, 20714
IAST [candidate] (19.962 ms) : 0, 19962
Remote Config [baseline] (601.569 µs) : 0, 602
Remote Config [candidate] (599.232 µs) : 0, 599
Telemetry [baseline] (7.26 ms) : 0, 7260
Telemetry [candidate] (6.575 ms) : 0, 6575
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (773.89 ms) : 0, 773890
BytebuddyAgent [candidate] (767.959 ms) : 0, 767959
GlobalTracer [baseline] (287.905 ms) : 0, 287905
GlobalTracer [candidate] (285.952 ms) : 0, 285952
AppSec [baseline] (54.102 ms) : 0, 54102
AppSec [candidate] (53.205 ms) : 0, 53205
IAST [baseline] (20.184 ms) : 0, 20184
IAST [candidate] (20.694 ms) : 0, 20694
Remote Config [baseline] (587.111 µs) : 0, 587
Remote Config [candidate] (583.564 µs) : 0, 584
Telemetry [baseline] (6.387 ms) : 0, 6387
Telemetry [candidate] (6.39 ms) : 0, 6390
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053139
Total [baseline] (9.354 s) : 0, 9353593
Agent [candidate] (1.063 s) : 0, 1062588
Total [candidate] (9.35 s) : 0, 9350312
section appsec
Agent [baseline] (1.152 s) : 0, 1151587
Total [baseline] (9.459 s) : 0, 9458914
Agent [candidate] (1.155 s) : 0, 1154983
Total [candidate] (9.493 s) : 0, 9493391
section iast
Agent [baseline] (1.177 s) : 0, 1176980
Total [baseline] (9.584 s) : 0, 9584326
Agent [candidate] (1.187 s) : 0, 1186743
Total [candidate] (9.644 s) : 0, 9644003
section profiling
Agent [baseline] (1.273 s) : 0, 1273002
Total [baseline] (9.591 s) : 0, 9590642
Agent [candidate] (1.277 s) : 0, 1277381
Total [candidate] (9.584 s) : 0, 9583639
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent appsec 1.152 s 98.448 ms (9.3%)
Agent iast 1.177 s 123.841 ms (11.8%)
Agent profiling 1.273 s 219.863 ms (20.9%)
Total tracing 9.354 s -
Total appsec 9.459 s 105.321 ms (1.1%)
Total iast 9.584 s 230.734 ms (2.5%)
Total profiling 9.591 s 237.049 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.155 s 92.395 ms (8.7%)
Agent iast 1.187 s 124.155 ms (11.7%)
Agent profiling 1.277 s 214.793 ms (20.2%)
Total tracing 9.35 s -
Total appsec 9.493 s 143.078 ms (1.5%)
Total iast 9.644 s 293.691 ms (3.1%)
Total profiling 9.584 s 233.327 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.249 ms) : 0, 665249
BytebuddyAgent [candidate] (671.143 ms) : 0, 671143
GlobalTracer [baseline] (295.292 ms) : 0, 295292
GlobalTracer [candidate] (297.816 ms) : 0, 297816
AppSec [baseline] (50.496 ms) : 0, 50496
AppSec [candidate] (51.058 ms) : 0, 51058
Remote Config [baseline] (658.656 µs) : 0, 659
Remote Config [candidate] (665.442 µs) : 0, 665
Telemetry [baseline] (7.199 ms) : 0, 7199
Telemetry [candidate] (7.314 ms) : 0, 7314
section appsec
BytebuddyAgent [baseline] (665.853 ms) : 0, 665853
BytebuddyAgent [candidate] (666.67 ms) : 0, 666670
GlobalTracer [baseline] (295.447 ms) : 0, 295447
GlobalTracer [candidate] (297.241 ms) : 0, 297241
AppSec [baseline] (148.525 ms) : 0, 148525
AppSec [candidate] (149.144 ms) : 0, 149144
Remote Config [baseline] (643.714 µs) : 0, 644
Remote Config [candidate] (646.678 µs) : 0, 647
Telemetry [baseline] (6.858 ms) : 0, 6858
Telemetry [candidate] (6.913 ms) : 0, 6913
section iast
BytebuddyAgent [baseline] (774.462 ms) : 0, 774462
BytebuddyAgent [candidate] (783.225 ms) : 0, 783225
GlobalTracer [baseline] (286.29 ms) : 0, 286290
GlobalTracer [candidate] (287.803 ms) : 0, 287803
AppSec [baseline] (51.645 ms) : 0, 51645
AppSec [candidate] (51.476 ms) : 0, 51476
IAST [baseline] (21.625 ms) : 0, 21625
IAST [candidate] (21.492 ms) : 0, 21492
Remote Config [baseline] (604.015 µs) : 0, 604
Remote Config [candidate] (625.011 µs) : 0, 625
Telemetry [baseline] (8.054 ms) : 0, 8054
Telemetry [candidate] (7.33 ms) : 0, 7330
section profiling
BytebuddyAgent [baseline] (662.419 ms) : 0, 662419
BytebuddyAgent [candidate] (665.391 ms) : 0, 665391
GlobalTracer [baseline] (374.584 ms) : 0, 374584
GlobalTracer [candidate] (376.032 ms) : 0, 376032
AppSec [baseline] (50.929 ms) : 0, 50929
AppSec [candidate] (50.861 ms) : 0, 50861
Remote Config [baseline] (985.261 µs) : 0, 985
Remote Config [candidate] (1.015 ms) : 0, 1015
Telemetry [baseline] (7.206 ms) : 0, 7206
Telemetry [candidate] (7.25 ms) : 0, 7250
ProfilingAgent [baseline] (122.622 ms) : 0, 122622
ProfilingAgent [candidate] (122.384 ms) : 0, 122384
Profiling [baseline] (122.646 ms) : 0, 122646
Profiling [candidate] (122.409 ms) : 0, 122409
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-01-12T13:18:02 2024-01-12T13:34:29
git_branch master ban/no-process-spans-for-agent
git_commit_date 1705056379 1705064087
git_commit_sha 656c660 39a26b8
release_version 1.28.0-SNAPSHOT~656c6600e0 1.28.0-SNAPSHOT~39a26b859c
start_time 2024-01-12T13:17:49 2024-01-12T13:34:16
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1705066720 1705066720
ci_job_id 407067830 407067830
ci_pipeline_id 26392532 26392532
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 8 metrics, 14 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0
    dateFormat X
    axisFormat %s
section baseline
no_agent (366.102 µs) : 346, 386
.   : milestone, 366,
iast (470.853 µs) : 450, 491
.   : milestone, 471,
iast_FULL (534.577 µs) : 514, 555
.   : milestone, 535,
iast_INACTIVE (442.485 µs) : 422, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (467.584 µs) : 447, 488
.   : milestone, 468,
tracing (441.406 µs) : 420, 463
.   : milestone, 441,
section candidate
no_agent (361.573 µs) : 342, 382
.   : milestone, 362,
iast (470.43 µs) : 450, 491
.   : milestone, 470,
iast_FULL (533.271 µs) : 513, 554
.   : milestone, 533,
iast_INACTIVE (445.973 µs) : 425, 467
.   : milestone, 446,
iast_TELEMETRY_OFF (465.799 µs) : 445, 486
.   : milestone, 466,
tracing (436.331 µs) : 416, 457
.   : milestone, 436,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 366.102 µs [345.736 µs, 386.468 µs] -
iast 470.853 µs [450.426 µs, 491.281 µs] 104.751 µs (28.6%)
iast_FULL 534.577 µs [513.89 µs, 555.264 µs] 168.475 µs (46.0%)
iast_INACTIVE 442.485 µs [421.738 µs, 463.233 µs] 76.383 µs (20.9%)
iast_TELEMETRY_OFF 467.584 µs [447.097 µs, 488.071 µs] 101.482 µs (27.7%)
tracing 441.406 µs [420.238 µs, 462.575 µs] 75.304 µs (20.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.573 µs [341.574 µs, 381.572 µs] -
iast 470.43 µs [449.911 µs, 490.95 µs] 108.857 µs (30.1%)
iast_FULL 533.271 µs [512.772 µs, 553.771 µs] 171.698 µs (47.5%)
iast_INACTIVE 445.973 µs [425.259 µs, 466.688 µs] 84.4 µs (23.3%)
iast_TELEMETRY_OFF 465.799 µs [445.267 µs, 486.331 µs] 104.226 µs (28.8%)
tracing 436.331 µs [416.014 µs, 456.647 µs] 74.758 µs (20.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.28.0-SNAPSHOT~39a26b859c, baseline=1.28.0-SNAPSHOT~656c6600e0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.36 ms) : 1341, 1379
.   : milestone, 1360,
appsec (1.77 ms) : 1745, 1796
.   : milestone, 1770,
iast (1.531 ms) : 1506, 1555
.   : milestone, 1531,
profiling (1.522 ms) : 1496, 1547
.   : milestone, 1522,
tracing (1.486 ms) : 1461, 1510
.   : milestone, 1486,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.747 ms) : 1722, 1772
.   : milestone, 1747,
iast (1.494 ms) : 1470, 1519
.   : milestone, 1494,
profiling (1.501 ms) : 1476, 1527
.   : milestone, 1501,
tracing (1.497 ms) : 1472, 1521
.   : milestone, 1497,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.36 ms [1.341 ms, 1.379 ms] -
appsec 1.77 ms [1.745 ms, 1.796 ms] 410.197 µs (30.2%)
iast 1.531 ms [1.506 ms, 1.555 ms] 170.744 µs (12.6%)
profiling 1.522 ms [1.496 ms, 1.547 ms] 161.657 µs (11.9%)
tracing 1.486 ms [1.461 ms, 1.51 ms] 125.74 µs (9.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.747 ms [1.722 ms, 1.772 ms] 392.381 µs (29.0%)
iast 1.494 ms [1.47 ms, 1.519 ms] 139.506 µs (10.3%)
profiling 1.501 ms [1.476 ms, 1.527 ms] 146.128 µs (10.8%)
tracing 1.497 ms [1.472 ms, 1.521 ms] 141.935 µs (10.5%)

@bantonsson bantonsson marked this pull request as ready for review January 12, 2024 13:48
@bantonsson bantonsson requested a review from a team as a code owner January 12, 2024 13:48
@bantonsson bantonsson enabled auto-merge January 12, 2024 13:51
@bantonsson bantonsson disabled auto-merge January 12, 2024 15:16
@bantonsson bantonsson merged commit f471f93 into master Jan 12, 2024
@bantonsson bantonsson deleted the ban/no-process-spans-for-agent branch January 12, 2024 15:16
@github-actions github-actions bot added this to the 1.28.0 milestone Jan 12, 2024
@bantonsson bantonsson added inst: others All other instrumentations and removed tag: no release notes Changes to exclude from release notes labels Jan 12, 2024
@smola smola mentioned this pull request Jan 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@smola smola smola approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh was automatically assigned from DataDog/apm-java

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer was automatically assigned from DataDog/apm-java

+1 more reviewer

@richardstartin richardstartin richardstartin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

inst: others All other instrumentations run-tests: all Run all tests

Projects

None yet

Milestone

1.28.0

Development

Successfully merging this pull request may close these issues.

4 participants

@bantonsson @smola @richardstartin