feat(bottlecap): add invocation span by duncanista · Pull Request #394 · DataDog/datadog-lambda-extension

duncanista · 2024-09-13T18:17:33Z

What?

Adds the aws.lambda invocation span to Universally Instrumented runtimes like .NET, Go, Java, and Ruby.

Motivation

#374 and SVLS-5438

Testing

Added some unit tests, integration tests will be added later
Tested manually for .NET

Notes

Some tags will be missing from the top-level span

…onzalez/bottlecap/create-lambda-invocation-span

also added some more fields and refactored it

…ion into jordan.gonzalez/bottlecap/create-lambda-invocation-span

…to jordan.gonzalez/bottlecap/create-lambda-invocation-span

duncanista · 2024-09-13T18:25:43Z

+}
+
+#[allow(clippy::module_name_repetitions)]
+pub struct ContextBuffer {


I decided to use the ContextBuffer so we can keep relevant data from the past 5 invocations in cache.

5 is just an arbitrary number, we can reduce/increase this number as desired.

should we remove it once it's read and used? In theory this setup would allow us to drop invocation contexts which we haven't used for some reason yet.

When a PlatformRuntime done hits, we effectively remove the Context in question, is this what you mean?

What do you mean by drop invocation context which we haven't used for some reason yet?

duncanista · 2024-09-13T18:39:17Z

+            let span_size = std::mem::size_of_val(&self.span);
+            let header_tags = tracer_header_tags::TracerHeaderTags {
+                lang: "",
+                lang_version: "",
+                lang_interpreter: "",
+                lang_vendor: "",
+                tracer_version: "",
+                container_id: "",
+                client_computed_top_level: false,
+                client_computed_stats: false,
+            };


definitely missing a lot of data here

Added a todo comment, not sure how we can get these ones easily

duncanista · 2024-09-13T18:39:29Z

+            span.meta
+                .insert("request_id".to_string(), request_id.clone());
+        }


Missing a bunch of other tags here too

…to jordan.gonzalez/bottlecap/create-lambda-invocation-span

I think we dont need it

duncanista · 2024-09-23T17:31:56Z

+                                        lambda_enhanced_metrics
+                                            .set_runtime_duration_metric(metrics.duration_ms);
+                                    }


I might want to delegate some of this metrics to the lifecycle processor eventually, maybe?

it'd be good to push this out somewhere which can do the work asynchronously, you're blocking the loop right now

astuyve · 2024-09-23T19:37:49Z

+                        Event::Telemetry(event) =>
+                            match event.record {
+                                TelemetryRecord::PlatformStart { request_id, .. } => {
+                                    let mut p = invocation_processor.lock().await;


do we need a lock here? this prevents us from processing any messages until it's acquired, what's it fully protecting here?

Looking what improvements I can make here

As per our conversation, will move the lock out of this in another PR when the time comes, will add a ticket for this

* decouple `hyper` from `trace_processor` * add `handle_traces` * fix tests * removed unused import * move `invocation_context` to `invocation::context` module also added some more fields and refactored it * add `new` and `get_sender_copy` to `trace_agent` * add `get_canonical_resource_name` to `tags_provider` * add `get_function_name` to `lambda::tags` * add `MS_TO_NS` constant * add `invocation::processor` * update use of `invocation::context` * make `lifecycle::listener` to use `invocation::processor` * use `invocation::processor` in `main.rs` * move `MS_TO_NS` to `invocation::processor` * remove unnecessary constant * add `Box::new` back to `trace_agent` * add some comments * add unit tests for `context.rs` * use `on_invocation_start` * rename `lambda_library_detected` to `tracer_detected` * fmt * remove `current_request_id` I think we dont need it * add comment * fmt

duncanista added 16 commits September 10, 2024 15:05

decouple hyper from trace_processor

0d11207

add handle_traces

be1a1ae

fix tests

8525801

removed unused import

002e9eb

Merge branch 'jordan.gonzalez/decouple-trace-processor' into jordan.g…

2e37b81

…onzalez/bottlecap/create-lambda-invocation-span

move invocation_context to invocation::context module

c7de652

also added some more fields and refactored it

add new and get_sender_copy to trace_agent

7e1be93

add get_canonical_resource_name to tags_provider

11e6ba1

add get_function_name to lambda::tags

68d12ac

add MS_TO_NS constant

5070e19

add invocation::processor

2d4d60c

update use of invocation::context

7089693

make lifecycle::listener to use invocation::processor

929741c

use invocation::processor in main.rs

dcba48e

Merge branch 'main' of ssh://github.com/DataDog/datadog-lambda-extens…

807802b

…ion into jordan.gonzalez/bottlecap/create-lambda-invocation-span

Merge branch 'jordan.gonzalez/bottlecap/universal-instrumentation' in…

a4d1eb5

…to jordan.gonzalez/bottlecap/create-lambda-invocation-span

astuyve reviewed Sep 13, 2024

View reviewed changes

Comment thread bottlecap/src/metrics/enhanced/constants.rs Outdated

duncanista commented Sep 13, 2024

View reviewed changes

move MS_TO_NS to invocation::processor

57a155c

duncanista commented Sep 13, 2024

View reviewed changes

Comment thread bottlecap/src/bin/bottlecap/main.rs Outdated

duncanista commented Sep 13, 2024

View reviewed changes

remove unnecessary constant

8ffd181

duncanista commented Sep 13, 2024

View reviewed changes

Comment thread bottlecap/src/bin/bottlecap/main.rs

duncanista added 3 commits September 13, 2024 14:45

add Box::new back to trace_agent

c793148

Merge branch 'jordan.gonzalez/bottlecap/universal-instrumentation' in…

f815b22

…to jordan.gonzalez/bottlecap/create-lambda-invocation-span

add some comments

83762a0

duncanista marked this pull request as ready for review September 20, 2024 17:45

duncanista requested a review from a team as a code owner September 20, 2024 17:45

add unit tests for context.rs

4f23d59

duncanista added 4 commits September 20, 2024 15:39

use on_invocation_start

7918a32

rename lambda_library_detected to tracer_detected

fca14ec

fmt

9413e1c

remove current_request_id

f03ddd0

I think we dont need it

duncanista commented Sep 23, 2024

View reviewed changes

astuyve reviewed Sep 23, 2024

View reviewed changes

astuyve approved these changes Sep 23, 2024

View reviewed changes

duncanista added 2 commits September 23, 2024 16:56

add comment

cfbbbde

fmt

6a8effa

duncanista merged commit d4e37d2 into jordan.gonzalez/bottlecap/universal-instrumentation Sep 27, 2024

duncanista deleted the jordan.gonzalez/bottlecap/create-lambda-invocation-span branch September 27, 2024 20:39

astuyve mentioned this pull request Oct 21, 2024

APIGW v1 + httpAPI/v1 API parameterized routes #419

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(bottlecap): add invocation span#394

feat(bottlecap): add invocation span#394
duncanista merged 28 commits intojordan.gonzalez/bottlecap/universal-instrumentationfrom
jordan.gonzalez/bottlecap/create-lambda-invocation-span

duncanista commented Sep 13, 2024 •

edited by atlassian Bot

Loading

Uh oh!

Uh oh!

duncanista Sep 13, 2024

Uh oh!

astuyve Sep 23, 2024

Uh oh!

duncanista Sep 23, 2024

Uh oh!

Uh oh!

duncanista Sep 13, 2024

Uh oh!

duncanista Sep 20, 2024 •

edited

Loading

Uh oh!

duncanista Sep 13, 2024

Uh oh!

Uh oh!

duncanista Sep 23, 2024

Uh oh!

astuyve Sep 23, 2024

Uh oh!

astuyve Sep 23, 2024

Uh oh!

duncanista Sep 24, 2024

Uh oh!

duncanista Sep 24, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

duncanista commented Sep 13, 2024 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What?

Motivation

Testing

Notes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duncanista Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

duncanista commented Sep 13, 2024 •

edited by atlassian Bot

Loading

duncanista Sep 20, 2024 •

edited

Loading