Skip to content

Aj/dsd channel aggregator#801

Merged
litianningdatadog merged 12 commits intomainfrom
aj/dsd-channel-aggregator
Aug 29, 2025
Merged

Aj/dsd channel aggregator#801
litianningdatadog merged 12 commits intomainfrom
aj/dsd-channel-aggregator

Conversation

@astuyve
Copy link
Copy Markdown
Contributor

@astuyve astuyve commented Aug 25, 2025

  1. Upgrades dogstatsd-rs so that we use the channel based, lock-free aggregator
  2. sets the context size to 5k contexts instead of 5, which is helpful for very very fast functions

TODO update git sha when we merge DataDog/serverless-components#38

@astuyve astuyve requested a review from a team as a code owner August 25, 2025 17:47
@@ -134,7 +134,7 @@ impl Default for ContextBuffer {
/// Creates a new `ContextBuffer` with a default capacity of 5.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Creates a new `ContextBuffer` with a default capacity of 5.
/// Creates a new `ContextBuffer` with a default capacity of 5000.

.expect("lock poisoned")
.insert(metric)
{
if let Err(e) = self.aggr_handle.insert_batch(vec![metric]) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we add another method where its just an insert so we don't allocate a vector all the time?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but a better option in this case will be to just insert them all in one call haha

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh but also the metric itself is heap allocated b/c it has strings and other nonconstant struct members so the vec wrapper is likely minimal actual overhead

Copy link
Copy Markdown
Contributor

@duncanista duncanista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good, I just don't like the vec[] allocation on every insert, wondering if we could just add another command for Insert w/o a batch, are even adding batches here at all?

@@ -134,7 +134,7 @@ impl Default for ContextBuffer {
/// Creates a new `ContextBuffer` with a default capacity of 5.
///
fn default() -> Self {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add some comment to explain why bumping up to a much bigger value?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

@litianningdatadog litianningdatadog force-pushed the aj/dsd-channel-aggregator branch from 6079f34 to fde9204 Compare August 29, 2025 03:46
@litianningdatadog litianningdatadog force-pushed the aj/dsd-channel-aggregator branch from fde9204 to b70af0c Compare August 29, 2025 03:47
@litianningdatadog litianningdatadog merged commit af5b0f5 into main Aug 29, 2025
46 checks passed
@litianningdatadog litianningdatadog deleted the aj/dsd-channel-aggregator branch August 29, 2025 04:07
@litianningdatadog litianningdatadog restored the aj/dsd-channel-aggregator branch August 29, 2025 13:51
@litianningdatadog litianningdatadog deleted the aj/dsd-channel-aggregator branch August 29, 2025 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants