Skip to content

Add new metric that quantifies how long batch ingest jobs waited for segment availability and whether or not that wait was successful#12002

Merged
capistrant merged 5 commits intoapache:masterfrom
capistrant:handoff-wait-metric
Dec 10, 2021
Merged

Add new metric that quantifies how long batch ingest jobs waited for segment availability and whether or not that wait was successful#12002
capistrant merged 5 commits intoapache:masterfrom
capistrant:handoff-wait-metric

Conversation

@capistrant
Copy link
Copy Markdown
Contributor

@capistrant capistrant commented Nov 29, 2021

Description

Supplement to #10676 and #11090 .... Adds a new metric that is emitted as a batch task completes it's wait for segment availability. The metric value is the milliseconds waited. The new dimension is a boolean that states if the wait was successful or not. The value of this metric is that it tells operators how long the coordinator is taking to distribute segments to historicals after indexing completes. As an operator myself, I track this value to make sure our coordinator performance is up to the expectation of users when it comes to how long after indexing completes their data is available.


Key changed/added classes in this PR
  • AbstractBatchIndexTask
  • IndexTaskTest

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • been tested in a test Druid cluster.

Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this @capistrant, seems like a useful metric!

Copy link
Copy Markdown
Contributor

@a2l007 a2l007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 after CI

@capistrant
Copy link
Copy Markdown
Contributor Author

Thanks for the review, @kfaraz and @a2l007 .. This metric has come in handy for my clusters as I work to make sure coordination is staying performant. I hope others find it useful as well

@capistrant capistrant merged commit 761fe9f into apache:master Dec 10, 2021
@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants