Skip to content

Process retrieval of parent and child segment ids in batches#16734

Merged
kfaraz merged 1 commit intoapache:masterfrom
AmatyaAvadhanula:retrieve_segment_upgrades_in_batches
Jul 15, 2024
Merged

Process retrieval of parent and child segment ids in batches#16734
kfaraz merged 1 commit intoapache:masterfrom
AmatyaAvadhanula:retrieve_segment_upgrades_in_batches

Conversation

@AmatyaAvadhanula
Copy link
Copy Markdown
Contributor

#16667 introduced two new metadata store queries to fetch parent ids and children ids corresponding to a given set of input segment ids using an IN clause.

This PR modifies these methods to process segmentIds in batches of size at most 100 for better performance and to hold the read transactions for shorter periods of time.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@AmatyaAvadhanula AmatyaAvadhanula requested a review from kfaraz July 15, 2024 10:14
@kfaraz kfaraz merged commit 6891866 into apache:master Jul 15, 2024
@kfaraz kfaraz deleted the retrieve_segment_upgrades_in_batches branch July 15, 2024 12:54
@kfaraz
Copy link
Copy Markdown
Contributor

kfaraz commented Jul 15, 2024

Thanks for the quick fix, @AmatyaAvadhanula !

@AmatyaAvadhanula
Copy link
Copy Markdown
Contributor Author

Thanks for the review, @kfaraz!

sreemanamala pushed a commit to sreemanamala/druid that referenced this pull request Aug 6, 2024
@kfaraz kfaraz added this to the 31.0.0 milestone Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants