Skip to content

Overlord can support many autoscalers of different categories#8989

Closed
VladimirIordanov wants to merge 23 commits intoapache:masterfrom
VladimirIordanov:feature-8695
Closed

Overlord can support many autoscalers of different categories#8989
VladimirIordanov wants to merge 23 commits intoapache:masterfrom
VladimirIordanov:feature-8695

Conversation

@VladimirIordanov
Copy link
Copy Markdown

@VladimirIordanov VladimirIordanov commented Dec 4, 2019

Fixes #8695.

Description

Added support of many autoscalers of different categories to serve tasks of according category.


This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths.
  • added integration tests.
  • been tested in a test Druid cluster.

Key changed/added classes in this PR
  • CategoriedWorkerBehaviorConfig
  • PendingTaskBasedWorkerProvisioningStrategy
  • SimpleWorkerProvisioningStrategy

Vladimir Iordanov added 18 commits December 4, 2019 15:37
 - Changed behavior config structure.
 - Small refactoring
 - Extended unit tests
 - Added "category" field into Autoscaler
 - Changed config format to support the "category" field
 - Fixed parameters of unit tests
 - Put category information into Autoscaler
 - Changed structure of behavior config
 - Changed CategoriedProvisioningStrategy accordingly
 - Fixed bug in PendingTaskBasedWorkerProvisioningStrategy
 - Extended unit tests
 - Code cleanup and refactoring
 - Introduced new logic into the existing strategies
 - Added more test cases
 - Refactored unit tests
@VladimirIordanov VladimirIordanov changed the title Feature 8695 Support different types of MiddleManagers by Autoscaler Dec 10, 2019
@VladimirIordanov VladimirIordanov changed the title Support different types of MiddleManagers by Autoscaler Support different types of Autoscalers Dec 11, 2019
@VladimirIordanov
Copy link
Copy Markdown
Author

Hi @gianm. Any chance for this PR to be reviewed?

Vladimir Iordanov added 5 commits January 29, 2020 12:10
 - Used Default Worker Category as a fallback for null worker categories
 - Refactored legacy code to avoid NPE
 - Removed duplicates
 - Added more debug info
 - Removed duplicated code
@VladimirIordanov VladimirIordanov requested review from himanshug and removed request for QiuMM February 7, 2020 11:35
@VladimirIordanov VladimirIordanov changed the title Support different types of Autoscalers Overlord can support many autoscalers of different categories Feb 7, 2020
sascha-coenen added a commit to smaato/druid that referenced this pull request Feb 11, 2020
Overlord can support many autoscalers of different categories apache#8989
@himanshug
Copy link
Copy Markdown
Contributor

I will try and review #9350 over the weekend or next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Overlord can support many autoscalers of different categories

2 participants