Kubernetes executionconfig API should return effective config#19014
Kubernetes executionconfig API should return effective config#19014GWphua wants to merge 10 commits intoapache:masterfrom
Conversation
|
the behaviour of this API changes. please also update the document |
| KubernetesTaskRunnerDynamicConfig currentConfig = getDynamicConfig(); | ||
| KubernetesTaskRunnerDynamicConfig mergedConfig = dynamicConfig; | ||
| KubernetesTaskRunnerDynamicConfig currentConfig = getCurrentConfiguration(); | ||
| KubernetesTaskRunnerDynamicConfig mergedConfig = currentConfig.merge(dynamicConfig); |
There was a problem hiding this comment.
P2 POST persists static fallback values as dynamic overrides
Merging the request into getCurrentConfiguration() means the value saved to metadata is the effective config, not just the dynamic override. For example, with no dynamic config and a static druid.indexer.runner.capacity of 10, a POST that only changes podTemplateSelectStrategy will persist capacity=10 as dynamic config. If the operator later changes the static capacity to 20 and restarts, the persisted dynamic capacity still wins, so the task runner unexpectedly stays capped at 10. The POST path should merge against the persisted dynamic config, or otherwise avoid writing static fallback values into dynamic config.
Description
#18591 introduced the capability to change the max number of K8s tasks created by Overlord, and we can track the values using
/druid/indexer/v1/k8s/taskrunner/executionconfigAPI.Execution Config Returns Static Config by Default
Currently, calling GET on the
/druid/indexer/v1/k8s/taskrunner/executionconfigAPI will return an empty payload if we did not submit any dynamic configs via POST on the same API. Users who depend on the API to retrieve the currently effective K8s task runner information will not see the information before the first dynamic config change is submitted.Changed the API to return the effective config, so the users will always see what is currently being used.
Release note
Prior to this change, GET
/druid/indexer/v1/k8s/taskrunner/executionconfigwill return an empty value when the dynamic config has not been updated via POST/druid/indexer/v1/k8s/taskrunner/executionconfig. This PR changes this behaviour to always reflect the dynamic config that will be used by the task runner to create K8s jobs.This PR has: