Simplify DruidCoordinatorConfig and binding of metadata cleanup duties

[kfaraz]

Motivation

• Simplify coordinator config handling
• After deprecate config-magic in favor of json configuration stuff #14695 , something similar needs to be done for DruidCoordinatorConfig. This change would help build towards that.

Changes

• Move following configs from CliCoordinator to DruidCoordinatorConfig:
  • druid.coordinator.kill.on
  • druid.coordinator.kill.pendingSegments.on
  • druid.coordinator.kill.supervisors.on
  • druid.coordinator.kill.rules.on
  • druid.coordinator.kill.audit.on
  • druid.coordinator.kill.datasource.on
  • druid.coordinator.kill.compaction.on
• In the Coordinator style used by historical management duties, always instantiate all the metadata cleanup duties but execute only if enabled. In the existing code, they are instantiated only when enabled by using optional binding with Guice.
• Add a wrapper MetadataManager which contains handles to all the different metadata managers for rules, supervisors, segments, etc.
• Add a CoordinatorConfigManager to simplify read and update of coordinator configs
• Remove persistence related methods from CoordinatorCompactionConfig and CoordinatorDynamicConfig as these are config classes and persistence logic should not live here.
• Remove annotations @CoordinatorIndexingServiceDuty, @CoordinatorMetadataStoreManagementDuty

Release Note

• Druid extensions cannot bind custom coordinator duties to the duty groups IndexingServiceDuties and MetadataStoreManagementDuties anymore. These are meant to be core coordinator built-in flows and should not be affected by custom duties.
• Users can still define a CustomCoordinatorDuty with a custom duty group and period.

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[clintropolis]

does this mean extensions can no longer provide custom coordinator duties? If so this might require a dev list thread to see if anyone is using this functionality.

It does make sense to update the built-in ones to be handled in a more direct manner, but I wonder if there is some way we could leave this stuff in place to retain this functionality as well.

[clintropolis]

oops, forgot about CoordinatorCustomDuty, carry on

[kfaraz]

Thanks for the review, @clintropolis !!