Skip to content

Add annotation for pod template#16772

Merged
georgew5656 merged 4 commits intoapache:masterfrom
georgew5656:addPodAnnotation
Jul 23, 2024
Merged

Add annotation for pod template#16772
georgew5656 merged 4 commits intoapache:masterfrom
georgew5656:addPodAnnotation

Conversation

@georgew5656
Copy link
Copy Markdown
Contributor

Add the template name as a annotation on the pod spec to tell which pods are running which templates in metrics.

Description

It can be useful to group k8s pods launched by mmless ingestion based on the template they used for metrics, logs, etc. To do this we can add the template name as a annotation.

Release note

  • In mmless ingestion, add the pod template name as a annotation to the created job/pod.
Key changed/added classes in this PR
  • PodTemplateTaskAdapter

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good, left minor suggestions.

* @return The pod template that should be used to run the task.
*/
@NotNull PodTemplate getPodTemplateForTask(Task task, Map<String, PodTemplate> templates);
@NotNull Pair<String, PodTemplate> getPodTemplateForTask(Task task, Map<String, PodTemplate> templates);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For code hygiene and readability, I would advise creating a dedicated class PodTemplateWithKey rather than use a Pair, as pairs are often difficult to interpret.

Also, the javadoc needs to be updated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

public Pair<String, PodTemplate> getPodTemplateForTask(Task task, Map<String, PodTemplate> templates)
{
return templates.getOrDefault(task.getType(), templates.get("base"));
String templateKey = templates.containsKey(task.getType()) ? task.getType() : "base";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "base" be kept as a constant somewhere?

.editTemplate()
.editOrNewMetadata()
.addToAnnotations(getPodTemplateAnnotations(task))
.addToAnnotations(DruidK8sConstants.TASK_JOB_TEMPLATE, podTemplatePair.lhs)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate line?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one line is adding it to the job spec and one to the pod template spec (which will become the created pod).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for the clarification, I had missed the editTemplate() invocation.

@georgew5656 georgew5656 requested a review from kfaraz July 22, 2024 16:37
Copy link
Copy Markdown
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 🚀

Can be merged once CI passes.

@georgew5656 georgew5656 merged commit a64e9a1 into apache:master Jul 23, 2024
sreemanamala pushed a commit to sreemanamala/druid that referenced this pull request Aug 6, 2024
* Add annotation for pod template

* pr comments

* add test cases

* add tests
@kfaraz kfaraz added this to the 31.0.0 milestone Oct 4, 2024
FrankChen021 pushed a commit that referenced this pull request Feb 3, 2025
* Add annotation for pod template

* pr comments

* add test cases

* add tests
GabrielCWT pushed a commit to GabrielCWT/druid that referenced this pull request Sep 9, 2025
* Add annotation for pod template

* pr comments

* add test cases

* add tests
RonShub pushed a commit to singular-labs/druid that referenced this pull request Oct 19, 2025
* Add annotation for pod template

* pr comments

* add test cases

* add tests
RonShub pushed a commit to singular-labs/druid that referenced this pull request Oct 20, 2025
* Add annotation for pod template

* pr comments

* add test cases

* add tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants