Skip to content

K8s mm less fixes#14028

Merged
abhishekagarwal87 merged 18 commits intoapache:masterfrom
nlippis:k8s-mm-less-fixes
Apr 5, 2023
Merged

K8s mm less fixes#14028
abhishekagarwal87 merged 18 commits intoapache:masterfrom
nlippis:k8s-mm-less-fixes

Conversation

@nlippis
Copy link
Copy Markdown
Contributor

@nlippis nlippis commented Apr 4, 2023

Update Fabric8 version and allow metrics monitors to be overriden

This PR pull in the same code for #13804 and includes the code to upgrade the PodTemplateTaskAdapter.

This PR contains 2 items.

  1. Upgrade the k8s client to support newer versions of k8s.
  2. Ability to override specific Metrics Monitors
    We found that when we used the TaskCountStatsMonitor in the overlord config, the peon tasks would not start because they inherit the monitors from the parent process, which used to be the Middle Manager (but that would never have that monitor originally). So now you can override this value with the following config:

druid.indexer.runner.peonMonitors

Release note

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

class LogWatchInputStreamTest

Check notice

Code scanning / CodeQL

Unused classes and interfaces

Unused class: LogWatchInputStreamTest is not referenced within this codebase. If not used as an external API it should be removed.
@Override
public void updateStatus(Task task, TaskStatus status)
{
log.info("Updating task: %s with status %s", task.getId(), status);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this log line left in on purpose?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was part of the original PR, IMO this should be a debug log.

.waitUntilCondition(
x -> (x == null) || (x.getStatus() != null && x.getStatus().getActive() == null),
x -> (x == null) || (x.getStatus() != null && x.getStatus().getActive() == null
&& (x.getStatus().getFailed() != null || x.getStatus().getSucceeded() != null)),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes sense to me but what was the reasoning for adding this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More context here

Copy link
Copy Markdown
Contributor

@abhishekagarwal87 abhishekagarwal87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the additional commits on top of the previous PR (#13804) that was already approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants