Add deep storage segment metric

[georgew5656]

Addressing some followup comments from the change in the metric made in #16020

Description

If a datasource has a replicaCount=0 rule that applies to some segments that are not covered by a replicaCount>0 rule, those segments will not be reported as unavailable in segment/unavailable/count. Since they are not queryable by native queries (only using the query from deep storage feature), we want a separate metric that lets us check how many segments fit these criteria.

Fixed the bug ...

Renamed the class ...

Added a forbidden-apis entry ...

Add metric segment/deepStorage/count that counts the number of segments that are only queryable through the query from deep storage metric.

Release note

New metric to support query from deep storage feature.

Key changed/added classes in this PR

• DruidCoordinator
• Stats

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[kfaraz]

If a datasource has a replicaCount=0 rule that applies to some segments that are not covered by a replicaCount>0 rule, those segments will not be reported as available in segment/unavailable/count

After this patch, will such segments be reported as available or unavailable?
If they need to be reported as unavailable, would we need to remove the replicaCount.required() == 0 condition from the computation of unavailable/count?

[kfaraz]

Nit: Some of the code in this method may be commoned out with getDatasourceToUnavailableSegmentCount.

[kfaraz]

How about the metric name deepStorageQueryOnly or better yet availableDeepStorageOnly as it aligns better with unavailable/count?

[georgew5656]

If a datasource has a replicaCount=0 rule that applies to some segments that are not covered by a replicaCount>0 rule, those segments will not be reported as available in segment/unavailable/count

After this patch, will such segments be reported as available or unavailable? If they need to be reported as unavailable, would we need to remove the replicaCount.required() == 0 condition from the computation of unavailable/count?

that should say "not be reported as unavailable", my bad.

i think they should not be reported as unavailable b/c that metric is supposed to be 0 during steady state and enabling query from deep storage shouldn't change this

[suneet-s]

What is the purpose of this metric? The doc says it's typical value is varies - is there a good or bad value?

I can imagine tracking the size of cold data as a useful metric, but I'm not sure about the use of seeing how the count of cold segments changes over time will be helpful. Maybe some doc updates about that would help clarify the use of the metric.

[kfaraz]

i think they should not be reported as unavailable b/c that metric is supposed to be 0 during steady state

Thanks, @georgew5656 , the steady state reasoning makes sense to me.

@suneet-s , one use of this metric would be to find out if the number of segments that are not available for querying using the native engine. For example, if there are some segments that are available for querying from deep storage using MSQ, they would not be counted towards the unavailable/count, but these would still not be available to the native engine. The new metric would thus be reported as non-zero for such cases.

[kfaraz]

LGTM.
As @suneet-s requested, you may add more info in the docs. But the current definition of the metric also seems sufficient to me.