MLProbe

Pre-Deployment Model Validation & Reliability Testing

MLProbe is a Python package for validating ML models before deployment. It provides automated checks for model performance, data drift, failure modes, and CI/CD integration.

Quick Start

import pandas as pd
from sklearn.linear_model import LogisticRegression
from mlprobe.validate import ValidationSuite
from mlprobe.drift import DriftDetector
from mlprobe.failmode import FailureModeAnalyzer
from mlprobe.report import ReportGenerator
from mlprobe.gate import DeploymentGate

# 1. Validate model performance on production logs
suite = ValidationSuite(
    model=clf,
    production_logs=prod_df,
    label_col="label",
    segments=["region"],
    metrics=["accuracy", "f1", "auc"],
)
val_report = suite.run()
print(val_report.overall)

# 2. Detect feature drift
detector = DriftDetector(training_data=train_df, production_data=prod_df)
drift_report = detector.analyze(methods=["ks", "psi"], threshold=0.05)
if drift_report.has_critical:
    print("Critical drift detected!")

# 3. Find underperforming slices
analyzer = FailureModeAnalyzer(model=clf, test_data=test_df, label_col="label")
slices = analyzer.find_underperforming_slices(min_slice_size=50)

# 4. Generate a report
gen = ReportGenerator()
html = gen.generate(val_report, drift_report, slices, format="html")
with open("report.html", "w") as f:
    f.write(html)

# 5. CI/CD gate
gate = DeploymentGate()
result = gate.check(model=clf, data=prod_df, label_col="label", training_data=train_df)
import sys; sys.exit(result.exit_code)

Modules

Module	Class	Purpose
mlprobe.validate	ValidationSuite	Segment-level metrics (accuracy, precision, recall, f1, auc)
mlprobe.drift	DriftDetector	Feature/label drift via KS, PSI, chi-squared
mlprobe.failmode	FailureModeAnalyzer	Decision-tree slice discovery
mlprobe.report	ReportGenerator	HTML & Markdown reports with deploy recommendation
mlprobe.gate	DeploymentGate	CI/CD pass/fail/warn with exit codes
mlprobe.utils	—	Logging, Pydantic config, stats helpers

Installation

pip install mlprobe
# or with uv:
uv add mlprobe

Development

uv sync --all-extras
uv run pytest tests/ -v --cov=src --cov-report=term-missing
uv run isort .
uv run black .

License

Apache 2.0 — Copyright 2026 Ramakrishnan Sathyavageeswaran