Skip to content

LLM Proxy

Jump to bottom
Nick edited this page Dec 26, 2025 · 1 revision

LLM Proxy - Safe-to-LLM Gateway

Automatic PII redaction for OpenAI-compatible APIs. Drop-in replacement for /v1/chat/completions.

Overview

Masker can act as a transparent proxy for OpenAI, Claude, and other LLMs. It automatically:

  • Detects and redacts PII from all messages
  • Forwards cleaned requests to upstream LLM
  • Returns responses with redaction metadata
  • Logs audit data (without storing raw text)

Quick Start

import openai

client = openai.OpenAI(
    base_url="https://masker.kikuai.dev/v1",
    api_key="your-openai-api-key",
    default_headers={"X-Api-Key": "your-masker-api-key"}
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "My email is john@example.com"}]
)
# PII automatically redacted before reaching OpenAI!

Configuration

Environment Variables

Variable Default Description
MASKER_API_KEYS "" API keys (format: key1:tenant1,key2:tenant2)
MASKER_UPSTREAM_URL OpenAI Upstream LLM endpoint
MASKER_UPSTREAM_TIMEOUT 60 Timeout in seconds
MASKER_DEFAULT_FAIL_MODE closed closed or open
MASKER_AUDIT_ENABLED true Enable audit logging
MASKER_POLICIES_DIR ./policies Policy YAML files

Authentication

The proxy uses X-Api-Key header for authentication:

curl -X POST "https://masker.kikuai.dev/v1/chat/completions" \
  -H "X-Api-Key: your-masker-key" \
  -H "Authorization: Bearer your-openai-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [...]}'

Policy System

Policies control how PII is redacted. Create YAML files in policies/ directory:

# policies/strict.yaml
version: 1
categories:
  email: drop      # Remove entirely
  phone: drop
  card: drop
  person: mask     # Replace with ***
fail_mode: closed  # Block on error

Use policy_id in requests:

{
  "model": "gpt-4",
  "messages": [...],
  "policy_id": "strict"
}

Actions

Action Description Result
mask Replace with *** ***
placeholder Replace with <TYPE> <EMAIL>
hash Replace with hash [a1b2c3d4]
drop Remove entirely (empty)
keep Do not redact (original)

Fail Modes

Mode Behavior
closed Block request if redaction fails
open Forward as-is if redaction fails (logged)

Response Metadata

Responses include _redaction field with audit data:

{
  "id": "chatcmpl-xxx",
  "choices": [...],
  "_redaction": {
    "request_id": "uuid",
    "entities_total": 2,
    "entities_by_type": {"EMAIL": 1, "PERSON": 1},
    "policy_id": "default",
    "redaction_ms": 15.5,
    "total_ms": 1250.0
  }
}

Audit Logging

Audit logs are written to JSONL files (daily rotation):

{
  "request_id": "uuid",
  "timestamp": "2025-01-01T00:00:00Z",
  "tenant_id": "my_company",
  "endpoint": "/v1/chat/completions",
  "entities_total": 5,
  "entities_by_type": {"EMAIL": 2, "PERSON": 3},
  "policy_id": "default",
  "redaction_ms": 25.5,
  "upstream_ms": 1200.0,
  "upstream_status": 200
}

Note: Raw text is NEVER logged.

Self-Hosted Deployment

# Clone repository
git clone https://github.com/KikuAI-Lab/masker.git
cd masker

# Configure
cp .env.example .env
# Edit .env with your settings

# Run
docker-compose up -d

Error Codes

Code Description
401 Missing or invalid API key
502 Upstream LLM error
503 Redaction failed (fail-closed mode)
504 Upstream timeout

Clone this wiki locally