Braintrust.init performs auto-instrumentation of LLM libraries

[delner]

Purpose

This PR introduces automatic LLM library instrumentation as part of Braintrust.init, making the SDK zero-configuration for most use cases. Previously, users had to manually call Braintrust.instrument!(:openai) (etc.) for each LLM library. Now, a single Braintrust.init call automatically detects and instruments all supported libraries.

Before:

require "braintrust"
require "openai"

Braintrust.init
Braintrust.instrument!(:openai)  # Manual step required

client = OpenAI::Client.new

After:

require "braintrust"
require "openai"

Braintrust.init  # Automatically instruments OpenAI

client = OpenAI::Client.new

Architectural Overview

New Components

Component	Location	Description
Braintrust.auto_instrument!	lib/braintrust/contrib.rb	Top-level API for triggering auto-instrumentation
Braintrust::Contrib.auto_instrument!	lib/braintrust/contrib.rb	Discovers and instruments available integrations
Braintrust::Internal::Env	lib/braintrust/internal/env.rb	Environment variable parsing utilities

Control Flow

Braintrust.init(auto_instrument: <config>)
    │
    ▼
Braintrust.auto_instrument!(<config>)
    │
    ├─► Resolve config (explicit vs env var)
    ├─► Resolve filters (only/except from config or env)
    │
    ▼
Braintrust::Contrib.auto_instrument!(only:, except:)
    │
    ├─► Get available integrations from registry
    ├─► Apply only/except filters
    │
    ▼
For each matching integration:
    └─► Braintrust::Contrib.instrument!(:name)

Environment Variables

Variable	Default	Description
BRAINTRUST_AUTO_INSTRUMENT	true	Set to "false" to disable
BRAINTRUST_INSTRUMENT_ONLY	(none)	Comma-separated whitelist (e.g., "openai,anthropic")
BRAINTRUST_INSTRUMENT_EXCEPT	(none)	Comma-separated blacklist (e.g., "ruby_llm")

Usage Examples

Default: Auto-instrument everything

Braintrust.init
# All detected LLM libraries are instrumented automatically

Disable auto-instrumentation

# Via code
Braintrust.init(auto_instrument: false)

# Via environment
BRAINTRUST_AUTO_INSTRUMENT=false

Selective instrumentation

# Only specific libraries
Braintrust.init(auto_instrument: { only: [:openai, :anthropic] })

# Exclude specific libraries
Braintrust.init(auto_instrument: { except: [:ruby_llm] })

# Via environment
BRAINTRUST_INSTRUMENT_ONLY=openai,anthropic
BRAINTRUST_INSTRUMENT_EXCEPT=ruby_llm

Standalone auto-instrument (without init)

# Use auto_instrument! directly for more control
Braintrust.auto_instrument!
Braintrust.auto_instrument!(only: [:openai])
Braintrust.auto_instrument!(false)  # Disable

Supported Integrations

Auto-instrumentation works with all registered integrations:

• :openai - OpenAI's official Ruby gem
• :ruby_openai - alexrudall's ruby-openai gem
• :anthropic - Anthropic's official Ruby gem
• :ruby_llm - RubyLLM unified interface

Key Design Decisions

1. Enabled by default: Auto-instrumentation is on unless explicitly disabled, providing the best out-of-box experience.

2. Config precedence: Explicit auto_instrument: parameter overrides environment variables, allowing code to take precedence when needed.

3. Graceful degradation: Only instruments libraries that are actually loaded; missing libraries are silently skipped.

4. Backward compatible: Existing Braintrust.instrument!(:name) calls continue to work. Users who prefer manual control can disable auto-instrumentation.

Files Changed

• lib/braintrust.rb - Added auto_instrument parameter to init
• lib/braintrust/contrib.rb - Added auto_instrument! methods at both levels
• lib/braintrust/internal/env.rb - New environment variable utilities
• README.md - Updated quick start and examples
• test/braintrust/contrib_test.rb - Comprehensive test coverage
• examples/setup/init.rb - Full demo of auto-instrumentation
• examples/setup/init_minimal.rb - Minimal example