Skip to content

Conversation

@delner
Copy link
Collaborator

@delner delner commented Jan 7, 2026

Purpose

This PR migrates the OpenAI instrumentation from a monolithic implementation (lib/braintrust/trace/contrib/openai.rb) to the new Integration API framework. By putting this behind a consistent programmatic interface, it will help facilitate the implementation of auto-instrumentation.

Architectural Overview

The new integration uses a three-layer architecture:

Integration → Patcher(s) → Instrumentation Modules

1. Integration (lib/braintrust/contrib/openai/integration.rb)

  • Defines metadata: gem names, version requirements, detection logic
  • Implements instrument! for high-level API
  • Delegates to one or more Patchers

2. Patcher (lib/braintrust/contrib/openai/patcher.rb)

  • ChatPatcher - Patches chat.completions methods
  • ResponsesPatcher - Patches responses API (new OpenAI feature)
  • Thread-safe via per-patcher mutex
  • Handles both class-level and instance-level patching

3. Instrumentation Modules (lib/braintrust/contrib/openai/instrumentation/)

  • Chat::Completions - Wraps create(), stream(), stream_raw()
  • Responses - Wraps responses API methods
  • Common - Shared utilities for parsing usage, aggregating streams

The instrumentation modules use Module#prepend to wrap methods, creating OpenTelemetry spans with input/output/metrics attributes.

Usage Examples

Class-level instrumentation (all clients):

require "braintrust"
require "openai"

Braintrust.init
Braintrust.instrument!(:openai)

# All OpenAI::Client instances are now traced
client = OpenAI::Client.new
client.chat.completions.create(...)  # Automatically traced

Instance-level instrumentation (specific client):

require "braintrust"
require "openai"

Braintrust.init

client_traced = OpenAI::Client.new
client_untraced = OpenAI::Client.new

# Only instrument the specific client
Braintrust.instrument!(:openai, target: client_traced)

client_traced.chat.completions.create(...)    # ✓ Traced
client_untraced.chat.completions.create(...)  # ✗ Not traced

Files Changed

  • Removed: lib/braintrust/trace/contrib/openai.rb (611 lines) - Monolithic implementation
  • Added: Modular structure under lib/braintrust/contrib/openai/:
    • integration.rb - Integration metadata and discovery
    • patcher.rb - ChatPatcher and ResponsesPatcher classes
    • instrumentation/chat.rb - Chat completions instrumentation
    • instrumentation/responses.rb - Responses API instrumentation
    • instrumentation/common.rb - Shared utilities
  • Added: Comprehensive test coverage with VCR cassettes for streaming and responses API

@delner delner requested review from clutchski and realark January 7, 2026 19:40
@delner delner self-assigned this Jan 7, 2026
@delner delner added the enhancement New feature or request label Jan 7, 2026
@realark
Copy link
Contributor

realark commented Jan 7, 2026

@delner delner force-pushed the auto_instrument/openai branch from 9121056 to 6076afe Compare January 8, 2026 18:31
@delner delner merged commit e52f321 into feature/auto_instrument Jan 9, 2026
@delner delner deleted the auto_instrument/openai branch January 9, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants