Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions content/providers/05-observability/helicone.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
---
title: Helicone
description: Monitor and optimize your AI SDK applications with minimal configuration using Helicone
---

# Helicone Observability

[Helicone](https://helicone.ai) is an open-source LLM observability platform that helps you monitor, analyze, and optimize your AI applications through a proxy-based approach, requiring minimal setup and zero additional dependencies.

## Setup

Setting up Helicone:

1. Create a Helicone account at [helicone.ai](https://helicone.ai)
2. Set your API key as an environment variable:
```bash filename=".env"
HELICONE_API_KEY=your-helicone-api-key
```
3. Update your model provider configuration to use Helicone's proxy:

```javascript
import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
baseURL: 'https://oai.helicone.ai/v1',
headers: {
'Helicone-Auth': `Bearer ${process.env.HELICONE_API_KEY}`,
},
});

// Use normally with AI SDK
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Hello world',
});
```

That's it! Your requests are now being logged and monitored through Helicone.

[→ Learn more about getting started with Helicone on AI SDK](https://docs.helicone.ai/getting-started/integration-method/vercelai)

## Integration Approach

While other observability solutions require OpenTelemetry instrumentation, Helicone uses a simple proxy approach:

<Tabs items={['Helicone Proxy (3 lines)', 'Typical OTEL Setup (simplified)']}>
<Tab>
```javascript
const openai = createOpenAI({
baseURL: "https://oai.helicone.ai/v1",
headers: { "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}` },
});
```
</Tab>

<Tab>
```javascript
// Install multiple packages
// @vercel/otel, @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node, etc.

// Create exporter
const exporter = new OtherProviderExporter({
projectApiKey: process.env.API_KEY
});

// Setup SDK
const sdk = new NodeSDK({
traceExporter: exporter,
instrumentations: [getNodeAutoInstrumentations()],
resource: new Resource({...}),
});

// Start SDK
sdk.start();

// Enable telemetry on each request
const response = await generateText({
model: openai("gpt-4o-mini"),
prompt: "Hello world",
experimental_telemetry: { isEnabled: true }
});

// Shutdown SDK to flush traces
await sdk.shutdown();
```
</Tab>
</Tabs>

**Characteristics of Helicone's Proxy Approach:**

- No additional packages required
- Compatible with JavaScript environments
- Minimal code changes to existing implementations
- Supports features such as caching and rate limiting

[→ Learn more about Helicone's proxy approach](https://docs.helicone.ai/references/proxy-vs-async)

## Core Features

### User Tracking

Monitor how individual users interact with your AI application:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Hello world',
headers: {
'Helicone-User-Id': 'user@example.com',
},
});
```

[→ Learn more about User Metrics](https://docs.helicone.ai/features/advanced-usage/user-metrics)

### Custom Properties

Add structured metadata to filter and analyze requests:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Translate this text to French',
headers: {
'Helicone-Property-Feature': 'translation',
'Helicone-Property-Source': 'mobile-app',
'Helicone-Property-Language': 'French',
},
});
```

[→ Learn more about Custom Properties](https://docs.helicone.ai/features/advanced-usage/custom-properties)

### Session Tracking

Group related requests into coherent conversations:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Tell me more about that',
headers: {
'Helicone-Session-Id': 'convo-123',
'Helicone-Session-Name': 'Travel Planning',
'Helicone-Session-Path': '/chats/travel',
},
});
```

[→ Learn more about Sessions](https://docs.helicone.ai/features/sessions)

## Advanced Configuration

### Request Caching

Reduce costs by caching identical requests:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'What is the capital of France?',
headers: {
'Helicone-Cache-Enabled': 'true',
},
});
```

[→ Learn more about Caching](https://docs.helicone.ai/features/advanced-usage/caching)

### Rate Limiting

Control usage by adding a rate limit policy:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Generate creative content',
headers: {
// Allow 10,000 requests per hour
'Helicone-RateLimit-Policy': '10000;w=3600',

// Optional: limit by user
'Helicone-User-Id': 'user@example.com',
},
});
```

Format: `[quota];w=[time_window];u=[unit];s=[segment]` where:

- `quota`: Maximum requests allowed in the time window
- `w`: Time window in seconds (minimum 60s)
- `u`: Optional unit - "request" (default) or "cents"
- `s`: Optional segment - "user", custom property, or global (default)

[→ Learn more about Rate Limiting](https://docs.helicone.ai/features/advanced-usage/custom-rate-limits)

### LLM Security

Protect against prompt injection, jailbreaking, and other LLM-specific threats:

```javascript
const response = await generateText({
model: openai('gpt-4o-mini'),
prompt: userInput,
headers: {
// Basic protection (Prompt Guard model)
'Helicone-LLM-Security-Enabled': 'true',

// Optional: Advanced protection (Llama Guard model)
'Helicone-LLM-Security-Advanced': 'true',
},
});
```

Protects against multiple attack vectors in 8 languages with minimal latency. Advanced mode adds protection across 14 threat categories.

[→ Learn more about LLM Security](https://docs.helicone.ai/features/advanced-usage/llm-security)

## Resources

- [Helicone Documentation](https://docs.helicone.ai)
- [GitHub Repository](https://github.com/Helicone/helicone)
- [Discord Community](https://discord.com/invite/2TkeWdXNPQ)
1 change: 1 addition & 0 deletions content/providers/05-observability/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ description: AI SDK Integration for monitoring and tracing LLM applications
Several LLM observability providers offer integrations with the AI SDK telemetry data:

- [Braintrust](/providers/observability/braintrust)
- [Helicone](/providers/observability/helicone)
- [Traceloop](/providers/observability/traceloop)
- [Langfuse](/providers/observability/langfuse)
- [LangSmith](/providers/observability/langsmith)
Expand Down
Loading