diff --git a/content/providers/05-observability/helicone.mdx b/content/providers/05-observability/helicone.mdx new file mode 100644 index 000000000000..cbf02779bfe0 --- /dev/null +++ b/content/providers/05-observability/helicone.mdx @@ -0,0 +1,223 @@ +--- +title: Helicone +description: Monitor and optimize your AI SDK applications with minimal configuration using Helicone +--- + +# Helicone Observability + +[Helicone](https://helicone.ai) is an open-source LLM observability platform that helps you monitor, analyze, and optimize your AI applications through a proxy-based approach, requiring minimal setup and zero additional dependencies. + +## Setup + +Setting up Helicone: + +1. Create a Helicone account at [helicone.ai](https://helicone.ai) +2. Set your API key as an environment variable: + ```bash filename=".env" + HELICONE_API_KEY=your-helicone-api-key + ``` +3. Update your model provider configuration to use Helicone's proxy: + + ```javascript + import { createOpenAI } from '@ai-sdk/openai'; + + const openai = createOpenAI({ + baseURL: 'https://oai.helicone.ai/v1', + headers: { + 'Helicone-Auth': `Bearer ${process.env.HELICONE_API_KEY}`, + }, + }); + + // Use normally with AI SDK + const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'Hello world', + }); + ``` + +That's it! Your requests are now being logged and monitored through Helicone. + +[→ Learn more about getting started with Helicone on AI SDK](https://docs.helicone.ai/getting-started/integration-method/vercelai) + +## Integration Approach + +While other observability solutions require OpenTelemetry instrumentation, Helicone uses a simple proxy approach: + + + + ```javascript + const openai = createOpenAI({ + baseURL: "https://oai.helicone.ai/v1", + headers: { "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}` }, + }); + ``` + + + + ```javascript + // Install multiple packages + // @vercel/otel, @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node, etc. + + // Create exporter + const exporter = new OtherProviderExporter({ + projectApiKey: process.env.API_KEY + }); + + // Setup SDK + const sdk = new NodeSDK({ + traceExporter: exporter, + instrumentations: [getNodeAutoInstrumentations()], + resource: new Resource({...}), + }); + + // Start SDK + sdk.start(); + + // Enable telemetry on each request + const response = await generateText({ + model: openai("gpt-4o-mini"), + prompt: "Hello world", + experimental_telemetry: { isEnabled: true } + }); + + // Shutdown SDK to flush traces + await sdk.shutdown(); + ``` + + + +**Characteristics of Helicone's Proxy Approach:** + +- No additional packages required +- Compatible with JavaScript environments +- Minimal code changes to existing implementations +- Supports features such as caching and rate limiting + +[→ Learn more about Helicone's proxy approach](https://docs.helicone.ai/references/proxy-vs-async) + +## Core Features + +### User Tracking + +Monitor how individual users interact with your AI application: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'Hello world', + headers: { + 'Helicone-User-Id': 'user@example.com', + }, +}); +``` + +[→ Learn more about User Metrics](https://docs.helicone.ai/features/advanced-usage/user-metrics) + +### Custom Properties + +Add structured metadata to filter and analyze requests: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'Translate this text to French', + headers: { + 'Helicone-Property-Feature': 'translation', + 'Helicone-Property-Source': 'mobile-app', + 'Helicone-Property-Language': 'French', + }, +}); +``` + +[→ Learn more about Custom Properties](https://docs.helicone.ai/features/advanced-usage/custom-properties) + +### Session Tracking + +Group related requests into coherent conversations: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'Tell me more about that', + headers: { + 'Helicone-Session-Id': 'convo-123', + 'Helicone-Session-Name': 'Travel Planning', + 'Helicone-Session-Path': '/chats/travel', + }, +}); +``` + +[→ Learn more about Sessions](https://docs.helicone.ai/features/sessions) + +## Advanced Configuration + +### Request Caching + +Reduce costs by caching identical requests: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'What is the capital of France?', + headers: { + 'Helicone-Cache-Enabled': 'true', + }, +}); +``` + +[→ Learn more about Caching](https://docs.helicone.ai/features/advanced-usage/caching) + +### Rate Limiting + +Control usage by adding a rate limit policy: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: 'Generate creative content', + headers: { + // Allow 10,000 requests per hour + 'Helicone-RateLimit-Policy': '10000;w=3600', + + // Optional: limit by user + 'Helicone-User-Id': 'user@example.com', + }, +}); +``` + +Format: `[quota];w=[time_window];u=[unit];s=[segment]` where: + +- `quota`: Maximum requests allowed in the time window +- `w`: Time window in seconds (minimum 60s) +- `u`: Optional unit - "request" (default) or "cents" +- `s`: Optional segment - "user", custom property, or global (default) + +[→ Learn more about Rate Limiting](https://docs.helicone.ai/features/advanced-usage/custom-rate-limits) + +### LLM Security + +Protect against prompt injection, jailbreaking, and other LLM-specific threats: + +```javascript +const response = await generateText({ + model: openai('gpt-4o-mini'), + prompt: userInput, + headers: { + // Basic protection (Prompt Guard model) + 'Helicone-LLM-Security-Enabled': 'true', + + // Optional: Advanced protection (Llama Guard model) + 'Helicone-LLM-Security-Advanced': 'true', + }, +}); +``` + +Protects against multiple attack vectors in 8 languages with minimal latency. Advanced mode adds protection across 14 threat categories. + +[→ Learn more about LLM Security](https://docs.helicone.ai/features/advanced-usage/llm-security) + +## Resources + +- [Helicone Documentation](https://docs.helicone.ai) +- [GitHub Repository](https://github.com/Helicone/helicone) +- [Discord Community](https://discord.com/invite/2TkeWdXNPQ) diff --git a/content/providers/05-observability/index.mdx b/content/providers/05-observability/index.mdx index fa54df2979d7..2f846d3a748d 100644 --- a/content/providers/05-observability/index.mdx +++ b/content/providers/05-observability/index.mdx @@ -8,6 +8,7 @@ description: AI SDK Integration for monitoring and tracing LLM applications Several LLM observability providers offer integrations with the AI SDK telemetry data: - [Braintrust](/providers/observability/braintrust) +- [Helicone](/providers/observability/helicone) - [Traceloop](/providers/observability/traceloop) - [Langfuse](/providers/observability/langfuse) - [LangSmith](/providers/observability/langsmith)