You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
go.opentelemetry.io/otel v1.43.0 is the Go implementation of the OpenTelemetry observability framework. This project uses the full suite: the core API (otel, otel/trace, otel/attribute, otel/codes, otel/propagation), the SDK (otel/sdk/trace, otel/sdk/resource, otel/semconv/v1.26.0), and the OTLP/HTTP exporter (otel/exporters/otlp/otlptrace/otlptracehttp).
Current Usage in gh-aw
The tracing integration lives primarily in internal/tracing/ and is used across the gateway's request pipeline.
The OTel integration is solid and follows most best practices:
✅ Noop provider fallback: When no OTLP endpoint is configured, a noop.NewTracerProvider() is used — zero overhead in production without tracing
✅ W3C TraceContext propagation: Both TraceContext and Baggage propagators are registered globally, enabling distributed trace continuation from upstream agents
✅ Sampler selection: Supports AlwaysSample, NeverSample, and TraceIDRatioBased with configurable rate — correct OTel pattern
✅ Batched export: sdktrace.WithBatcher(exporter) is used for async, buffered OTLP export (not WithSyncer)
✅ Error recording: Both span.RecordError(err) and span.SetStatus(codes.Error, reason) are used together — correct pattern
✅ SpanKinds: Correctly set to Server, Internal, or Client based on role
✅ Test infra: tracetest.InMemoryExporter and sdktrace.NewSimpleSpanProcessor used in unit tests — idiomatic
Recent Updates (v1.43.0)
OTel Go 1.43.0 is recent and stable. Key features available:
trace/noop package (stable) — already in use ✅
resource.WithTelemetrySDK() — adds SDK version info to resource
sdktrace.WithRawSpanLimits() for fine-grained span attribute/event limits
Improvement Opportunities
🏃 Quick Wins
1. Add semconv.ServiceVersion to resource (internal/tracing/provider.go)
The service version is available via version.Get() but is not included in the OTel resource. Tracing backends (Jaeger, Honeycomb, Datadog) use service version for deployment-aware tracing.
// In provider.go, import internal/versionres, err:=resource.New(ctx,
resource.WithAttributes(
semconv.ServiceName(serviceName),
semconv.ServiceVersion(version.Get()), // ← add this
),
resource.WithProcessPID(),
resource.WithHost(),
)
2. Add resource.WithSchemaURL(semconv.SchemaURL)
Per OTel spec, resources that use semconv attributes should declare their SchemaURL so backends can interpret them correctly. The semconv package already exports semconv.SchemaURL.
4. Cache tracer in unified.go and proxy/handler.go
Both files call tracing.Tracer() inside the request handler closure, which invokes otel.Tracer(instrumentationName) on every request. Caching at construction time is the established pattern used in http_helpers.go:
// Current (called per-request in unified.go and proxy/handler.go):ctx, toolSpan:=tracing.Tracer().Start(ctx, "mcp.tool_call", ...)
// Better (cache at server construction, like http_helpers.go does):t:=tracing.Tracer()
// ...inside handler:ctx, toolSpan:=t.Start(ctx, "mcp.tool_call", ...)
✨ Feature Opportunities
5. Add resource.WithTelemetrySDK()
OTel recommends including SDK version information in the resource for correlating telemetry with SDK upgrade effects:
The mcp.tool_call span in unified.go is described as "spanning all phases 0–6" but no events mark the boundaries between phases. Span events are the OTel-native way to capture lifecycle milestones within a span:
This makes distributed traces much more useful for diagnosing latency within tool calls.
📐 Best Practice Alignment
7. Consolidate WrapHTTPHandler and WithOTELTracing
Two nearly-identical span-wrapping helpers exist:
tracing.WrapHTTPHandler in internal/tracing/http.go — generic, uses explicit tracer, logs per-request
server.WithOTELTracing in internal/server/http_helpers.go — adds session.id post-request, used in production middleware stack
They perform the same W3C context extraction + span creation + SpanKindServer pattern. Consider having WithOTELTracing delegate to WrapHTTPHandler (with extra attrs for session.id), or consolidate into a single well-tested implementation.
Recommendations
Prioritized by impact-to-effort ratio:
🥇 Add semconv.ServiceVersion to resource — 1 line change, high diagnostic value, uses existing version.Get()
🥇 Add resource.WithSchemaURL(semconv.SchemaURL) — 1 line change, correctness improvement
🥈 Use semconv constants for HTTP attributes — straightforward refactor across 4 files, improves backend compatibility
🥈 Cache tracing.Tracer() at construction — minor cleanup, aligns with existing pattern
🎯 Span events for tool call phases — higher effort but significantly improves tracing utility for perf analysis
🎯 Consolidate span helpers — reduces duplication, improves test coverage
Next Steps
Apply quick wins (items 1–4) in a single PR — all in internal/tracing/provider.go and span call sites
Evaluate span events for mcp.tool_call phases as a separate observability improvement
Consider consolidating WrapHTTPHandler / WithOTELTracing as tech debt cleanup
Generated by Go Fan 🐹 · Round-robin selection after github.com/tetratelabs/wazero Module summary saved to: specs/mods/go-opentelemetry-otel.md (pending write access) Run: §24442419216
Note
🔒 Integrity filter blocked 11 items
The following items were blocked because they don't meet the GitHub integrity level.
🐹 Go Fan Report: OpenTelemetry Go SDK
Module Overview
go.opentelemetry.io/otelv1.43.0 is the Go implementation of the OpenTelemetry observability framework. This project uses the full suite: the core API (otel,otel/trace,otel/attribute,otel/codes,otel/propagation), the SDK (otel/sdk/trace,otel/sdk/resource,otel/semconv/v1.26.0), and the OTLP/HTTP exporter (otel/exporters/otlp/otlptrace/otlptracehttp).Current Usage in gh-aw
The tracing integration lives primarily in
internal/tracing/and is used across the gateway's request pipeline.TracerProvider,Tracer.Start,Span.End,Span.SetAttributes,Span.SetStatus,Span.RecordError,otel.SetTracerProvider,otel.GetTextMapPropagator,propagation.NewCompositeTextMapPropagator,sdktrace.NewTracerProvider,resource.New,otlptracehttp.NewSpans Instrumented
gateway.requestinternal/server/http_helpers.gomcp.tool_callinternal/server/unified.gogateway.backend.executeinternal/server/unified.goproxy.difc_pipelineinternal/proxy/handler.goproxy.backend.forwardinternal/proxy/handler.goResearch Findings
What's Working Well 🌟
The OTel integration is solid and follows most best practices:
noop.NewTracerProvider()is used — zero overhead in production without tracingTraceContextandBaggagepropagators are registered globally, enabling distributed trace continuation from upstream agentsAlwaysSample,NeverSample, andTraceIDRatioBasedwith configurable rate — correct OTel patternsdktrace.WithBatcher(exporter)is used for async, buffered OTLP export (notWithSyncer)span.RecordError(err)andspan.SetStatus(codes.Error, reason)are used together — correct patternServer,Internal, orClientbased on roletracetest.InMemoryExporterandsdktrace.NewSimpleSpanProcessorused in unit tests — idiomaticRecent Updates (v1.43.0)
OTel Go 1.43.0 is recent and stable. Key features available:
trace/nooppackage (stable) — already in use ✅resource.WithTelemetrySDK()— adds SDK version info to resourcesemconv/v1.26.0stable HTTP conventions (http.request.method,url.path,http.response.status_code)sdktrace.WithRawSpanLimits()for fine-grained span attribute/event limitsImprovement Opportunities
🏃 Quick Wins
1. Add
semconv.ServiceVersionto resource (internal/tracing/provider.go)The service version is available via
version.Get()but is not included in the OTel resource. Tracing backends (Jaeger, Honeycomb, Datadog) use service version for deployment-aware tracing.2. Add
resource.WithSchemaURL(semconv.SchemaURL)Per OTel spec, resources that use semconv attributes should declare their SchemaURL so backends can interpret them correctly. The semconv package already exports
semconv.SchemaURL.3. Use semconv constants for HTTP span attributes
Multiple call sites use raw attribute strings that don't match the stable semconv v1.26.0 HTTP conventions:
Affected files:
internal/server/http_helpers.go,internal/server/unified.go,internal/proxy/handler.go,internal/tracing/http.go.4. Cache tracer in
unified.goandproxy/handler.goBoth files call
tracing.Tracer()inside the request handler closure, which invokesotel.Tracer(instrumentationName)on every request. Caching at construction time is the established pattern used inhttp_helpers.go:✨ Feature Opportunities
5. Add
resource.WithTelemetrySDK()OTel recommends including SDK version information in the resource for correlating telemetry with SDK upgrade effects:
6. Add span events for tool call phase boundaries
The
mcp.tool_callspan inunified.gois described as "spanning all phases 0–6" but no events mark the boundaries between phases. Span events are the OTel-native way to capture lifecycle milestones within a span:This makes distributed traces much more useful for diagnosing latency within tool calls.
📐 Best Practice Alignment
7. Consolidate
WrapHTTPHandlerandWithOTELTracingTwo nearly-identical span-wrapping helpers exist:
tracing.WrapHTTPHandlerininternal/tracing/http.go— generic, uses explicit tracer, logs per-requestserver.WithOTELTracingininternal/server/http_helpers.go— addssession.idpost-request, used in production middleware stackThey perform the same W3C context extraction + span creation + SpanKindServer pattern. Consider having
WithOTELTracingdelegate toWrapHTTPHandler(with extra attrs forsession.id), or consolidate into a single well-tested implementation.Recommendations
Prioritized by impact-to-effort ratio:
semconv.ServiceVersionto resource — 1 line change, high diagnostic value, uses existingversion.Get()resource.WithSchemaURL(semconv.SchemaURL)— 1 line change, correctness improvementtracing.Tracer()at construction — minor cleanup, aligns with existing patternresource.WithTelemetrySDK()— low-effort, adds observability metadataNext Steps
internal/tracing/provider.goand span call sitesmcp.tool_callphases as a separate observability improvementWrapHTTPHandler/WithOTELTracingas tech debt cleanupGenerated by Go Fan 🐹 · Round-robin selection after
github.com/tetratelabs/wazeroModule summary saved to: specs/mods/go-opentelemetry-otel.md (pending write access)
Run: §24442419216
Note
🔒 Integrity filter blocked 11 items
The following items were blocked because they don't meet the GitHub integrity level.
get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".get_latest_release: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".list_commits: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".get_file_contents: has lower integrity than agent requires. The agent cannot read data with integrity below "unapproved".To allow these resources, lower
min-integrityin your GitHub frontmatter: