fix(tracing): use noop spans when tracing disabled

[siavashs]

This change ensures that when tracing is disabled, we use noop spans instead of creating actual spans that are then discarded. This avoids allocating memory for spans that will never be used.

Pull Request Checklist

Please check all the applicable boxes.

• Please list all open issue(s) discussed with maintainers related to this change
  • Fixes #
• Is this a new Receiver integration?
  • I have already tried to use the Webhook Receiver Integration and 3rd party integrations before adding this new Receiver Integration
• Is this a bugfix?
  • I have added tests that can reproduce the bug which pass with this bugfix applied
• Is this a new feature?
  • I have added tests that test the new feature's functionality
• Does this change affect performance?
  • I have provided benchmarks comparison that shows performance is improved or is not degraded
    • You can use benchstat to compare benchmarks
  • I have added new benchmarks if required or requested by maintainers
• Is this a breaking change?
  • My changes do not break the existing cluster messages
  • My changes do not break the existing api
• I have added/updated the required documentation
• I have signed-off my commits
• I will follow best practices for contributing to this project

Which user-facing changes does this PR introduce?

NONE

Summary by CodeRabbit

• Refactor
  • Improved tracing infrastructure with conditional tracing support. Tracing is now automatically disabled when no endpoint is configured, optimizing performance in non-tracing scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR refactors OpenTelemetry tracer initialization across multiple Alertmanager modules by replacing direct otel.Tracer() calls with a custom tracing.NewTracer() factory function. A new conditional tracer mechanism toggles tracing on/off via a tracingEnabled flag controlled by configuration, enabling dynamic runtime control over span generation.

Changes

Cohort / File(s)	Summary
Module Tracer Refactoring api/v2/api.go, dispatch/dispatch.go, inhibit/inhibit.go, notify/notify.go, provider/mem/mem.go, silence/silence.go	Replaced direct OpenTelemetry otel.Tracer() initialization with tracing.NewTracer() factory function. Updated imports to include github.com/prometheus/alertmanager/tracing. Existing tracer usage (tracer.Start(...) calls) remains unchanged.
Core Tracing Infrastructure tracing/tracing.go	Introduced NewTracer() public factory function returning a conditional tracer. Implemented conditionalTracer type that respects a global atomic tracingEnabled flag: returns noops when disabled, delegates to OTel tracer when enabled. Updated Manager.ApplyConfig() to toggle tracingEnabled based on endpoint configuration and install appropriate OTel tracer provider.

Sequence Diagram

sequenceDiagram
    participant Config as Configuration
    participant Manager as tracing.Manager
    participant Flag as tracingEnabled Flag
    participant Factory as NewTracer()
    participant Tracer as conditionalTracer
    participant OTel as OTel Provider
    participant Module as Alertmanager Module

    Config->>Manager: ApplyConfig(cfg)
    alt cfg.Endpoint != ""
        Manager->>Flag: Set true
        Manager->>OTel: Install OTel Tracer Provider
    else cfg.Endpoint == ""
        Manager->>Flag: Set false
        Manager->>OTel: Install noop Tracer Provider
    end

    Module->>Factory: NewTracer("module/name")
    Factory->>Tracer: Create conditionalTracer
    Tracer-->>Module: Return tracer instance

    Module->>Tracer: Start(ctx, spanName, ...)
    alt tracingEnabled == true
        Tracer->>OTel: Delegate to OTel tracer
        OTel-->>Tracer: Return real span + context
    else tracingEnabled == false
        Tracer-->>Module: Return noopSpan + context
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Hopping through the code with glee,
Tracers now dance conditionally,
A flag to turn them on and off,
No more unwanted telemetry scoff,
Configuration sets them free!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description provides a clear summary of the change but lacks required details such as linked issues, test coverage information, and benchmark comparison for the stated performance improvement.	Add linked issue(s) using 'Fixes #' format, confirm test coverage for the new tracing behavior, and provide benchmark comparisons demonstrating the performance improvement mentioned in the description.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: replacing real spans with noop spans when tracing is disabled, which matches the core objective of the changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

tracing/tracing.go (1)

103-127: ⚠️ Potential issue | 🟠 Major

Clear the tracing state before rebuilding the provider.

After m.shutdownFunc() returns, the old shutdown function and the previous tracingEnabled value stay live until the new provider is fully installed. If buildTracerProvider fails, conditionalTracer.Start still behaves as if tracing is enabled, and the next reload will try to shut the already-torn-down provider down again.

Proposed fix

 func (m *Manager) ApplyConfig(cfg TracingConfig) error {
 	// Update only if a config change is detected. If TLS configuration is
 	// set, we have to restart the manager to make sure that new TLS
 	// certificates are picked up.
 	var blankTLSConfig commoncfg.TLSConfig
 	if reflect.DeepEqual(m.config, cfg) && (m.config.TLSConfig == nil || *m.config.TLSConfig == blankTLSConfig) {
 		return nil
 	}
 
+	wasEnabled := tracingEnabled.Load()
+	tracingEnabled.Store(false)
 	if m.shutdownFunc != nil {
 		if err := m.shutdownFunc(); err != nil {
+			tracingEnabled.Store(wasEnabled)
 			return fmt.Errorf("failed to shut down the tracer provider: %w", err)
 		}
+		m.shutdownFunc = nil
 	}
 
 	// If no endpoint is set, assume tracing should be disabled.
 	if cfg.Endpoint == "" {
-		tracingEnabled.Store(false)
 		m.config = cfg
-		m.shutdownFunc = nil
 		otel.SetTracerProvider(noop.NewTracerProvider())
 		m.logger.Info("Tracing provider uninstalled.")
 		return nil
 	}

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tracing/tracing.go` around lines 103 - 127, After calling and awaiting
m.shutdownFunc() in Reload (or the function that reinstalls the tracer),
immediately clear the previous tracing state so a failed rebuild won't leave
tracing marked enabled or keep an old shutdown func: set
tracingEnabled.Store(false), set m.shutdownFunc = nil, and install a noop tracer
provider via otel.SetTracerProvider(noop.NewTracerProvider()) (and update
m.config accordingly). Then proceed to call buildTracerProvider(ctx, cfg) and
only on success assign m.shutdownFunc, m.config and tracingEnabled.Store(true).
This ensures conditionalTracer.Start and subsequent reloads see tracing as
disabled while the new provider is being built.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tracing/tracing.go`:
- Around line 43-60: The Start method in conditionalTracer returns the original
ctx when tracing is disabled, which violates trace.Tracer.Start's contract;
update conditionalTracer.Start (the method on type conditionalTracer that checks
tracingEnabled.Load()) to return a context that has noopSpan attached (use the
OpenTelemetry helper that derives a new context with the provided Span) and
return noopSpan as the Span so downstream callers (e.g.,
provider/mem.Alerts.Put) get a context carrying the noop span.

---

Outside diff comments:
In `@tracing/tracing.go`:
- Around line 103-127: After calling and awaiting m.shutdownFunc() in Reload (or
the function that reinstalls the tracer), immediately clear the previous tracing
state so a failed rebuild won't leave tracing marked enabled or keep an old
shutdown func: set tracingEnabled.Store(false), set m.shutdownFunc = nil, and
install a noop tracer provider via
otel.SetTracerProvider(noop.NewTracerProvider()) (and update m.config
accordingly). Then proceed to call buildTracerProvider(ctx, cfg) and only on
success assign m.shutdownFunc, m.config and tracingEnabled.Store(true). This
ensures conditionalTracer.Start and subsequent reloads see tracing as disabled
while the new provider is being built.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f0fcc991-7cf6-41e7-955f-b0f1adc28167

📥 Commits

Reviewing files that changed from the base of the PR and between 8ae3869 and 370c3c5.

📒 Files selected for processing (7)

• api/v2/api.go
• dispatch/dispatch.go
• inhibit/inhibit.go
• notify/notify.go
• provider/mem/mem.go
• silence/silence.go
• tracing/tracing.go