Skip to content

feat(node): Avoid OTEL instrumentation for outgoing requests on Node 22+#17355

Merged
andreiborza merged 21 commits intodevelopfrom
fn/avoid-otel-http-instrumentation
Mar 12, 2026
Merged

feat(node): Avoid OTEL instrumentation for outgoing requests on Node 22+#17355
andreiborza merged 21 commits intodevelopfrom
fn/avoid-otel-http-instrumentation

Conversation

@mydea
Copy link
Member

@mydea mydea commented Aug 8, 2025

Registers diagnostics channels for outgoing requests on Node >= 22 that takes
care of creating spans, rather than relying on OTEL instrumentation.

Closes #18497 (added automatically)

@mydea mydea self-assigned this Aug 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2025

size-limit report 📦

⚠️ Warning: Base artifact is not the latest one, because the latest workflow run is not done yet. This may lead to incorrect results. Try to re-run all tests to get up to date results.

Path Size % Change Change
@sentry/browser 25.64 kB - -
@sentry/browser - with treeshaking flags 24.14 kB - -
@sentry/browser (incl. Tracing) 42.62 kB - -
@sentry/browser (incl. Tracing, Profiling) 47.28 kB - -
@sentry/browser (incl. Tracing, Replay) 81.42 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 71 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 86.12 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 98.37 kB - -
@sentry/browser (incl. Feedback) 42.45 kB - -
@sentry/browser (incl. sendFeedback) 30.31 kB - -
@sentry/browser (incl. FeedbackAsync) 35.36 kB - -
@sentry/browser (incl. Metrics) 26.92 kB - -
@sentry/browser (incl. Logs) 27.07 kB - -
@sentry/browser (incl. Metrics & Logs) 27.74 kB - -
@sentry/react 27.39 kB - -
@sentry/react (incl. Tracing) 44.95 kB - -
@sentry/vue 30.08 kB - -
@sentry/vue (incl. Tracing) 44.48 kB - -
@sentry/svelte 25.66 kB - -
CDN Bundle 28.27 kB - -
CDN Bundle (incl. Tracing) 43.5 kB - -
CDN Bundle (incl. Logs, Metrics) 29.13 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 44.34 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 68.2 kB - -
CDN Bundle (incl. Tracing, Replay) 80.32 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 81.22 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 85.86 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 86.76 kB - -
CDN Bundle - uncompressed 82.56 kB - -
CDN Bundle (incl. Tracing) - uncompressed 128.5 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 85.43 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 131.37 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 209.06 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 245.35 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 248.21 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 258.26 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 261.11 kB - -
@sentry/nextjs (client) 47.37 kB - -
@sentry/sveltekit (client) 43.07 kB - -
@sentry/node-core 56.34 kB +7.8% +4.07 kB 🔺
@sentry/node 173.18 kB -1.12% -1.95 kB 🔽
@sentry/node - without tracing 96.34 kB -1.12% -1.09 kB 🔽
@sentry/aws-serverless 113.33 kB +0.1% +102 B 🔺

View base workflow run

@mydea mydea force-pushed the fn/avoid-otel-http-instrumentation branch from 67942ee to b8b33f4 Compare August 8, 2025 09:29
@andreiborza andreiborza force-pushed the fn/avoid-otel-http-instrumentation branch 2 times, most recently from a66e597 to 002fb1e Compare December 14, 2025 17:10
@github-actions
Copy link
Contributor

github-actions bot commented Dec 14, 2025

node-overhead report 🧳

Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.

Scenario Requests/s % of Baseline Prev. Requests/s Change %
GET Baseline 9,065 - 9,465 -4%
GET With Sentry 1,613 18% 1,666 -3%
GET With Sentry (error only) 6,073 67% 6,276 -3%
POST Baseline 1,183 - 1,213 -2%
POST With Sentry 590 50% 587 +1%
POST With Sentry (error only) 1,038 88% 1,065 -3%
MYSQL Baseline 3,201 - 3,254 -2%
MYSQL With Sentry 431 13% 442 -2%
MYSQL With Sentry (error only) 2,616 82% 2,658 -2%

View base workflow run

@andreiborza andreiborza force-pushed the fn/avoid-otel-http-instrumentation branch 2 times, most recently from dcb74c7 to 4023787 Compare December 14, 2025 18:48
mydea and others added 2 commits December 14, 2025 19:55
Registers diagnostics channels for outgoing requests on Node >= 22 that takes
care of creating spans, rather than relying on OTEL instrumentation.
@andreiborza andreiborza force-pushed the fn/avoid-otel-http-instrumentation branch from 4023787 to 1dad574 Compare December 14, 2025 18:57
Comment on lines +67 to +68
* This is a feature flag that should be enabled by SDKs when the runtime supports it (Node 22+).
* Individual users should not need to configure this directly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment sound very directed to us as SDK maintainers. As this comment is public-facing I would write that a bit differently as it can be confusing how to act on this as a user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a suggestion? The comment already calls out individual users should not set this.

*
* @default `true`
*/
spans?: boolean;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: I'm wondering why we need this second option...would someone ever want to set this to false?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users with custom OTel setups that add @opentelemetry/instrumentation-http will want to set this, see: https://docs.sentry.io/platforms/javascript/guides/node/opentelemetry/custom-setup/#custom-http-instrumentation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, thanks!

// In this case, `http.client.response.finish` is not triggered
subscribe('http.client.request.error', onHttpClientRequestError);

if (this.getConfig().createSpansForOutgoingRequests) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the span option, we should also check for this.getConfig().spans (if I understood that correctly).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is checked in the handler itself but I agree, we can check this earlier. Currently it can be misleading to set spans: false and then still get the Handling started outgoing request log.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

Codecov Results 📊

27 passed | Total: 27 | Pass Rate: 100% | Execution Time: 11.46s

All tests are passing successfully.


Generated by Codecov Action

@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

Codecov Results 📊

23 passed | ⏭️ 7 skipped | Total: 30 | Pass Rate: 76.67% | Execution Time: 10.18s

All tests are passing successfully.


Generated by Codecov Action

@andreiborza andreiborza requested a review from s1gr1d February 18, 2026 14:52
breadcrumbs: options.breadcrumbs,
propagateTraceInOutgoingRequests: !useOtelHttpInstrumentation,
propagateTraceInOutgoingRequests: FULLY_SUPPORTS_HTTP_DIAGNOSTICS_CHANNEL || !useOtelHttpInstrumentation,
createSpansForOutgoingRequests: FULLY_SUPPORTS_HTTP_DIAGNOSTICS_CHANNEL,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing integration test for new diagnostics channel spans

Low Severity

This is a feat PR that introduces diagnostics-channel-based outgoing request span creation, but the diff does not include a new integration or E2E test that specifically exercises this code path. The existing http-basic integration test may cover it implicitly on Node 22.12+ CI runners, but there's no test that explicitly verifies the createSpansForOutgoingRequests / diagnostics channel flow or differentiates it from the OTEL-based flow. Adding a targeted integration test would guard against regressions in this specific feature.

Fix in Cursor Fix in Web

Triggered by project rule: PR Review Guidelines for Cursor Bot

The assertion that both outgoing requests share the same trace ID only
holds on Node 22+ (diagnostics channel path). On Node <22, OTEL creates
separate spans per request, each with their own trace ID.
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Missing integration or E2E test for new feature
    • Added two integration tests that verify span creation, trace propagation, and the interaction between createSpansForOutgoingRequests and the spans option for outgoing HTTP requests using diagnostics channels on Node 22.12+.

Create PR

Or push these changes by commenting:

@cursor push b5fc75535a
Preview (b5fc75535a)
diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/instrument.mjs b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/instrument.mjs
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/instrument.mjs
@@ -1,0 +1,10 @@
+import * as Sentry from '@sentry/node';
+import { loggingTransport } from '@sentry-internal/node-integration-tests';
+
+Sentry.init({
+  dsn: 'https://public@dsn.ingest.sentry.io/1337',
+  release: '1.0',
+  tracesSampleRate: 1.0,
+  integrations: [Sentry.httpIntegration({ spans: false })],
+  transport: loggingTransport,
+});

diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/scenario.mjs b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/scenario.mjs
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/scenario.mjs
@@ -1,0 +1,23 @@
+import * as Sentry from '@sentry/node';
+import * as http from 'http';
+
+Sentry.startSpan({ name: 'test_span' }, async () => {
+  await makeHttpRequest(`${process.env.SERVER_URL}/api/test`);
+});
+
+function makeHttpRequest(url) {
+  return new Promise((resolve, reject) => {
+    http
+      .request(url, httpRes => {
+        httpRes.on('data', () => {
+          // we don't care about data
+        });
+        httpRes.on('end', () => {
+          resolve();
+        });
+        httpRes.on('error', reject);
+      })
+      .on('error', reject)
+      .end();
+  });
+}

diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/test.ts b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/test.ts
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans-disabled/test.ts
@@ -1,0 +1,41 @@
+import { createTestServer } from '@sentry-internal/test-utils';
+import { parseSemver } from '@sentry/core';
+import { describe, expect } from 'vitest';
+import { createEsmAndCjsTests } from '../../../../utils/runner';
+
+const NODE_VERSION = parseSemver(process.versions.node);
+
+const supportsHttpDiagnosticsChannel =
+  (NODE_VERSION.major === 22 && NODE_VERSION.minor >= 12) ||
+  (NODE_VERSION.major === 23 && NODE_VERSION.minor >= 2) ||
+  NODE_VERSION.major >= 24;
+
+const testIfSupported = supportsHttpDiagnosticsChannel ? describe : describe.skip;
+
+testIfSupported('outgoing http with diagnostics channel spans disabled', () => {
+  createEsmAndCjsTests(__dirname, 'scenario.mjs', 'instrument.mjs', (createRunner, test) => {
+    test('does not create spans when spans option is false', async () => {
+      expect.assertions(3);
+
+      const [SERVER_URL, closeTestServer] = await createTestServer()
+        .get('/api/test', headers => {
+          expect(headers['sentry-trace']).toEqual(expect.any(String));
+        })
+        .start();
+
+      await createRunner()
+        .withEnv({ SERVER_URL })
+        .expect({
+          transaction: event => {
+            expect(event.transaction).toBe('test_span');
+
+            const httpClientSpans = event.spans?.filter(span => span.op === 'http.client');
+            expect(httpClientSpans).toHaveLength(0);
+          },
+        })
+        .start()
+        .completed();
+      closeTestServer();
+    });
+  });
+});

diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/instrument.mjs b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/instrument.mjs
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/instrument.mjs
@@ -1,0 +1,10 @@
+import * as Sentry from '@sentry/node';
+import { loggingTransport } from '@sentry-internal/node-integration-tests';
+
+Sentry.init({
+  dsn: 'https://public@dsn.ingest.sentry.io/1337',
+  release: '1.0',
+  tracesSampleRate: 1.0,
+  integrations: [],
+  transport: loggingTransport,
+});

diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/scenario.mjs b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/scenario.mjs
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/scenario.mjs
@@ -1,0 +1,23 @@
+import * as Sentry from '@sentry/node';
+import * as http from 'http';
+
+Sentry.startSpan({ name: 'test_span' }, async () => {
+  await makeHttpRequest(`${process.env.SERVER_URL}/api/test`);
+});
+
+function makeHttpRequest(url) {
+  return new Promise((resolve, reject) => {
+    http
+      .request(url, httpRes => {
+        httpRes.on('data', () => {
+          // we don't care about data
+        });
+        httpRes.on('end', () => {
+          resolve();
+        });
+        httpRes.on('error', reject);
+      })
+      .on('error', reject)
+      .end();
+  });
+}

diff --git a/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/test.ts b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/test.ts
new file mode 100644
--- /dev/null
+++ b/dev-packages/node-integration-tests/suites/tracing/requests/http-diagnostics-channel-spans/test.ts
@@ -1,0 +1,65 @@
+import { createTestServer } from '@sentry-internal/test-utils';
+import { parseSemver } from '@sentry/core';
+import { describe, expect } from 'vitest';
+import { createEsmAndCjsTests } from '../../../../utils/runner';
+
+const NODE_VERSION = parseSemver(process.versions.node);
+
+// The `http.client.request.created` diagnostics channel was added in Node 22.12.0 and 23.2.0
+const supportsHttpDiagnosticsChannel =
+  (NODE_VERSION.major === 22 && NODE_VERSION.minor >= 12) ||
+  (NODE_VERSION.major === 23 && NODE_VERSION.minor >= 2) ||
+  NODE_VERSION.major >= 24;
+
+const testIfSupported = supportsHttpDiagnosticsChannel ? describe : describe.skip;
+
+testIfSupported('outgoing http with diagnostics channel spans', () => {
+  createEsmAndCjsTests(__dirname, 'scenario.mjs', 'instrument.mjs', (createRunner, test) => {
+    test('creates spans for outgoing requests and propagates trace context within span', async () => {
+      expect.assertions(8);
+
+      let outgoingSpanId: string | undefined;
+
+      const [SERVER_URL, closeTestServer] = await createTestServer()
+        .get('/api/test', headers => {
+          // Verify trace propagation headers are present
+          expect(headers['baggage']).toEqual(expect.any(String));
+          expect(headers['sentry-trace']).toEqual(expect.stringMatching(/^([a-f\d]{32})-([a-f\d]{16})-1$/));
+
+          // Extract the span ID from the sentry-trace header
+          const sentryTrace = headers['sentry-trace'] as string;
+          outgoingSpanId = sentryTrace.split('-')[1];
+
+          // Verify we're not propagating all-zero trace IDs
+          expect(headers['sentry-trace']).not.toEqual('00000000000000000000000000000000-0000000000000000-1');
+        })
+        .start();
+
+      await createRunner()
+        .withEnv({ SERVER_URL })
+        .expect({
+          transaction: event => {
+            expect(event.transaction).toBe('test_span');
+
+            // Verify that an http.client span was created
+            const httpClientSpans = event.spans?.filter(span => span.op === 'http.client');
+            expect(httpClientSpans).toHaveLength(1);
+
+            const httpSpan = httpClientSpans![0];
+            expect(httpSpan?.description).toMatch(/^GET /);
+
+            // Verify the propagated span ID matches the created span
+            if (outgoingSpanId) {
+              expect(httpSpan?.span_id).toBe(outgoingSpanId);
+            }
+
+            // Verify span attributes include sentry.origin
+            expect(httpSpan?.data?.['sentry.origin']).toBe('auto.http.otel.http');
+          },
+        })
+        .start()
+        .completed();
+      closeTestServer();
+    });
+  });
+});

This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

});

return span;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing integration or E2E test for new feature

Low Severity

This is a feat PR introducing a significant new capability (outgoing request span creation via diagnostics channels). The diff only contains unit tests for _shouldUseOtelHttpInstrumentation and mergeBaggageHeaders, but no integration or E2E test verifying the new span creation flow, trace propagation within span context, or the interaction between createSpansForOutgoingRequests/disableOutgoingRequestInstrumentation. Adding at least one integration test covering end-to-end span creation and propagation for outgoing HTTP requests on Node 22+ would help catch regressions.

Additional Locations (1)
Fix in Cursor Fix in Web

Triggered by project rule: PR Review Guidelines for Cursor Bot

@andreiborza andreiborza merged commit ec13b40 into develop Mar 12, 2026
229 checks passed
@andreiborza andreiborza deleted the fn/avoid-otel-http-instrumentation branch March 12, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(node): Avoid OTEL instrumentation for outgoing requests on Node 22+

4 participants