Skip to content

feat: use @adobe/fetch Response to prevent helix-universal crash on Vault failure#1554

Open
abhishekgarg18 wants to merge 2 commits intomainfrom
fix/vault-secrets-response-headers-raw
Open

feat: use @adobe/fetch Response to prevent helix-universal crash on Vault failure#1554
abhishekgarg18 wants to merge 2 commits intomainfrom
fix/vault-secrets-response-headers-raw

Conversation

@abhishekgarg18
Copy link
Copy Markdown

@abhishekgarg18 abhishekgarg18 commented Apr 23, 2026

Problem

When Vault egress is blocked (e.g. NAT/WAF returning 403) — as happened on dev 2026-04-22 ~22:10 UTCevery API endpoint returns HTTP 500 with:

x-error: response.headers.raw is not a function

Reported in Slack `#spacecat-ops` across `spacecat-api-service`, `spacecat-audit-worker`, and all other services using this package.


Root Cause Chain

Vault 403
  └─ vaultSecrets catch block
       └─ returns new Response('', { status: 502 })   ← native Node.js Response
            └─ native Headers has NO .raw() method
                 └─ compressResponse returns it unchanged (empty body, not compressible)
                      └─ helix-universal aws-adapter.js calls response.headers.raw()
                           └─ 💥 TypeError: response.headers.raw is not a function
                                └─ HTTP 500 on ALL endpoints

The `vault-secrets-wrapper.js` error handler was using the native Node.js `new Response()` instead of the `@adobe/fetch` version. Native Web API `Headers` lack the `.raw()` method that `helix-universal`'s `aws-adapter.js` calls unconditionally on every response before sending it back through API Gateway.

Every other middleware in this repo already does this correctly:

File Imports Response from
`auth-wrapper.js` ✅ `@adobe/fetch`
`s2s-wrapper.js` ✅ `@adobe/fetch`
`sqs.js` ✅ `@adobe/fetch`
`vault-secrets-wrapper.js` ❌ native ← this PR

Fix

One-line import addition — `@adobe/fetch` is already a declared dependency of this package:

+import { Response } from '@adobe/fetch';
+
 import VaultClient from './vault-client.js';
 import { loadBootstrapConfig } from './bootstrap.js';

`@adobe/fetch`'s `Headers` class implements `.raw()`, restoring full compatibility with `helix-universal`'s `aws-adapter.js`.


Tests Added

5 new regression tests in `vault-secrets-wrapper.test.js`:

Test What it verifies
`502 response uses @adobe/fetch Headers — .raw() is available` Regression guard — asserts `.raw()` is callable and returns the correct header map
`502 from Vault 403 also has @adobe/fetch Headers with .raw()` Exact production failure path: Vault NAT/WAF block → AppRole login 403
`returns 502 when Vault AppRole login returns 403` Auth failure scenario coverage
`returns 502 when Vault secret read fails` Secret read failure coverage
`logs the error message when vault fails` Confirms error is logged with correct message

Impact After Release

Once a new patch version of `@adobe/spacecat-shared-vault-secrets` is published, version bumps are needed in 13 services — Renovate will auto-raise those PRs:

Priority Service
🔴 High `spacecat-api-service`, `spacecat-audit-worker`
🟡 Others `spacecat-auth-service`, `mysticat-projector-service`, `spacecat-content-processor`, `spacecat-content-scraper`, `spacecat-fulfillment-worker`, `spacecat-import-job-manager`, `spacecat-import-worker`, `spacecat-jobs-dispatcher`, `spacecat-reporting-worker`, `spacecat-scrape-job-manager`

Related

  • Slack `#spacecat-ops` — "Vault authentication failed: 403" + `response.headers.raw is not a function` (2026-04-22)

@abhishekgarg18 abhishekgarg18 force-pushed the fix/vault-secrets-response-headers-raw branch from 879b065 to 64293f6 Compare April 23, 2026 10:30
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 23, 2026

This PR will trigger a minor release when merged.

@abhishekgarg18 abhishekgarg18 changed the title fix: use @adobe/fetch Response to restore .raw() compa… fix(vault-secrets): use @adobe/fetch Response to prevent helix-universal crash on Vault failure Apr 23, 2026
@abhishekgarg18 abhishekgarg18 changed the title fix(vault-secrets): use @adobe/fetch Response to prevent helix-universal crash on Vault failure feat: use @adobe/fetch Response to prevent helix-universal crash on Vault failure Apr 23, 2026
@abhishekgarg18 abhishekgarg18 force-pushed the fix/vault-secrets-response-headers-raw branch from 64293f6 to 5d15941 Compare April 23, 2026 10:36
The error response returned on Vault failure used the native Node.js
`new Response()` which produces native Web API Headers. Those headers
lack the `.raw()` method that `helix-universal`'s `aws-adapter.js`
calls unconditionally on every response.

When the dev Vault egress was blocked (403), every Lambda invocation
hit the vault error handler, returned a native Response, and crashed
in the adapter with `response.headers.raw is not a function` —
surfaced as HTTP 500 on all API endpoints.

Fix: import Response from `@adobe/fetch` (already a declared
dependency). Its Headers class implements `.raw()`, making it
compatible with the helix-universal adapter.

Added regression tests that assert `.raw()` is callable on the 502
error response, covering bootstrap failure, Vault 403, and secret
read failure scenarios.

Made-with: Cursor
@abhishekgarg18 abhishekgarg18 force-pushed the fix/vault-secrets-response-headers-raw branch from 5d15941 to ea9376f Compare April 23, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant