Refactor e2e tests to no longer use "trigger" endpoint#958
Merged
TooTallNate merged 17 commits intomainfrom Feb 7, 2026
Merged
Refactor e2e tests to no longer use "trigger" endpoint#958TooTallNate merged 17 commits intomainfrom
TooTallNate merged 17 commits intomainfrom
Conversation
🦋 Changeset detectedLatest commit: 92dcd4e The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (157 failed)mongodb (39 failed):
redis (39 failed):
starter (40 failed):
turso (39 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)No data available Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
Member
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the core end-to-end tests to stop relying on the workbench /api/trigger endpoint by starting runs directly via the runtime using workflow IDs from the published manifest, and removes the now-unused trigger endpoints across the various workbench framework implementations.
Changes:
- Remove
/api/triggerhandlers from multiple workbench apps (Vite, SvelteKit, Nuxt/Nitro v2/v3, Next.js app router, Astro, “example”, plus framework servers like Express/Fastify/Hono/Nest). - Update
packages/core/e2e/e2e.test.tsto fetch/.well-known/workflow/v1/manifest.jsonand usestart({ workflowId })+run.returnValue/run.getReadable()instead of HTTP polling. - Ensure CI/build environment exposes the manifest (
WORKFLOW_PUBLIC_MANIFEST=1) and update the dev test matrix to reference/api/chatinstead of/api/trigger.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| workbench/vite/routes/api/trigger.post.ts | Removes Vite trigger POST endpoint implementation. |
| workbench/vite/routes/api/trigger.get.ts | Removes Vite trigger GET endpoint implementation. |
| workbench/sveltekit/src/routes/api/trigger/+server.ts | Removes SvelteKit trigger endpoint implementation. |
| workbench/nuxt/server/api/trigger.post.ts | Removes Nuxt trigger POST endpoint implementation. |
| workbench/nuxt/server/api/trigger.get.ts | Removes Nuxt trigger GET endpoint implementation. |
| workbench/nitro-v3/routes/api/trigger.post.ts | Removes Nitro v3 trigger POST endpoint implementation. |
| workbench/nitro-v3/routes/api/trigger.get.ts | Removes Nitro v3 trigger GET endpoint implementation. |
| workbench/nitro-v2/server/api/trigger.post.ts | Removes Nitro v2 trigger POST endpoint implementation. |
| workbench/nitro-v2/server/api/trigger.get.ts | Removes Nitro v2 trigger GET endpoint implementation. |
| workbench/nextjs-webpack/app/api/trigger/route.ts | Removes Next.js (webpack) app-router trigger endpoint. |
| workbench/nextjs-turbopack/app/api/trigger/route.ts | Removes Next.js (turbopack) app-router trigger endpoint. |
| workbench/nest/src/app.controller.ts | Removes Nest trigger routes and related imports. |
| workbench/hono/src/index.ts | Removes Hono trigger routes; keeps _workflows in graph via side-effect import. |
| workbench/fastify/src/index.ts | Removes Fastify trigger routes; keeps _workflows in graph via side-effect import. |
| workbench/express/src/index.ts | Removes Express trigger routes; keeps _workflows in graph via side-effect import. |
| workbench/example/api/trigger.ts | Removes example trigger endpoint implementation. |
| workbench/astro/src/pages/api/trigger.ts | Removes Astro trigger endpoint implementation. |
| turbo.json | Adds WORKFLOW_PUBLIC_MANIFEST as a build task env input for Turbo caching. |
| scripts/create-test-matrix.mjs | Updates Vite dev-test API file path from trigger to chat. |
| packages/core/e2e/e2e.test.ts | Switches e2e workflow startup to manifest-based start({workflowId}) and updates streaming/error assertions accordingly. |
| .github/workflows/tests.yml | Sets WORKFLOW_PUBLIC_MANIFEST=1 for relevant CI jobs so the manifest is accessible. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
VaguelySerious
approved these changes
Feb 6, 2026
Member
Author
Merge activity |
…rtions, fix dev test config
…ng IDs manually Fetch the manifest from GET /manifest.json on the deployment to discover the correct workflow IDs. This handles symlink resolution differences between frameworks since the manifest contains the exact IDs produced by the SWC transform during the actual build.
… is served before catch-all route
- Update manifest fetch URL to /.well-known/workflow/v1/manifest.json - Remove publicAssets config from hono/express/fastify nitro configs (no longer needed since Nitro virtual handler serves the manifest) - Add WORKFLOW_PUBLIC_MANIFEST=1 to all CI e2e test jobs - Note: Vercel prod tests also need this env var set in each Vercel project's environment settings for the deployed builds
The data dir discovery approach failed for Nitro-based apps because the data directory doesn't exist until the app processes its first queue message. By that time, the test runner had already created its own .workflow-data at the monorepo root, causing the test runner and app to use different data directories. Fix: explicitly compute the data dir path based on the app name and framework convention (.next/workflow-data for Next.js, .workflow-data for everything else).
… HMR The Nitro dev server uses the dependency graph to determine which file changes trigger a dev:reload. Removing the _workflows.js import from src/index.ts broke HMR rebuilds for workflow file changes because Nitro no longer knew that src/index.ts depended on the workflow files. Re-add the import as a side-effect import (no named exports) to maintain the dependency chain for Nitro's file watcher.
…run.returnValue directly
- Replace startWorkflow() with direct start(await e2e('fn'), args) calls
- Replace run.returnValue access via getWorkflowReturnValue() with direct run.returnValue
- Add getWorkflowMetadata() helper that returns { workflowId } for start()
- Add e2e() shorthand for workflows/99_e2e.ts lookups
- Pages Router tests use startWorkflowViaHttp() explicitly
- Remove debug logging from all builder packages
…trigger The benchmarks were using the deleted /api/trigger endpoint, causing 'No benchmark data found' in CI. Apply the same refactoring as the e2e tests: use start() directly with manifest-based workflow ID lookup. Also adds WORKFLOW_PUBLIC_MANIFEST=1 to all benchmark CI jobs.
…hmarks The Vercel benchmark job was missing VERCEL_DEPLOYMENT_ID, causing the World to default to local instead of Vercel. Also include the generated packages/core/src/version.ts in build artifacts. Added fail-fast check for missing VERCEL_DEPLOYMENT_ID when WORKFLOW_VERCEL_ENV is set, and a 2-minute timeout wrapper around run.returnValue to prevent benchmarks from hanging indefinitely if the workflow never completes.
Add 30s AbortSignal.timeout to fetchManifest() to prevent hanging on unresponsive Vercel deployments. Log response URL to detect redirects.
…Sync The Nitro virtual handler used readFileSync with an absolute path baked in at build time. In prod mode on a different machine (CI), this path doesn't exist, causing 404s. Fix: in dev mode, keep the virtual handler (same machine, path is valid). In prod mode, copy the manifest to Nitro's public output directory (.output/public/) during the 'compiled' hook, so Nitro serves it as a static asset.
Contributor
|
Deployment failed with the following error: |
…tual module Virtual modules in Nitro are resolved once during setup and can't be updated later. For prod builds, write a physical manifest-handler.mjs file with inlined manifest content during build:before (after the builder generates the manifest). Nitro's rollup then bundles this file into the compiled output. Dev mode continues to use a virtual handler with readFileSync since the absolute path is valid on the build machine.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactors the E2E tests to call
start()fromworkflow/apidirectly instead of going through the/api/triggerHTTP endpoint in each workbench app. This removes a layer of indirection — the tests now use the same API that users would use to start workflows programmatically.Before
triggerWorkflow()sent an HTTP POST to/api/triggeron the workbench appstart(), and returned the run IDgetWorkflowReturnValue()polledGET /api/trigger?runId=...until the workflow completedAfter
e2e()/getWorkflowMetadata()fetches the manifest from/.well-known/workflow/v1/manifest.jsonto look up the correctworkflowIdstart()is called directly from the test process via the configured Worldrun.returnValuepolls for completion via the World (no HTTP polling endpoint needed)Changes
packages/core/e2e/e2e.test.tstriggerWorkflow()andgetWorkflowReturnValue()helpersfetchManifest()to fetch and cache the workflow manifest from the deploymentgetWorkflowMetadata(file, fn)to look up{ workflowId }from the manifeste2e(fn)shorthand for the common case ofworkflows/99_e2e.tsstart()andrun.returnValuedirectly.catch()to inspectWorkflowRunFailedErrorrun.getReadable()directly (skipped on local world where cross-process streaming isn't supported)beforeAllconfigures the local World with the correct data directory and base URLstartWorkflowViaHttp()to specifically validate the HTTP trigger pathWorkbench apps (hono, express, fastify, nest)
/api/triggerroute handlers/api/hook,/api/test-direct-step-call,/api/test-health-checkendpoints_workflows.jsside-effect import for hono/express/fastify to maintain Nitro's HMR dependency graphDeleted trigger-only route files from: nextjs-turbopack, nextjs-webpack, vite, sveltekit, astro, nuxt, nitro-v2, nitro-v3, example
.github/workflows/tests.ymlWORKFLOW_PUBLIC_MANIFEST: '1'to all E2E test jobsDependencies
Stacked on #963 which adds
WORKFLOW_PUBLIC_MANIFESTsupport to all framework builders.