Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ jobs:
run: pnpm build:test

- name: ♿ Accessibility audit (Lighthouse - ${{ matrix.mode }} mode)
run: ./scripts/lighthouse-a11y.sh
run: pnpm test:a11y:prebuilt
env:
LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
LIGHTHOUSE_COLOR_MODE: ${{ matrix.mode }}
Expand Down
46 changes: 42 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ This focus helps guide our project decisions as a community and what we choose t
- [Testing](#testing)
- [Unit tests](#unit-tests)
- [Component accessibility tests](#component-accessibility-tests)
- [Lighthouse accessibility tests](#lighthouse-accessibility-tests)
- [End to end tests](#end-to-end-tests)
- [Test fixtures (mocking external APIs)](#test-fixtures-mocking-external-apis)
- [Submitting changes](#submitting-changes)
Expand Down Expand Up @@ -111,6 +112,7 @@ pnpm test # Run all Vitest tests
pnpm test:unit # Unit tests only
pnpm test:nuxt # Nuxt component tests
pnpm test:browser # Playwright E2E tests
pnpm test:a11y # Lighthouse accessibility audits
```

### Project structure
Expand Down Expand Up @@ -598,6 +600,40 @@ A coverage test in `test/unit/a11y-component-coverage.spec.ts` ensures all compo
> [!IMPORTANT]
> Just because axe-core doesn't find any obvious issues, it does not mean a component is accessible. Please do additional checks and use best practices.

### Lighthouse accessibility tests

In addition to component-level axe audits, the project runs full-page accessibility audits using [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci). These test the rendered pages in both light and dark mode against Lighthouse's accessibility category, requiring a perfect score.

#### How it works

1. The project is built in test mode (`pnpm build:test`), which activates server-side fixture mocking
2. Lighthouse CI starts a preview server and audits three URLs: `/`, `/search?q=nuxt`, and `/package/nuxt`
3. A Puppeteer setup script (`lighthouse-setup.cjs`) runs before each audit to set the color mode and intercept client-side API requests using the same fixtures as the E2E tests

#### Running locally

```bash
# Build + run both light and dark audits
pnpm test:a11y

# Or against an existing test build
pnpm test:a11y:prebuilt

# Or run a single color mode manually
pnpm build:test
LIGHTHOUSE_COLOR_MODE=dark ./scripts/lighthouse-a11y.sh
```

This requires Chrome or Chromium to be installed. The script will auto-detect common installation paths. Results are printed to the terminal and saved in `.lighthouseci/`.

#### Configuration

| File | Purpose |
| ---------------------------- | --------------------------------------------------------- |
| `.lighthouserc.cjs` | Lighthouse CI config (URLs, assertions, Chrome path) |
| `lighthouse-setup.cjs` | Puppeteer script for color mode + client-side API mocking |
| `scripts/lighthouse-a11y.sh` | Shell wrapper that runs the audit for a given color mode |

### End to end tests

Write end-to-end tests using Playwright:
Expand All @@ -619,10 +655,12 @@ E2E tests use a fixture system to mock external API requests, ensuring tests are
- Serves pre-recorded fixture data from `test/fixtures/`
- Enabled via `NUXT_TEST_FIXTURES=true` or Nuxt test mode

**Client-side mocking** (`test/e2e/test-utils.ts`):
**Client-side mocking** (`test/fixtures/mock-routes.cjs`):

- Uses Playwright's route interception to mock browser requests
- All test files import from `./test-utils` instead of `@nuxt/test-utils/playwright`
- Shared URL matching and response generation logic used by both Playwright E2E tests and Lighthouse CI
- Playwright tests (`test/e2e/test-utils.ts`) use this via `page.route()` interception
- Lighthouse tests (`lighthouse-setup.cjs`) use this via Puppeteer request interception
- All E2E test files import from `./test-utils` instead of `@nuxt/test-utils/playwright`
- Throws a clear error if an unmocked external request is detected

#### Fixture files
Expand Down Expand Up @@ -670,7 +708,7 @@ URL: https://registry.npmjs.org/some-package
You need to either:

1. Add a fixture file for that package/endpoint
2. Update the mock handlers in `test/e2e/test-utils.ts` (client) or `modules/runtime/server/cache.ts` (server)
2. Update the mock handlers in `test/fixtures/mock-routes.cjs` (client) or `modules/runtime/server/cache.ts` (server)

## Submitting changes

Expand Down
2 changes: 1 addition & 1 deletion knip.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ const config: KnipConfig = {
'@vercel/kv',
'@voidzero-dev/vite-plus-core',
'vite-plus!',
'h3',
'puppeteer',
/** Needs to be explicitly installed, even though it is not imported, to avoid type errors. */
'unplugin-vue-router',
'vite-plugin-pwa',
Expand Down
69 changes: 68 additions & 1 deletion lighthouse-setup.cjs
Original file line number Diff line number Diff line change
@@ -1,13 +1,28 @@
/**
* Lighthouse CI puppeteer setup script.
* Sets the color mode (light/dark) before running accessibility audits.
*
* Sets the color mode (light/dark) before running accessibility audits
* and intercepts client-side API requests using the same fixture data
* as the Playwright E2E tests.
*
* The color mode is determined by the LIGHTHOUSE_COLOR_MODE environment variable.
* If not set, defaults to 'dark'.
*
* Request interception uses CDP (Chrome DevTools Protocol) Fetch domain
* at the browser level, which avoids conflicts with Lighthouse's own
* Puppeteer-level request interception.
*/

const mockRoutes = require('./test/fixtures/mock-routes.cjs')

module.exports = async function setup(browser, { url }) {
const colorMode = process.env.LIGHTHOUSE_COLOR_MODE || 'dark'

// Set up browser-level request interception via CDP Fetch domain.
// This operates below Puppeteer's request interception layer so it
// doesn't conflict with Lighthouse's own setRequestInterception usage.
await setupCdpRequestInterception(browser)

const page = await browser.newPage()

// Set localStorage before navigating so @nuxtjs/color-mode picks it up
Expand All @@ -21,3 +36,55 @@ module.exports = async function setup(browser, { url }) {
// Close the page - Lighthouse will open its own with localStorage already set
await page.close()
}

/**
* Set up request interception using CDP's Fetch domain on the browser's
* default context. This intercepts requests at a lower level than Puppeteer's
* page.setRequestInterception(), avoiding "Request is already handled!" errors
* when Lighthouse sets up its own interception.
*
* @param {import('puppeteer').Browser} browser
*/
async function setupCdpRequestInterception(browser) {
// Build URL pattern list for CDP Fetch.enable from our route definitions
const cdpPatterns = mockRoutes.routes.map(route => ({
urlPattern: route.pattern.replace('/**', '/*'),
requestStage: 'Request',
}))
Comment on lines +50 to +53
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

find . -name "mock-routes*" -o -name "lighthouse-setup*" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 115


🏁 Script executed:

git ls-files | grep -E "(mock-routes|lighthouse-setup)" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 111


🏁 Script executed:

git ls-files | grep -E "\.cjs$" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 129


🏁 Script executed:

cat -n lighthouse-setup.cjs | sed -n '40,65p'

Repository: npmx-dev/npmx.dev

Length of output: 1238


🏁 Script executed:

cat -n test/fixtures/mock-routes.cjs

Repository: npmx-dev/npmx.dev

Length of output: 15791


🏁 Script executed:

grep -r "downloads/point" test/ --include="*.ts" --include="*.js" --include="*.cjs"

Repository: npmx-dev/npmx.dev

Length of output: 471


🏁 Script executed:

grep -r "v1/packages" test/ --include="*.ts" --include="*.js" --include="*.cjs"

Repository: npmx-dev/npmx.dev

Length of output: 43


🏁 Script executed:

grep -r "lighthouse-setup" test/ .github/ --include="*.ts" --include="*.js" --include="*.cjs" --include="*.json" --include="*.yaml" --include="*.yml"

Repository: npmx-dev/npmx.dev

Length of output: 148


🏁 Script executed:

cat -n test/nuxt/composables/use-npm-registry.spec.ts | sed -n '1,50p'

Repository: npmx-dev/npmx.dev

Length of output: 1954


🏁 Script executed:

rg "https://api.npmjs.org/downloads" test/ -A 2 -B 2

Repository: npmx-dev/npmx.dev

Length of output: 1423


🏁 Script executed:

cat -n lighthouse-setup.cjs | sed -n '1,100p'

Repository: npmx-dev/npmx.dev

Length of output: 3971


🏁 Script executed:

rg "Fetch\.enable" lighthouse-setup.cjs -A 5 -B 5

Repository: npmx-dev/npmx.dev

Length of output: 734


The pattern conversion from /** to /* will break CDP interception for multi-segment paths.

CDP Fetch patterns use glob syntax where * matches a single path segment and /** matches recursively. Converting patterns like https://api.npmjs.org/** to https://api.npmjs.org/* prevents CDP from pausing requests to endpoints with multiple path segments (e.g., https://api.npmjs.org/downloads/point/last-week/vue). Requests that don't match CDP's patterns won't pause, so mockRoutes.matchRoute() is never called and the mock response is never served.

Keep the patterns unchanged or use /** syntax if that's supported by CDP's urlPattern implementation.


// Listen for new targets so we can attach CDP interception to each page
browser.on('targetcreated', async target => {
if (target.type() !== 'page') return

try {
const cdp = await target.createCDPSession()

cdp.on('Fetch.requestPaused', async event => {
const requestUrl = event.request.url
const result = mockRoutes.matchRoute(requestUrl)

if (result) {
const body = Buffer.from(result.response.body).toString('base64')
await cdp.send('Fetch.fulfillRequest', {
requestId: event.requestId,
responseCode: result.response.status,
responseHeaders: [
{ name: 'Content-Type', value: result.response.contentType },
{ name: 'Access-Control-Allow-Origin', value: '*' },
],
body,
})
} else {
await cdp.send('Fetch.continueRequest', {
requestId: event.requestId,
})
}
})
Comment on lines +62 to +82
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add error handling within the Fetch.requestPaused handler.

The cdp.send() calls inside the event handler are not wrapped in a try/catch. If the target closes whilst a request is paused (before fulfillRequest or continueRequest completes), this will throw an unhandled promise rejection.

🛡️ Proposed fix to add error handling
       cdp.on('Fetch.requestPaused', async event => {
+        try {
           const requestUrl = event.request.url
           const result = mockRoutes.matchRoute(requestUrl)
 
           if (result) {
             const body = Buffer.from(result.response.body).toString('base64')
             await cdp.send('Fetch.fulfillRequest', {
               requestId: event.requestId,
               responseCode: result.response.status,
               responseHeaders: [
                 { name: 'Content-Type', value: result.response.contentType },
                 { name: 'Access-Control-Allow-Origin', value: '*' },
               ],
               body,
             })
           } else {
             await cdp.send('Fetch.continueRequest', {
               requestId: event.requestId,
             })
           }
+        } catch {
+          // Target may have closed mid-request; safe to ignore.
+        }
       })


await cdp.send('Fetch.enable', { patterns: cdpPatterns })
} catch {
// Target may have been closed before we could attach.
// This is expected for transient targets like service workers.
}
})
}
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
"generate:fixtures": "node scripts/generate-fixtures.ts",
"generate:lexicons": "lex build --lexicons lexicons --out shared/types/lexicons --clear",
"test": "vite test",
"test:a11y": "pnpm build:test && pnpm test:a11y:prebuilt",
"test:a11y:prebuilt": "LIGHTHOUSE_COLOR_MODE=dark ./scripts/lighthouse-a11y.sh && LIGHTHOUSE_COLOR_MODE=light ./scripts/lighthouse-a11y.sh",
"test:browser": "pnpm build:test && pnpm test:browser:prebuilt",
"test:browser:prebuilt": "playwright test",
"test:browser:ui": "pnpm build:test && pnpm test:browser:prebuilt --ui",
Expand Down
Loading
Loading