Conversation
Move the cucumber+Playwright suite from CI-only to runnable on any Nix
host. tests/nix/ is a self-contained bundle (shell.nix + mod.just +
README) — its own nixpkgs pin is required because playwright-driver
must match the npm playwright version (1.57.0). The bundle is set up
to be lifted into a shared repo alongside chrome-devtools when a third
consumer appears; until then each repo carries its own copy.
CI (.github/workflows/ci.yaml) is untouched — apt-provided Chromium
on Ubuntu runners is cheaper than spinning up a Nix devshell, and CI
remains the single source of truth for the {live, static} matrix.
Extract the e2e-live / e2e-static recipe body into a private `_e2e mode` recipe; the public recipes become argument-passing dependencies. The live/static matrix stays atomic (CI ownership argument from the prior Lowy round still holds), but the recipe body lives in one place.
`run *cmd` interpolates the caller's command inside single-quotes for `nix-shell --run`, so a literal single-quote in `cmd` breaks shell parsing. The README markets this bundle for cross-repo adoption, so the constraint is now stated at the recipe site.
Project rule (.claude/skills/code-police): every just recipe must carry a doc comment, including private ones.
just modules default to running recipes in the module's own directory, which made `just e2e run "cd tests && ..."` fail with `cd: tests: No such file or directory` — the inner shell had its cwd at tests/nix/. [no-cd] keeps the cwd at the caller's invocation point so `cd tests` resolves against the consumer's project root.
`npm install` regenerates the lockfile (adds peer: true annotations) even when no real deps changed, leaving a dirty tree after every local run. `npm ci` is lockfile-respecting and is what CI uses, so local now matches CI exactly.
Owner
Author
Hickey/Lowy Analysis
Hickey rationaleThe only structural issue introduced by this PR was duplication between the two recipe bodies (Finding 1) — extracted into a private Lowy rationaleAll four pre-considered questions resolve to no-ops. The decomposition correctly encapsulates independent axes of change without leakage:
The independent nixpkgs pin is decoupled from the parent flake on purpose: a Playwright upgrade rotates this pin and |
Owner
Author
|
| Step | Status | Duration | Verification |
|---|---|---|---|
| sync | ✓ | 1s | git fetch ok; forge=github; noGit=false |
| research | ✓ | 2m 45s | Confirmed scope: nixpkgs in flake.lock has playwright-driver 1.53.1, but tests/package.json pins playwright 1.57.0 — version skew means tests/nix/shell.nix must own its own nixpkgs pin (matches Kolu rev f8573b9c, has playwright-driver 1.57.0). Just 1.42.2 supports source_directory() for module-relative paths. |
| branch | ✓ | 3s | On branch e2e-just (created from origin/master) |
| implement | ✓ | 1m 40s | Created tests/nix/{shell.nix,mod.just,README.md}; added mod e2e + e2e-live/e2e-static recipes to top-level justfile; rewrote tests/README.md Running section. |
| check | ✓ | 30s | cabal build all: exit 0, all 56 modules + emanote exe built clean. |
| docs | ✓ | 16s | Docs in sync. No prose docs reference the e2e suite. |
| fmt | ✓ | 30s | pre-commit run --all-files: cabal-fmt + fourmolu + hlint + nixpkgs-fmt all Passed. |
| commit | ✓ | 14s | Commit 397c94a on branch e2e-just; pushed to origin/e2e-just. |
| hickey+lowy | ✓ | 5m 6s | Hickey: 5 findings (2 Fix in this PR, 3 No-op). Lowy: 4 No-ops. Two follow-up commits: 7723e3b (refactor) + b4c3b50 (docs). |
| police | ✓ | 2m 27s | Rules: 1 violation (missing doc comment on _e2e), fact-check clean, elegance clean. Fix: 3d7a1f5. |
| test | ✓ | 1m 59s | just e2e-live: 12/12 scenarios passed; just e2e-static: 12/12. Path bug found and fixed: [no-cd] attr in 3c4d1d4. |
| create-pr | ✓ | 2m 28s | Draft PR #660 + hickey/lowy comment. Lockfile-drift fix 5536036 (npm ci) attached after creation. |
| ci | ✓ | 5m 11s | flake-parts-docs SUCCESS, e2e-tests(live) SUCCESS, e2e-tests(static) SUCCESS, website SKIPPED. |
| done | ✓ | 0s | PR #660 ready |
| Total | 24m 48s |
Slowest step: ci (5m 11s)
Optimization suggestions
ci(5m 11s) is the single largest fixed cost. Re-runs after a small fix should use--from ci-onlyto skip everything before. The 5m is dominated bye2e-testsmatrix (apt + npm install + emanote build per leg, x2). Cutting this further would mean caching the npm dependencies on Actions runners, not a workflow-side change.hickey+lowy(5m 6s) already ran the two sub-agents in parallel. Wall-time is bounded by the slower lens (hickey at ~3m); not further reducible without trimming sub-agent prompts.testretried once because the firstjust e2e-livehit a path bug (just module's default cwd is the module's own directory, not the project root). The[no-cd]discovery is something a smoke-test of any newmod.justrecipe would have caught earlier — worth folding into the implement step's verification next time, before commit.policerules pass found one issue (missing doc comment on_e2e mode) — manual pre-flight of new recipes against the project's justfile rule would skip the police-fix loop entirely.
Workflow completed at 2026-04-25T15:03:22Z.
Drop the tests/nix/ subdirectory and its README. shell.nix and mod.just now sit directly under tests/. Mod-import path becomes `mod e2e 'tests/mod.just'`. Stripped the now-irrelevant cross-repo language from shell.nix's header comment. Layout list in tests/README.md points at the two files directly. The portable-bundle posture is gone; this is just the e2e suite's local Nix runtime, no more, no less.
just --list takes only the line immediately preceding the recipe (or its attribute) as the doc comment. The multi-line constraint comment clobbered the descriptive line — `just --list e2e` was showing "wherever they invoked just." instead of the recipe summary. Move the caller-notes to a file-level header block; keep one descriptive line above the recipe.
So the do workflow's test step picks them up alongside `just test`.
srid
commented
Apr 25, 2026
Drop the just-module indirection. `_e2e mode` now invokes `nix-shell tests/shell.nix --run "..."` directly. One fewer file, one fewer hop, same behavior.
Drops the hardcoded fetchTree rev in tests/shell.nix; it now reads the locked rev from flake.lock's `nixpkgs-latest` input. The new input tracks nixpkgs-unstable, decoupled from the main `nixpkgs` input (which the Haskell build pins). Bumping playwright is now: nix flake update nixpkgs-latest npm install --package-lock-only --prefix tests …paired so playwright-driver and the npm playwright stay aligned. Bumps tests/package.json to playwright 1.58.2 to match the rev that nixpkgs-latest currently locks (playwright-driver 1.58.2). 12/12 scenarios pass in both live and static modes. Addresses PR review comment from @srid on tests/shell.nix.
2 tasks
srid
added a commit
to juspay/skills
that referenced
this pull request
Apr 25, 2026
## Summary - New `nix-playwright` skill capturing the pattern from [srid/emanote#660](srid/emanote#660) for running an existing Playwright e2e suite locally on NixOS / pure-Nix hosts - Covers the `nixpkgs-latest` flake input, a self-contained `tests/shell.nix` that reads the parent lock to provide `playwright-driver.browsers` + `PLAYWRIGHT_BROWSERS_PATH`, and a justfile entry - Flags the critical driver ↔ npm `playwright` version coupling (drift triggers "browser revision X not found") ## Test plan - [ ] Verify SKILL.md frontmatter and table entry render - [ ] Apply skill to a project with an existing Playwright suite and confirm `just e2e` works on NixOS 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cucumber+Playwright suite at
tests/was CI-only by design —Playwright's bundled Chromium download wants
apt-get+sudo, whicha Nix devshell can't provide. So the tests gating every PR couldn't
be exercised locally; you had to push and wait. This wires up
pkgs.playwright-driver.browsers+PLAYWRIGHT_BROWSERS_PATH+PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1intests/shell.nixsojust e2e-live/just e2e-staticwork on any Nix host.The Playwright-side nixpkgs is a separate
nixpkgs-latestflakeinput, decoupled from the main
nixpkgspin (which the Haskellbuild owns). Bumping Playwright is a paired step:
nix flake update nixpkgs-latestplus a matching version bump intests/package.json. Drift between the two breaks Playwright atlaunch with a "browser revision X not found" error — the rationale
is documented at the bump site in
tests/shell.nix.Touches #657 (the "Run e2e tests locally" bullet of finish-AI-config).
Try it locally