[Testing] E2E: establish suite(s) + CI for web and iOS (parent)

[auerbachb]

Summary

Parent tracker for end-to-end (E2E) testing that covers critical auth lifecycle flows on web and iOS.

Strategy (pick 1 or 2 suites — document decision in child PRs)

• Web: Prefer Playwright (or equivalent) against a local or preview deployment with a dedicated test DB / disposable accounts; never commit secrets.
• iOS: Prefer XCUITest (or Maestro) driving the Still Point app against a configurable API base URL pointed at staging/preview/local.

Two separate suites are expected (different runtimes), but one CI job matrix can orchestrate both.

Goals

• Document how to run E2E locally and in CI (README or AGENTS.md / docs/).
• Stable test accounts or per-run signup with unique emails (avoid flaking on duplicate email).
• CI wiring: optional workflow_dispatch or path filters so E2E does not block every tiny PR until stable.

Child issues

• (Will be filled after children are created — see linked issues.)

Acceptance criteria

• Chosen runner(s) committed with minimal smoke config.
• At least one green path documented for web and one for iOS (can be stub until child issues land).
• No production credentials in repo; use GitHub Actions secrets / local env only.

Related

• App Store compliance / account deletion: App Store: Guideline 5.1.1(v) — account deletion required #126

[github-actions]

@coderabbitai plan

[auerbachb]

Child issues

• [Testing] E2E (web): signup happy path #153 — E2E (web): signup happy path
• [Testing] E2E (web): account deletion flow #154 — E2E (web): account deletion flow
• [Testing] E2E (iOS): signup happy path #155 — E2E (iOS): signup happy path
• [Testing] E2E (iOS): account deletion flow #156 — E2E (iOS): account deletion flow

[coderabbitai]

🔗 Related PRs

#52 - Add TestFlight CI/CD workflow and update signing config [merged]
#93 - ci(ios): Xcode 26 / macOS 26 runner for TestFlight (ITMS-90725) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

Coding Plan

Summary

• Establish Playwright for web E2E with auth lifecycle smoke tests, using per-run unique signups for test isolation
• Implement Maestro for iOS E2E after making the API base URL configurable via build settings
• Create separate CI workflows with workflow_dispatch triggers and optional path-filtered PR triggers to avoid blocking unrelated changes
• Use a dedicated test database (POSTGRES_URL_TEST) and GitHub Actions secrets to ensure no production credentials are committed

Design Choices

Design Choice 1: iOS E2E Framework Selection

Options Considered:

1. XCUITest — Native Apple framework, requires Swift, tightly coupled to Xcode project structure
2. Maestro — YAML-based declarative tests, simpler setup, no Xcode project modifications needed, fast iteration

Chosen Option: 2

Rationale: Easier to bootstrap without modifying the existing XcodeGen project structure, and YAML-based tests are more accessible for contributors unfamiliar with Swift/XCTest.

Design Choice 2: Test Account Strategy

Options Considered:

1. Stable seed accounts — Use existing ava_seed/leo_seed accounts from seed script
2. Per-run unique signup — Generate unique email/username per test run using timestamps

Chosen Option: 2

Rationale: Avoids test pollution and race conditions in parallel runs; tests remain isolated and idempotent without requiring pre-seeded database state.

Design Choice 3: Test Database Approach

Options Considered:

1. Separate Neon branch — Use POSTGRES_URL_TEST pointing to an isolated Neon database branch
2. Cleanup after tests — Use production-like DB but delete test accounts after each run

Chosen Option: 1

Rationale: Safer isolation, aligns with existing POSTGRES_URL_TEST pattern in .env.example, and avoids any risk to real user data.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Web E2E Suite (Playwright)

Establish Playwright-based E2E testing for the web application with auth lifecycle coverage, test utilities, and CI integration.

Task 1: Install and Configure Playwright

Set up Playwright with Next.js-appropriate configuration for testing against a local dev server or preview deployment.

• Install @playwright/test as a dev dependency and run npx playwright install for browser binaries
• Create playwright.config.ts at project root with settings for: base URL (http://localhost:3000 default, overridable via env), webServer config to auto-start npm run dev, and appropriate timeouts
• Configure single browser (Chromium) initially to keep CI fast; expand later if needed
• Set test directory to e2e/ to separate from future unit tests

Task 2: Create Auth Test Utilities

Build reusable test helpers for authentication flows to reduce duplication across tests.

• Create e2e/utils/auth.ts with helper functions for: generating unique test emails/usernames (timestamp + random suffix pattern), signup flow helper, login flow helper, and account cleanup helper
• Create e2e/utils/api.ts with direct API request helpers using Playwright's request context for programmatic auth setup
• Helpers should return the authenticated page or context for chaining with subsequent test actions

Task 3: Write Auth Lifecycle Smoke Tests

Implement the minimum viable test coverage for the critical auth flows.

• Create e2e/auth.spec.ts covering: signup with unique credentials → verify redirect to app, login with valid credentials → verify authenticated state, logout → verify return to unauthenticated state
• Tests should use per-run unique accounts and clean up created accounts via DELETE /api/account in afterEach
• Include basic assertions for UI state (e.g., presence of main app content vs auth screen)

Task 4: Add npm Scripts and CI Workflow

Wire up test execution for local development and GitHub Actions.

• Add to package.json: "test:e2e": "playwright test", "test:e2e:ui": "playwright test --ui" for local debugging
• Create .github/workflows/e2e-web.yml with: workflow_dispatch trigger for manual runs, optional pull_request trigger with path filters (src/**, e2e/**, playwright.config.ts), Node.js 22 setup, Playwright browser installation (with caching), environment secrets (POSTGRES_URL_TEST, JWT_SECRET)
• Workflow should start the Next.js dev server and run tests against it
• Document required GitHub Actions secrets in workflow comments

Task 5: Document Web E2E Setup

Add documentation for running and extending web E2E tests.

• Create docs/testing.md with sections for: local prerequisites, running tests locally, CI configuration, writing new tests, test account strategy explanation
• Update AGENTS.md with brief testing section pointing to docs

🤖 Prompt for AI agents

Implement the Playwright-based web E2E testing infrastructure for a Next.js app.
The goal is to install and configure Playwright, build auth test utilities,
write smoke tests, wire up CI, and add documentation.

**1. Install and Configure Playwright**
- Install `@playwright/test` as a dev dependency
- Create `playwright.config.ts` at the project root with:
- Base URL defaulting to `http://localhost:3000`, overridable via environment
variable
  - `webServer` config to auto-start `npm run dev`
  - Appropriate timeouts
  - Single browser: Chromium only
  - Test directory set to `e2e/`

**2. Create Auth Test Utilities**
- Create `e2e/utils/auth.ts` with helpers for:
  - Generating unique test emails/usernames using timestamp + random suffix
  - Signup flow helper
  - Login flow helper
  - Account cleanup helper
- Create `e2e/utils/api.ts` with direct API request helpers using Playwright's
`request` context for programmatic auth setup
- Helpers should return the authenticated page or context for chaining

**3. Write Auth Lifecycle Smoke Tests**
- Create `e2e/auth.spec.ts` covering:
  - Signup with unique credentials → verify redirect to app
  - Login with valid credentials → verify authenticated state
  - Logout → verify return to unauthenticated state
- Use per-run unique accounts; clean up created accounts via `DELETE
/api/account` in `afterEach`
- Include basic UI state assertions (main app content vs. auth screen)

**4. Add npm Scripts and CI Workflow**
- Add to `package.json`:
  - `"test:e2e": "playwright test"`
  - `"test:e2e:ui": "playwright test --ui"`
- Create `.github/workflows/e2e-web.yml` with:
  - `workflow_dispatch` trigger for manual runs
- Optional `pull_request` trigger with path filters: `src/**`, `e2e/**`,
`playwright.config.ts`
  - Node.js 22 setup
  - Playwright browser installation with caching
  - Environment secrets: `POSTGRES_URL_TEST`, `JWT_SECRET`
  - Steps to start the Next.js dev server and run tests
  - Comments documenting required GitHub Actions secrets

**5. Documentation**
- Create `docs/testing.md` with sections for: local prerequisites, running tests
locally, CI configuration, writing new tests, and test account strategy
explanation
- Update `AGENTS.md` with a brief testing section pointing to `docs/testing.md`

Phase 2: iOS E2E Suite (Maestro)

Establish Maestro-based E2E testing for the iOS app with configurable API targeting and auth flow coverage.

Task 1: Make iOS API Base URL Configurable

Enable the iOS app to target different API environments (staging, local, preview) without code changes.

• Update ios/project.yml to add a new build configuration or scheme for testing that sets an API_BASE_URL user-defined build setting
• Modify ios/StillPointApp/Info.plist to include API_BASE_URL key that reads from build settings: $(API_BASE_URL)
• Update ios/StillPointShared/Sources/StillPointShared/APIClient.swift to read API_BASE_URL from Bundle.main.infoDictionary at init, falling back to production URL if not set
• Ensure the existing setBaseURL method remains available for runtime overrides during testing

Task 2: Set Up Maestro Test Structure

Initialize Maestro test infrastructure for iOS testing.

• Create .maestro/ directory at project root for test flows
• Create .maestro/config.yaml with app identifier (com.stillpoint.app or as defined in project.yml) and default settings
• Add Maestro installation instructions to documentation (Homebrew: brew install maestro)
• Create .maestro/utils/ directory for reusable flow fragments

Task 3: Write iOS Auth Flow Tests

Implement Maestro flows for the critical iOS authentication paths.

• Create .maestro/flows/auth-signup.yaml: launch app → tap Sign Up → enter unique email/username/password → submit → assert main app screen visible
• Create .maestro/flows/auth-login.yaml: launch app → enter credentials → submit → assert authenticated state
• Create .maestro/flows/auth-logout.yaml: from authenticated state → navigate to logout → confirm → assert auth screen visible
• Use Maestro's runFlow for composition and JavaScript for generating unique test credentials
• Account cleanup can call API directly via Maestro's evalScript with fetch or be handled by test DB reset

Task 4: Create iOS E2E CI Workflow

Set up GitHub Actions for running Maestro tests on iOS simulator.

• Create .github/workflows/e2e-ios.yml with: workflow_dispatch trigger for manual runs, macos-latest runner (or macos-14 for M1), Xcode setup step, XcodeGen installation, Maestro installation via Homebrew
• Workflow should: generate Xcode project, build app for simulator with test scheme/configuration, boot iOS simulator, install app, run Maestro tests
• Configure environment secrets for test API URL (or use local/preview deployment URL)
• Add path filters for ios/**, .maestro/** if enabling PR triggers

Task 5: Document iOS E2E Setup

Complete documentation for iOS testing workflow.

• Extend docs/testing.md with iOS section covering: Maestro installation, running tests locally against simulator, configuring API target environment, CI workflow explanation
• Document the test scheme/configuration and how to switch API targets
• Add troubleshooting section for common simulator/Maestro issues

🤖 Prompt for AI agents

Implement the Maestro-based iOS E2E testing infrastructure. The goal is to make
the iOS API base URL configurable via build settings, set up Maestro test
structure, write auth flow tests, create a CI workflow, and extend
documentation.

**1. Make iOS API Base URL Configurable**
- Update `ios/project.yml` to add a new build configuration or scheme for
testing with an `API_BASE_URL` user-defined build setting
- Modify `ios/StillPointApp/Info.plist` to add an `API_BASE_URL` key reading
from build settings: `$(API_BASE_URL)`
- Update `ios/StillPointShared/Sources/StillPointShared/APIClient.swift` to:
  - Read `API_BASE_URL` from `Bundle.main.infoDictionary` at init
  - Fall back to the production URL if not set
  - Keep the existing `setBaseURL` method available for runtime overrides

**2. Set Up Maestro Test Structure**
- Create `.maestro/` directory at the project root
- Create `.maestro/config.yaml` with app identifier (`com.stillpoint.app` or as
defined in `project.yml`) and default settings
- Create `.maestro/utils/` directory for reusable flow fragments

**3. Write iOS Auth Flow Tests**
- Create `.maestro/flows/auth-signup.yaml`: launch app → tap Sign Up → enter
unique email/username/password → submit → assert main app screen visible
- Create `.maestro/flows/auth-login.yaml`: launch app → enter credentials →
submit → assert authenticated state
- Create `.maestro/flows/auth-logout.yaml`: from authenticated state → navigate
to logout → confirm → assert auth screen visible
- Use Maestro's `runFlow` for composition and JavaScript for generating unique
test credentials
- Account cleanup can call the API directly via Maestro's `evalScript` with
fetch, or be handled by test DB reset

**4. Create iOS E2E CI Workflow**
- Create `.github/workflows/e2e-ios.yml` with:
  - `workflow_dispatch` trigger for manual runs
  - `macos-latest` runner (or `macos-14` for M1)
- Steps for: Xcode setup, XcodeGen installation, Maestro installation via
Homebrew
- Steps to: generate Xcode project, build app for simulator using the test
scheme/configuration, boot iOS simulator, install app, run Maestro tests
  - Environment secrets for test API URL
  - Path filters for `ios/**`, `.maestro/**` if enabling PR triggers

**5. Documentation**
- Extend `docs/testing.md` with an iOS section covering: Maestro installation,
running tests locally against simulator, configuring API target environment, and
CI workflow explanation
- Document the test scheme/configuration and how to switch API targets
- Add a troubleshooting section for common simulator/Maestro issues

Research

The project is a Next.js meditation app with an iOS companion. Currently, there is no testing infrastructure—no test frameworks, test files, or test scripts exist. The web app uses cookie-based JWT authentication (sp_token cookie) with public endpoints for signup/login and protected endpoints for sessions, thoughts, and friends. The iOS app has a hardcoded API base URL with an unused setBaseURL method. CI consists of three workflows: web build verification, iOS TestFlight deployment, and CodeRabbit planning. A seed script exists with safety guards for non-production databases.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement the Playwright-based web E2E testing infrastructure for a Next.js app.
The goal is to install and configure Playwright, build auth test utilities,
write smoke tests, wire up CI, and add documentation.

**1. Install and Configure Playwright**
- Install `@playwright/test` as a dev dependency
- Create `playwright.config.ts` at the project root with:
- Base URL defaulting to `http://localhost:3000`, overridable via environment
variable
  - `webServer` config to auto-start `npm run dev`
  - Appropriate timeouts
  - Single browser: Chromium only
  - Test directory set to `e2e/`

**2. Create Auth Test Utilities**
- Create `e2e/utils/auth.ts` with helpers for:
  - Generating unique test emails/usernames using timestamp + random suffix
  - Signup flow helper
  - Login flow helper
  - Account cleanup helper
- Create `e2e/utils/api.ts` with direct API request helpers using Playwright's
`request` context for programmatic auth setup
- Helpers should return the authenticated page or context for chaining

**3. Write Auth Lifecycle Smoke Tests**
- Create `e2e/auth.spec.ts` covering:
  - Signup with unique credentials → verify redirect to app
  - Login with valid credentials → verify authenticated state
  - Logout → verify return to unauthenticated state
- Use per-run unique accounts; clean up created accounts via `DELETE
/api/account` in `afterEach`
- Include basic UI state assertions (main app content vs. auth screen)

**4. Add npm Scripts and CI Workflow**
- Add to `package.json`:
  - `"test:e2e": "playwright test"`
  - `"test:e2e:ui": "playwright test --ui"`
- Create `.github/workflows/e2e-web.yml` with:
  - `workflow_dispatch` trigger for manual runs
- Optional `pull_request` trigger with path filters: `src/**`, `e2e/**`,
`playwright.config.ts`
  - Node.js 22 setup
  - Playwright browser installation with caching
  - Environment secrets: `POSTGRES_URL_TEST`, `JWT_SECRET`
  - Steps to start the Next.js dev server and run tests
  - Comments documenting required GitHub Actions secrets

**5. Documentation**
- Create `docs/testing.md` with sections for: local prerequisites, running tests
locally, CI configuration, writing new tests, and test account strategy
explanation
- Update `AGENTS.md` with a brief testing section pointing to `docs/testing.md`
===============================================================================

Task: 2

Implement the Maestro-based iOS E2E testing infrastructure. The goal is to make
the iOS API base URL configurable via build settings, set up Maestro test
structure, write auth flow tests, create a CI workflow, and extend
documentation.

**1. Make iOS API Base URL Configurable**
- Update `ios/project.yml` to add a new build configuration or scheme for
testing with an `API_BASE_URL` user-defined build setting
- Modify `ios/StillPointApp/Info.plist` to add an `API_BASE_URL` key reading
from build settings: `$(API_BASE_URL)`
- Update `ios/StillPointShared/Sources/StillPointShared/APIClient.swift` to:
  - Read `API_BASE_URL` from `Bundle.main.infoDictionary` at init
  - Fall back to the production URL if not set
  - Keep the existing `setBaseURL` method available for runtime overrides

**2. Set Up Maestro Test Structure**
- Create `.maestro/` directory at the project root
- Create `.maestro/config.yaml` with app identifier (`com.stillpoint.app` or as
defined in `project.yml`) and default settings
- Create `.maestro/utils/` directory for reusable flow fragments

**3. Write iOS Auth Flow Tests**
- Create `.maestro/flows/auth-signup.yaml`: launch app → tap Sign Up → enter
unique email/username/password → submit → assert main app screen visible
- Create `.maestro/flows/auth-login.yaml`: launch app → enter credentials →
submit → assert authenticated state
- Create `.maestro/flows/auth-logout.yaml`: from authenticated state → navigate
to logout → confirm → assert auth screen visible
- Use Maestro's `runFlow` for composition and JavaScript for generating unique
test credentials
- Account cleanup can call the API directly via Maestro's `evalScript` with
fetch, or be handled by test DB reset

**4. Create iOS E2E CI Workflow**
- Create `.github/workflows/e2e-ios.yml` with:
  - `workflow_dispatch` trigger for manual runs
  - `macos-latest` runner (or `macos-14` for M1)
- Steps for: Xcode setup, XcodeGen installation, Maestro installation via
Homebrew
- Steps to: generate Xcode project, build app for simulator using the test
scheme/configuration, boot iOS simulator, install app, run Maestro tests
  - Environment secrets for test API URL
  - Path filters for `ios/**`, `.maestro/**` if enabling PR triggers

**5. Documentation**
- Extend `docs/testing.md` with an iOS section covering: Maestro installation,
running tests locally against simulator, configuring API target environment, and
CI workflow explanation
- Document the test scheme/configuration and how to switch API targets
- Add a troubleshooting section for common simulator/Maestro issues

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!