Skip to content

test(e2e): onboarding simple vs advanced mode spec#1989

Merged
senamakel merged 7 commits into
tinyhumansai:mainfrom
senamakel:test/onboarding-flow
May 17, 2026
Merged

test(e2e): onboarding simple vs advanced mode spec#1989
senamakel merged 7 commits into
tinyhumansai:mainfrom
senamakel:test/onboarding-flow

Conversation

@senamakel
Copy link
Copy Markdown
Member

@senamakel senamakel commented May 17, 2026

Summary

  • Adds onboarding-modes.spec.ts — a Docker/Linux E2E that walks the post-login onboarding wizard in both Simple (Cloud) and Advanced (Custom) modes and asserts the resulting state lands in the core's ${OPENHUMAN_WORKSPACE}/config.toml.
  • Upgrades opentelemetry{,_sdk,-otlp} 0.31 → 0.32 to drop the unbuildable tonic 0.14.6 from the workspace graph (E0463/E0223 cascade on the standing toolchain).
  • Fixes a stale RPC method name and adds a data-testid pair to VoicePanel so the spec can drive the Voice-Configure path deterministically.
  • Persists the CEF download cache in a named docker volume so the ~400 MB download doesn't redo every container run.

Problem

  • We had no E2E coverage for the onboarding wizard's two top-level modes (Cloud vs Custom), nor for the claim that advanced wizard settings persist to config.toml immediately. The wizard has been refactored multiple times this quarter and silent regressions on the Custom path have shipped twice.
  • The Linux E2E docker pipeline was already broken on main before this branch: opentelemetry-proto 0.31's gen-tonic-messages feature pulls tonic 0.14.6 even when the workspace only enables HTTP-proto transport. The build dies with cascading E0463: can't find crate for http/http_body/bytes/... errors inside tonic.

Solution

  • opentelemetry 0.31 → 0.32 — in 0.32 gen-tonic-messages requires only prost; tonic is gated behind gen-tonic. The crate disappears from the graph and cargo check is clean on both the root crate and the Tauri shell. Zero source changes required.
  • onboarding-modes.spec.ts — drives the wizard via stable data-testid selectors:
    • Phase A: Cloud path → /home, asserts onboarding_completed = true in config.toml.
    • Phase B: Custom path, Default on every step → asserts each wizard panel renders (Inference / Voice / OAuth) and onboarding completes.
    • Phase C: Custom path, Configure on Voice → flips stt-provider-select and asserts [local_ai] stt_provider updates in config.toml via the same voice_set_providers RPC the panel uses. Originally targeted auto_start, but that section is gated by disabled = !sttReady which is always true in CI (no Whisper assets) — switched to the always-visible provider selector.
  • Tiny helper at app/test/e2e/helpers/config-toml.ts — a regex reader for the workspace config.toml (no TOML parser dep).
  • Docker compose — added e2e-cef-cache:/root/Library/Caches/tauri-cef volume.

Verified locally inside ghcr.io/tinyhumansai/openhuman_ci:latest:

✓ simple/cloud path: welcome → runtime-choice → cloud → home
✓ simple/cloud path: config.toml reflects onboarding_completed=true
✓ advanced/custom path: walks all wizard steps with Default choice
✓ advanced/custom path: Configure on Voice toggles auto_start into config.toml
4 passing (7.7s)

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
  • N/A: this PR adds an E2E spec — Vitest/Rust line coverage on the WDIO spec itself is not what the gate measures; the dep bump in Cargo.toml is one-line config with no executable surface; the two data-testid additions in VoicePanel are JSX attributes only. diff-cover will treat the spec source as covered by its own run.
  • N/A: behaviour-only change — no feature added/removed/renamed in the matrix.
  • N/A: no feature IDs touched.
  • No new external network dependencies introduced (mock backend used per Testing Strategy)
  • N/A: no release-cut surfaces touched.
  • N/A: no linked issue.

Impact

  • Desktop only. No production runtime behavior changes. The two data-testid additions to VoicePanel are inert JSX attributes. The opentelemetry minor bump (0.31 → 0.32) is a pure dep refresh — no API surface used by the app changed, and cargo check is clean on both crates.
  • Removes ~28 yanked-or-unbuildable crates from Cargo.lock (tonic, tonic-prost, tonic-prost-build, prost-types, and friends) along the way.

Related

  • Closes:
  • Follow-up PR(s)/TODOs: none.

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: test/onboarding-flow
  • Commit SHA: a615e09

Validation Run

  • pnpm --filter openhuman-app format:check (pre-push hook auto-formatted spec/helper; re-committed as a615e099)
  • N/A: pnpm typecheck — spec is @ts-nocheck per existing convention in app/test/e2e/specs/*.
  • Focused tests: bash app/scripts/e2e-run-session.sh test/e2e/specs/onboarding-modes.spec.ts onboarding-modes (inside docker compose) — 4/4 passing.
  • Rust fmt/check (if changed): cargo check clean on root + Tauri shell after the opentelemetry 0.32 bump.
  • Tauri fmt/check (if changed): same as above.

Validation Blocked

  • command: N/A
  • error: N/A
  • impact: N/A

Behavior Changes

  • Intended behavior change: None at runtime. New test asserts existing behavior.
  • User-visible effect: None.

Parity Contract

  • Legacy behavior preserved: yes — no product code paths altered beyond two data-testid attributes.
  • Guard/fallback/dispatch parity checks: N/A.

Duplicate / Superseded PR Handling

  • Duplicate PR(s): none.
  • Canonical PR: this one.
  • Resolution: N/A.

Summary by CodeRabbit

  • Chores

    • Updated OpenTelemetry, OpenTelemetry SDK, and OTLP dependencies to 0.32
    • Persisted CEF cache in the Docker environment to avoid repeated downloads
  • Tests

    • Added comprehensive E2E onboarding tests covering simple and advanced flows, including persistence checks
    • Introduced test helpers for reading and polling the configuration file
    • Added UI test identifiers for voice settings to improve test reliability

Review Change Stack

senamakel added 6 commits May 16, 2026 19:12
Adds `onboarding-modes.spec.ts` covering:
- Cloud (simple) path: welcome → runtime choice → /home, asserts
  `onboarding_completed = true` lands in `${OPENHUMAN_WORKSPACE}/config.toml`.
- Custom (advanced) path with Default on every wizard step: asserts the
  Inference, Voice, and OAuth wizard panels all render via their
  `data-testid`s and onboarding completes.
- Custom (advanced) Configure path on Voice: toggles `auto_start` via the
  embedded `VoicePanel`, clicks Save, and asserts `voice_server.auto_start`
  updates immediately on disk.

Adds `helpers/config-toml.ts` — a tiny regex reader for the workspace
config.toml so specs can assert persisted state without parsing the full
TOML grammar. Adds `data-testid`s for the voice auto-start toggle and the
voice save button so the spec can drive them.
tonic 0.14.6 declares `edition = "2024"` but ships code with
`Self::Item` ambiguity, missing `#[pin]` proc-macro scopes, and
`E0463` crate-not-found errors that break the build under any
currently-shipping Rust toolchain.

The root cause: opentelemetry-otlp 0.31's `http-proto` feature
activates `opentelemetry-proto/gen-tonic-messages`, which in
proto 0.31 unconditionally pulls in `tonic` (optional dep with
`features = ["codegen"]`), dragging in tonic 0.14.6.  We use only
the HTTP-proto transport — gRPC/tonic is never needed.

In opentelemetry-proto 0.32, `gen-tonic-messages` was refactored
to require only `prost`; tonic is now gated behind `gen-tonic`.
Upgrading opentelemetry/{sdk,otlp} 0.31 → 0.32 removes tonic
from the dependency graph entirely, fixing the build with no
source changes required.

Verified: `cargo check` on the root crate and the Tauri shell both
finish cleanly after the update.
…ethod name

All controller methods are registered under the openhuman.{namespace}_{function}
naming scheme (see core/all.rs rpc_method_name). The spec was calling
config.set_onboarding_completed (bare namespace.function), which the dispatcher
does not recognise, producing "unknown method" and aborting Phase B and C.
cef-dll-sys (the CEF bindings build script) writes the downloaded
CEF binary to ~/Library/Caches/tauri-cef/ inside the container.
Without a named volume this path is ephemeral — each new container
re-downloads the ~400 MB CEF archive, which fails when the
CDN is unreachable or DNS is broken in the container.

Add e2e-cef-cache mapped to /root/Library/Caches/tauri-cef so
the download persists across container runs, matching how
e2e-tauri-target already caches the Rust build artifacts.
…nfigure phase

The auto-start checkbox + Save button in VoicePanel sit inside a
`!disabled && settings` block, where `disabled = !sttReady`. In the
linux CI container we never download Whisper assets, so sttReady is
false and that block never renders — the spec was failing on the
`voice-auto-start-toggle` testid lookup at 290:67.

Switch Phase C to drive the always-visible `stt-provider-select`
dropdown. Flipping it fires `voice_set_providers` which writes
`config.local_ai.stt_provider` to config.toml via `config.save()` —
same end-to-end claim, but on the path that does render in CI.

The synthetic change-event uses the native value setter so React
sees the new value even though the alternate `<option>` is rendered
disabled.

Add `readSectionString` helper to config-toml for quoted-string keys.
@senamakel senamakel requested a review from a team May 17, 2026 08:37
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 17, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f8fc53fc-4f1f-408c-94ee-e6f907fc5d66

📥 Commits

Reviewing files that changed from the base of the PR and between a615e09 and 7b2e871.

📒 Files selected for processing (1)
  • app/test/e2e/specs/onboarding-modes.spec.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/test/e2e/specs/onboarding-modes.spec.ts

📝 Walkthrough

Walkthrough

This PR adds E2E onboarding tests and config.toml helpers, instruments VoicePanel with test IDs, bumps OpenTelemetry crates to 0.32, and adds a Docker Compose named volume to cache CEF binaries for E2E runs.

Changes

E2E Onboarding Testing

Layer / File(s) Summary
Config TOML Helper Library
app/test/e2e/helpers/config-toml.ts
New utility module exports functions to resolve the config.toml path from OPENHUMAN_WORKSPACE, synchronously read the file, and parse top-level or section-scoped key-value pairs using regex. Includes boolean and string-value readers that handle quote stripping.
Voice Settings Test Instrumentation
app/src/components/settings/panels/VoicePanel.tsx
Added data-testid attributes (voice-auto-start-toggle and voice-save-settings) to the auto-start checkbox and save button to enable E2E test targeting.
Onboarding E2E Test Suite
app/test/e2e/specs/onboarding-modes.spec.ts
New E2E spec file with test helpers for hash-based navigation, onboarding state reset via RPC, and UI interaction via data-testid. Implements three test phases: Phase A validates the Simple/Cloud onboarding path, Phase B exercises Advanced/Custom with default selections across Runtime, Inference, Voice, and OAuth steps, and Phase C verifies Advanced/Custom with Voice configuration toggling and config.toml persistence polling. All phases assert landing on /home and confirm onboarding_completed=true in config.toml.

Infrastructure and Dependency Updates

Layer / File(s) Summary
OpenTelemetry Dependency Upgrade
Cargo.toml
Bumped opentelemetry, opentelemetry_sdk, and opentelemetry-otlp from version 0.31 to 0.32 while preserving feature flags (trace, metrics) and default-features = false configuration.
E2E Docker Compose CEF Caching
e2e/docker-compose.yml
Added named volume e2e-cef-cache to persist the CEF binary cache under /root/Library/Caches/tauri-cef across container runs, reducing repeated downloads in CI environments.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🐰 Three onboarding paths now tested,
Config helpers parse with care,
CEF cache stays between builds—
E2E tests hop forth and fair!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main addition: a new E2E test spec for onboarding flows, which is the primary change in this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added the working A PR that is being worked on by the team. label May 17, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/test/e2e/specs/onboarding-modes.spec.ts`:
- Around line 16-20: Update the Phase C description to match what the test
actually asserts: change references to toggling "voice_server.auto_start" and
saving to instead mention changing the "local_ai.stt_provider" via the provider
selector and saving, so failure triage points to the real assertion target;
specifically edit the Phase C wording in onboarding-modes.spec.ts (the block
that currently references `voice_server.auto_start`) and any other occurrence
around line 267 to mention `local_ai.stt_provider` and the provider selector
interaction.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 887f5cda-b0b5-4c3d-a880-babcddf67140

📥 Commits

Reviewing files that changed from the base of the PR and between 3aa8477 and a615e09.

⛔ Files ignored due to path filters (2)
  • Cargo.lock is excluded by !**/*.lock
  • app/src-tauri/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • Cargo.toml
  • app/src/components/settings/panels/VoicePanel.tsx
  • app/test/e2e/helpers/config-toml.ts
  • app/test/e2e/specs/onboarding-modes.spec.ts
  • e2e/docker-compose.yml

Comment thread app/test/e2e/specs/onboarding-modes.spec.ts Outdated
@senamakel senamakel merged commit 2e58438 into tinyhumansai:main May 17, 2026
23 of 24 checks passed
AusAgentSmith pushed a commit to AusAgentSmith/openhuman that referenced this pull request May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

working A PR that is being worked on by the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant