Skip to content

test: expand unit + e2e coverage from test-map (batch 2)#1752

Merged
senamakel merged 1 commit into
tinyhumansai:mainfrom
senamakel:test/expand-coverage-from-test-map-batch1
May 14, 2026
Merged

test: expand unit + e2e coverage from test-map (batch 2)#1752
senamakel merged 1 commit into
tinyhumansai:mainfrom
senamakel:test/expand-coverage-from-test-map-batch1

Conversation

@senamakel
Copy link
Copy Markdown
Member

@senamakel senamakel commented May 14, 2026

Summary

  • Adds ~115 Rust unit tests across 8 new files (~2,082 lines) targeting high-priority coverage gaps identified from test-map.md.
  • Extends mega-flow.spec.ts with 3 new e2e scenarios (WhatsApp tool, account isolation across resets, Composio webhook roundtrip).
  • All new tests pass locally; cargo fmt + tsc clean.

Problem

test-map.md enumerates 1,220 synthesized features; many high-priority items have no direct unit-test coverage in core (agent harness loop guards, account scoping, WhatsApp data store, Meet URL handling, local-AI preset tiering, personalization cache, accessibility permission poller). This PR is batch 2 of an ongoing campaign (PR #1724 was batch 1) closing those gaps.

Solution

Per-domain unit tests colocated as *_tests.rs siblings, included via #[cfg(test)] #[path = "..."] mod ...;. Each new file targets narrow, deterministic behaviour — no network, no time flakes.

Unit (Rust):

  • agent/harness/harness_gap_tests.rs — tool_loop, dispatcher fallback parsing (XML / <invoke> / markdown fence), tool-access boundary, context guard + tool timeout, follow-up resolution, silence timer, datetime grounding
  • accessibility/permissions_tests.rs — macOS poller state transitions, granted/denied detection, refresh
  • learning/prompt_sections_tests.rs — personalization cache facet ranking, capping, eviction
  • local_ai/presets_tests.rs — 5-tier RAM presets + vision-mode resolution per tier
  • meet/ops_tests.rs — Meet URL normalization, host + display-name validation
  • meet_agent/ops_tests.rs — VAD lifecycle, sample-rate validation
  • whatsapp_data/schemas_tests.rs — agent-facing tool registry contract (3 read-only tools; ingest excluded)
  • whatsapp_data/store_tests.rs — account isolation, dedup, pruning
  • credentials/ops_tests.rs extended — per-account scoping + session isolation

E2E (app/test/e2e/specs/mega-flow.spec.ts):

  • Scenario 7 — WhatsApp list_chats RPC contract (uses internal whatsapp_data_ingest to seed)
  • Scenario 10 — account switch + isolation across config_reset_local_data cycles
  • Scenario 11 — Composio enable_trigger → webhook ingress roundtrip

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
  • Diff coverage ≥ 80% — this PR is all added test code; coverage gate is trivially satisfied.
  • N/A: behaviour-only change to test coverage; no feature rows added/removed in docs/TEST-COVERAGE-MATRIX.md.
  • N/A: no feature IDs touched (test-only PR).
  • No new external network dependencies introduced (mock backend used per Testing Strategy)
  • N/A: no release-surface change.
  • N/A: no linked issue.

Impact

  • Runtime/platform: none — pure test additions, no shipped product change.
  • CI: adds ~115 unit tests to cargo test --lib (sub-second per module locally) + 3 e2e scenarios to the mega-flow spec.

Related

  • Closes:
  • Follow-up PR(s)/TODOs:
    • Skipped from this batch: spawn-depth e2e (no openhuman.agent_run RPC; mock LLM can't force nested tool calls deterministically), accessibility-permission e2e (no openhuman.accessibility_* controllers registered yet). Will revisit once those RPC surfaces land.
    • Continue test-map coverage campaign in batch 3 (multimodal/ppt artifact path, cross-chat memory retrieval beyond personalization, skill auth-mode parser).

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: test/expand-coverage-from-test-map-batch1
  • Commit SHA: a04a4c9

Validation Run

  • pnpm --filter openhuman-app format:check — auto-fix applied by pre-push hook on mega-flow.spec.ts, amended into the commit before push
  • pnpm typecheck — clean in app/
  • Focused tests: cargo test --lib whatsapp_data (33/33), local_ai (241/241), screen_intelligence (89/89), meet::ops (22/22), meet_agent::ops (24/24), credentials::ops (32/32), learning::prompt_sections (15/15), harness (284/284) — all green
  • Rust fmt/check (if changed): cargo fmt applied; cargo check --tests clean
  • N/A: Tauri shell unchanged

Validation Blocked

  • command: N/A
  • error: N/A
  • impact: N/A

Behavior Changes

  • Intended behavior change: none — test-only PR.
  • User-visible effect: none.

Parity Contract

  • Legacy behavior preserved: yes — no production code semantics changed. The only prod-file edits are #[cfg(test)] mod declarations that include the new sibling test files.
  • Guard/fallback/dispatch parity checks: N/A.

Duplicate / Superseded PR Handling

  • Duplicate PR(s): N/A
  • Canonical PR: N/A
  • Resolution (closed/superseded/updated): N/A

Note: 2 pre-existing failures observed in meet_agent::brain::tests and meet_agent::rpc::tests on main — not introduced by this PR. Flagged for follow-up.

Summary by CodeRabbit

  • Tests
    • Expanded end-to-end test coverage with new scenarios for WhatsApp operations, account-switching validation, and webhook roundtrip testing
    • Added comprehensive unit tests across core modules including accessibility permissions, credential isolation, agent harness behavior, model tier selection, Meet validation, and data store operations

Review Change Stack

Adds ~115 Rust unit tests across 8 new files (~2,082 lines) and 3 new
e2e scenarios in mega-flow.spec.ts. Targets high-priority coverage gaps
from test-map.md spanning agent harness, accessibility, local-AI
presets, credentials/account scoping, learning cache, meet agent, and
WhatsApp data store + RPC.

Unit (Rust):
- agent/harness/harness_gap_tests.rs: tool_loop, dispatcher fallback
  parsing, tool-access boundary, context guard/timeout, follow-up
  resolution, silence timer, datetime grounding
- accessibility/permissions_tests.rs: macOS poller state transitions,
  granted/denied detection, refresh
- learning/prompt_sections_tests.rs: personalization cache facet
  ranking, capping, eviction
- local_ai/presets_tests.rs: 5-tier RAM presets + vision-mode resolution
- meet/ops_tests.rs: Meet URL normalization, host + name validation
- meet_agent/ops_tests.rs: VAD lifecycle, sample-rate validation
- whatsapp_data/schemas_tests.rs: agent-facing tool registry contract
  (ingest excluded)
- whatsapp_data/store_tests.rs: account isolation, dedup, pruning
- credentials/ops_tests.rs: per-account scoping + session isolation

E2E (mega-flow.spec.ts):
- Scenario 7: WhatsApp list_chats RPC contract
- Scenario 10: account switch + isolation across reset cycles
- Scenario 11: Composio enable_trigger -> webhook ingress roundtrip
@senamakel senamakel requested a review from a team May 14, 2026 18:14
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 14, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e7ec4081-52c6-4308-b320-f3b57c28b89d

📥 Commits

Reviewing files that changed from the base of the PR and between 4d5e0d5 and a04a4c9.

📒 Files selected for processing (18)
  • app/test/e2e/specs/mega-flow.spec.ts
  • src/openhuman/accessibility/permissions.rs
  • src/openhuman/accessibility/permissions_tests.rs
  • src/openhuman/agent/harness/harness_gap_tests.rs
  • src/openhuman/agent/harness/mod.rs
  • src/openhuman/credentials/ops_tests.rs
  • src/openhuman/learning/prompt_sections.rs
  • src/openhuman/learning/prompt_sections_tests.rs
  • src/openhuman/local_ai/presets.rs
  • src/openhuman/local_ai/presets_tests.rs
  • src/openhuman/meet/ops.rs
  • src/openhuman/meet/ops_tests.rs
  • src/openhuman/meet_agent/ops.rs
  • src/openhuman/meet_agent/ops_tests.rs
  • src/openhuman/whatsapp_data/schemas.rs
  • src/openhuman/whatsapp_data/schemas_tests.rs
  • src/openhuman/whatsapp_data/store.rs
  • src/openhuman/whatsapp_data/store_tests.rs

📝 Walkthrough

Walkthrough

This PR extends test coverage across multiple Rust modules and the E2E test suite. Four new E2E scenarios exercise WhatsApp ingest/read flows, account switching, and Composio trigger/webhook roundtrips. Ten Rust subsystems (accessibility, agent harness, credentials, learning, local_ai, meet, meet_agent, whatsapp_data schemas/store) receive comprehensive unit test suites extracted into dedicated test files, improving code organization and coverage without modifying production APIs.

Changes

Test Coverage Expansion and Refactoring

Layer / File(s) Summary
E2E mega-flow scenario expansion
app/test/e2e/specs/mega-flow.spec.ts
Four new scenarios (7-11) added: WhatsApp ingest/list flow, skipped spawn-depth/accessibility flows, account switching with thread visibility, and Composio trigger + webhook roundtrip. All scenarios verify RPC liveness via core.ping.
Permissions and accessibility platform testing
src/openhuman/accessibility/permissions.rs, src/openhuman/accessibility/permissions_tests.rs
Unit tests for permission_to_str, microphone_denied_message, platform-gated detect_permissions, serde round-trip serialization of PermissionStatus, and internal automation_state_stale_cache denial-reset behavior. Tests extracted into permissions_tests.rs via conditional module wiring.
Agent harness gap coverage and context/tool behavior
src/openhuman/agent/harness/mod.rs, src/openhuman/agent/harness/harness_gap_tests.rs
Tests for tool-loop end-to-end flow, iteration exhaustion, tool-name whitelist filtering, context guard exhaustion/compaction, tool-call parsing (<invoke> and markdown variants), timeout defaults, and datetime formatting. Defines ScriptedProvider, EchoTool, PingTool test doubles.
Credential and session workspace isolation
src/openhuman/credentials/ops_tests.rs
Three tokio tests validating account-scoped credential and session storage: separate workspaces remain isolated, clearing one account does not affect another, and each workspace sees only its own provider credentials.
Facet cache and prompt section rendering
src/openhuman/learning/prompt_sections.rs, src/openhuman/learning/prompt_sections_tests.rs
Tests for load_learned_from_cache top-K capping (25), facet ordering by stability, pinned marker rendering, dropped-state exclusion, per-class rendering patterns, empty-cache handling, and FacetCache::drop_below_threshold Active-row protection. Helpers create in-memory cache and ProfileFacet instances.
Model tier parsing, preset metadata, and tier-based behavior
src/openhuman/local_ai/presets.rs, src/openhuman/local_ai/presets_tests.rs
Tests for ModelTier::from_str_opt parsing/aliases, round-trip serialization, preset_for_tier selection, vision_mode_for_tier mappings, preset metadata consistency, apply_preset_to_config enable semantics, current_tier_from_config preference logic, and supports_screen_summary tier gating.
Meet URL and display-name validation
src/openhuman/meet/ops.rs, src/openhuman/meet/ops_tests.rs
Tests for validate_meet_url (Meet/lookup forms, whitespace, rejection of invalid segments) and validate_display_name (64-char boundary, whitespace-only, control-character, unicode validation). Inline tests removed from ops.rs and extracted to ops_tests.rs.
Meet agent audio signal processing and voice-activity detection
src/openhuman/meet_agent/ops.rs, src/openhuman/meet_agent/ops_tests.rs
Tests for validate_sample_rate, sanitize_request_id trimming/validation, frame_rms numerical correctness, and Vad state transitions with hangover timing, utterance boundaries, and idle emissions. Includes loud_frame helper.
WhatsApp controller schema and function validation
src/openhuman/whatsapp_data/schemas.rs, src/openhuman/whatsapp_data/schemas_tests.rs
Tests asserting agent-facing registry contains three read-only tools (list_chats, list_messages, search_messages) excluding ingest, schema count consistency, internal controller count (4), namespace validation, and input contract requirements (query/chat_id as non-optional strings).
WhatsApp store account isolation and data consistency
src/openhuman/whatsapp_data/store.rs, src/openhuman/whatsapp_data/store_tests.rs
Tests for account-scoped isolation in list_chats/list_messages/search_messages, deduplication on upserts with content/metadata updates, chat-stat correctness (message_count, last_message_ts) after inserts and pruning, and graceful handling of invalid/empty IDs. Includes test helpers (make_store, chat_meta, simple_message).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • tinyhumansai/openhuman#1373: The E2E mega-flow scenario 7 exercises the read-only WhatsApp RPC surface (openhuman.whatsapp_data_ingest and openhuman.whatsapp_data_list_chats) that was introduced in this PR.
  • tinyhumansai/openhuman#1308: The mega-flow scenarios 7/10/11 directly exercise WhatsApp and Composio RPC endpoints that depend on functionality from this PR.
  • tinyhumansai/openhuman#425: The local_ai presets unit test additions directly complement the tier-based preset and VisionMode implementation introduced in that PR.

Suggested reviewers

  • graycyrus

Poem

🐰 Hops through a forest of tests,
Each module now wears its best
Schemas and schemas, permutations galore,
WhatsApp and credence fortified more!
From harness to agent, from scope to store,
Coverage blooms like clover on the floor! 🍀

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: expanding unit and e2e test coverage across multiple modules. It is concise, specific, and directly reflects the content of the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Comment @coderabbitai help to get the list of available commands and usage tips.

@senamakel senamakel merged commit 7d8826f into tinyhumansai:main May 14, 2026
30 of 37 checks passed
AusAgentSmith pushed a commit to AusAgentSmith/openhuman that referenced this pull request May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant