Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/settings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"enabledPlugins": {
"frontend-design@claude-plugins-official": true,
"plugin-dev@claude-plugins-official": true
"plugin-dev@claude-plugins-official": true,
"feature-dev@claude-plugins-official": true
Comment on lines +4 to +5
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove personal Claude AI configuration from version control.

This file appears to be a personal Claude AI assistant configuration that was accidentally committed. The .claude/settings.json file and directory are typically local development tool settings and should not be tracked in version control, as they:

  • Are unrelated to the PR's ROB-2 functionality
  • Could force personal Claude plugin preferences on all developers
  • Serve no functional purpose for the codebase
🗑️ Recommended actions
  1. Remove this file from the commit
  2. Add .claude/ to .gitignore to prevent future accidental commits
# Add to .gitignore
echo ".claude/" >> .gitignore
🤖 Prompt for AI Agents
In @.claude/settings.json around lines 4 - 5, Remove the personal Claude
settings file from the repo by unstaging and deleting .claude/settings.json from
the commit/index and committing that removal, then prevent future commits by
adding the .claude/ directory to .gitignore and committing the updated
.gitignore; ensure no functional code references .claude/settings.json remain in
the PR before pushing the updated commit.

}
}
156 changes: 156 additions & 0 deletions packages/docs/audits/error-boundary-assessment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Error Boundary Implementation Assessment

## What Was Done

### 1. Centralized Error Logger (`packages/web/src/lib/errorLogger.js`)

Created a new centralized error logging module that provides:

- **`logError(error, context)`** - Logs caught exceptions with component/action context
- **`logWarning(message, context)`** - Logs non-fatal issues (cache misses, degraded functionality)
- **`logInfo(message, context)`** - Logs important state transitions
- **`bestEffort(promise, context)`** - Wraps operations that can fail silently (cleanup, cache updates)
- **`withErrorLogging(component, action)`** - Decorator for async functions
- **`logAndRethrow(component, action)`** - For catch blocks that need to log but propagate errors

All functions normalize errors through `@corates/shared` and include structured context (component name, action, timestamp).

### 2. Enhanced Error Boundaries

**AppErrorBoundary** (main component):

- Now uses `logError` instead of raw `console.error`
- Passes component name context for better debugging

**SectionErrorBoundary** (new capabilities):

- Added `name` prop for identifying which section failed
- Added `onRetry` callback for custom retry logic (e.g., query invalidation)
- Added `retryLabel` prop for custom button text
- Error message now includes section name: "Error in Projects"

### 3. Section-Level Error Isolation

Wrapped major UI sections with `SectionErrorBoundary`:

| Location | Sections Wrapped |
| -------------------- | ------------------------------------------------------------- |
| `Dashboard.jsx` | Projects, Local Appraisals |
| `ProjectView.jsx` | Overview, All Studies, To-Do, Reconcile, Completed (each tab) |
| `AdminLayout.jsx` | Admin content area |
| `SettingsLayout.jsx` | Settings content area |

### 4. Best-Effort Operation Cleanup

Replaced silent `.catch(() => {})` patterns with `bestEffort()`:

```javascript
// Before
clearFormState(type).catch(() => {});

// After
bestEffort(clearFormState(type), { operation: 'clearFormState', type });
```

This ensures failures are logged as warnings rather than completely swallowed.

### 5. Admin Queries Refactor

- Switched from raw `fetch` to `apiFetch` for consistent error handling
- Extracted duplicate config into `ADMIN_QUERY_CONFIG` constant
- Removed redundant comments

---

## Why It Was Done

### Problem 1: Silent Failures

Best-effort operations were using `.catch(() => {})` which completely swallowed errors. If cleanup routines started failing (e.g., IndexedDB quota exceeded), there was no visibility into the issue.

### Problem 2: No Error Context

When `AppErrorBoundary` caught errors, it logged them with minimal context. Debugging required correlating timestamps with user actions manually.

### Problem 3: Catastrophic Failures

A single component error (e.g., bad data in one project card) would crash the entire dashboard. Users lost access to all functionality instead of just the affected section.

### Problem 4: No Monitoring Integration Point

Error logging was scattered across the codebase. Integrating Sentry or similar would require finding and modifying dozens of locations.

### Problem 5: Inconsistent Error Handling in Admin

Admin queries used raw `fetch` while the rest of the app used `apiFetch`, leading to inconsistent error normalization and handling.

---

## Next Steps

### 1. Integrate Sentry (or Alternative)

The `errorLogger.js` module includes commented placeholder code for Sentry integration. When ready:

```javascript
// In errorLogger.js, uncomment and configure:
if (typeof window !== 'undefined' && window.Sentry) {
window.Sentry.captureException(error, {
tags: { component, action },
extra: context,
});
}
```

**Why:** Centralized logging is only useful if errors are aggregated somewhere. Sentry provides alerting, deduplication, and release tracking.

### 2. Add Error Boundaries to Remaining High-Risk Areas

Current coverage is good for main layouts, but these areas should be considered:

- Individual study cards in lists (prevent one bad study from hiding all)
- Checklist domain sections (isolate domain rendering failures)
- PDF viewer (already somewhat isolated, but could benefit from explicit boundary)
- Modal/dialog content (prevent modal errors from crashing parent)

**Why:** Finer-grained boundaries mean smaller blast radius. A corrupted study shouldn't hide the entire study list.

### 3. Add User-Facing Error Reporting

The current error UI shows "Try Again" but doesn't let users report issues. Consider:

- "Report this issue" button that captures error context
- Session replay integration for debugging user-reported issues
- Error ID display so users can reference specific failures in support requests

**Why:** Users encountering errors are a valuable signal. Making it easy to report helps identify edge cases.

### 4. Add Error Boundary Recovery Strategies

`SectionErrorBoundary` now supports `onRetry` for custom recovery. Use this for:

- Query-backed sections: invalidate and refetch on retry
- WebSocket sections: reconnect on retry
- Form sections: restore from draft state on retry

**Why:** Generic "reset and re-render" often fails for the same reason. Section-specific recovery can actually fix the issue.

### 5. Add Error Metrics/Analytics

Track error rates over time:

- Errors per session
- Errors by component/section
- Error recovery success rate (did retry work?)

**Why:** Helps identify regressions. If error rate spikes after a deploy, you know something broke.

### 6. Consider Suspense Boundaries

SolidJS supports Suspense for async loading. Combining error boundaries with suspense boundaries would provide:

- Loading states during data fetch
- Error states on fetch failure
- Smooth transitions between states

**Why:** Currently some components handle their own loading/error states inconsistently. Suspense + ErrorBoundary provides a unified pattern.
182 changes: 182 additions & 0 deletions packages/docs/audits/rob2-checklist-implementation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
# ROB-2 Checklist Implementation

**Date:** 2026-01-10
**Branch:** `262-add-rob-2`
**Status:** Core implementation complete

## Overview

Implemented the RoB 2 (Risk of Bias 2) checklist for assessing risk of bias in randomized trials. This is the third checklist type in CoRATES, following AMSTAR2 and ROBINS-I.

ROB-2 is the Cochrane Collaboration's tool for assessing risk of bias in randomized controlled trials. It evaluates bias across 5 domains with signalling questions that feed into algorithmic judgements.

## What Was Accomplished

### Shared Package (`@corates/shared`)

Created the core ROB-2 logic in `packages/shared/src/checklists/rob2/`:

| File | Purpose |
| ------------ | ----------------------------------------------------------------------------------- |
| `schema.ts` | Question definitions, domain structures, response types, constants |
| `scoring.ts` | Decision algorithms for each domain (from official ROB-2 decision diagrams) |
| `create.ts` | Factory function `createROB2Checklist()` |
| `answers.ts` | Utilities: `scoreROB2Checklist`, `isROB2Complete`, `getAnswers`, `getDomainSummary` |
| `index.ts` | Module exports |

**Key schema elements:**

- 5 domains with signalling questions
- Domain 2 has two variants: 2a (effect of assignment/ITT) and 2b (effect of adhering/per-protocol)
- Response types: Y (Yes), PY (Probably Yes), PN (Probably No), N (No), NI (No Information), NA (Not Applicable)
- Judgement levels: Low, Some concerns, High
- Preliminary section for study metadata and aim selection

### UI Components (`packages/web`)

Created components in `packages/web/src/components/checklist/ROB2Checklist/`:

| Component | Purpose |
| ------------------------ | ------------------------------------------------- |
| `ROB2Checklist.jsx` | Main orchestrating component |
| `PreliminarySection.jsx` | Study design, aims, interventions, sources |
| `DomainSection.jsx` | Individual domain with questions and auto-scoring |
| `SignallingQuestion.jsx` | Response buttons for each question |
| `DomainJudgement.jsx` | Judgement display badges |
| `ScoringSummary.jsx` | Compact summary strip with domain chips |
| `OverallSection.jsx` | Final overall risk of bias section |
| `checklist.js` | Helper functions and re-exports |
| `checklist-map.js` | Schema re-exports from shared package |
| `index.js` | Module entry point |

### Yjs Integration

Created `packages/web/src/primitives/useProject/checklists/handlers/rob2.js`:

- `ROB2Handler` class for real-time collaborative editing
- Methods: `extractAnswersFromTemplate`, `createAnswersYMap`, `serializeAnswers`, `updateAnswer`, `getTextGetter`

### Registry Integration

Modified files to register ROB-2:

- `packages/web/src/checklist-registry/types.js` - Added ROB2 type constant and metadata
- `packages/web/src/checklist-registry/index.js` - Registered scoring and creation functions
- `packages/web/src/primitives/useProject/checklists/index.js` - Added handler and `getRob2Text()`
- `packages/web/src/components/checklist/GenericChecklist.jsx` - Added ROB2Checklist rendering
- `packages/shared/package.json` - Added export paths for checklists

## Key Features

### Auto-Scoring

Domain judgements are automatically calculated from signalling question responses using the official ROB-2 decision algorithms. The overall risk of bias is then derived from all domain judgements:

- If any domain is "High" -> Overall is "High"
- If any domain is "Some concerns" (and none High) -> Overall is "Some concerns"
- If all domains are "Low" -> Overall is "Low"

### Domain 2 Variants

The preliminary section includes an "aim" selection that determines which Domain 2 variant to show:

- **Assignment (ITT)**: Shows Domain 2a - Effect of assignment to intervention
- **Adhering (per-protocol)**: Shows Domain 2b - Effect of adhering to intervention

### Collaborative Editing

Full Yjs integration enables real-time collaboration:

- All text fields (experimental intervention, comparator, numerical result) are Y.Text
- Signalling question responses sync across users
- Domain judgements update automatically as questions are answered

## File Structure

```
packages/shared/src/checklists/rob2/
schema.ts # Questions, domains, constants
scoring.ts # Decision algorithms
create.ts # Factory function
answers.ts # Answer utilities
index.ts # Exports

packages/web/src/components/checklist/ROB2Checklist/
ROB2Checklist.jsx
PreliminarySection.jsx
DomainSection.jsx
SignallingQuestion.jsx
DomainJudgement.jsx
ScoringSummary.jsx
OverallSection.jsx
checklist.js
checklist-map.js
index.js

packages/web/src/primitives/useProject/checklists/handlers/
rob2.js # Yjs handler
```

## Verification

- Build passes: `pnpm --filter web build`
- Type check passes: `pnpm --filter @corates/shared typecheck`
- No ROB2-related lint errors
- Unit tests pass: `pnpm --filter @corates/shared test` (64 ROB-2 tests)

## Testing

Unit tests are located at `packages/shared/src/checklists/__tests__/rob2.test.ts` and cover:

- `createROB2Checklist` - Factory function validation and initialization
- `scoreRob2Domain` - All decision tree paths for each domain:
- Domain 1 (Randomization): 8 test cases covering all paths
- Domain 2a (Assignment/ITT): 5 test cases including Part 1/Part 2 combination
- Domain 2b (Adhering): 5 test cases covering major paths
- Domain 3 (Missing data): 5 test cases
- Domain 4 (Measurement): 8 test cases including NI branches
- Domain 5 (Selection): 6 test cases
- `scoreAllDomains` - Overall calculation with different aim selections
- `scoreROB2Checklist` - High-level scoring
- `isROB2Complete` - Completion detection
- `getAnswers` - Answer extraction

Run tests with: `pnpm --filter @corates/shared test`

## Next Steps

### Immediate

1. **Integration Testing** - Test the full flow in the browser:
- Create a new ROB-2 checklist from the study view
- Complete preliminary section and verify Domain 2 variant switching
- Answer signalling questions and verify auto-scoring
- Test collaborative editing with multiple users

### Short-term

2. **Question Notes** - Add support for free-text notes on individual signalling questions (similar to AMSTAR2)
3. **Direction of Bias** - Implement predicted direction of bias per domain (currently only overall)
4. **Export/Import** - Add CSV export functionality (similar to AMSTAR2)
5. **Reconciliation** - Implement checklist comparison and reconciliation for ROB-2

### Future Enhancements

6. **Validation Warnings** - Show warnings for incomplete or inconsistent responses
7. **Conditional Questions** - Some questions should be skipped based on earlier answers (currently all shown)
8. **Help Text** - Add inline help text for signalling questions from the official guidance document
9. **Traffic Light Visualization** - Add the standard ROB-2 traffic light plot for visualizing results

## Decision Diagram Sources

The scoring algorithms were implemented from the decision diagrams in:

- `packages/web/src/components/checklist/ROB2Checklist/scoring/decision-diagrams/`

These files contain the official ROB-2 decision algorithms that determine domain judgements based on signalling question responses.

## References

- [RoB 2 Tool (Official)](https://www.riskofbias.info/welcome/rob-2-0-tool)
- [Cochrane Handbook Chapter 8](https://training.cochrane.org/handbook/current/chapter-08)
- [RoB 2 Detailed Guidance Document](https://drive.google.com/file/d/19R9savfPdCHC8XLz2iiMvL_71lPJERWK/view)
Loading