Skip to content

[codex] Make onboarding demo reset provenance-based#2300

Merged
riderx merged 10 commits into
mainfrom
codex/safe-demo-reset
May 20, 2026
Merged

[codex] Make onboarding demo reset provenance-based#2300
riderx merged 10 commits into
mainfrom
codex/safe-demo-reset

Conversation

@riderx
Copy link
Copy Markdown
Member

@riderx riderx commented May 19, 2026

Summary (AI generated)

  • Added service-role-only provenance tracking for onboarding demo rows.
  • Changed demo reset and legacy onboarding cleanup to delete only tracked demo-owned rows and guard against cascading into untracked real data.
  • Updated demo seeding to insert fresh demo rows and fail on collisions instead of upserting or reusing real app rows.
  • Added regression tests for mixed real/demo data, cascade refusal, completion-trigger cleanup, and RPC privileges.
  • Regenerated Supabase types for the new schema/functions and existing webhook delivery_version columns.

Motivation (AI generated)

The previous reset path used app-wide cleanup keyed by app_id and need_onboarding. That is unsafe once a pending onboarding app can contain both demo and production data. Resetting demo data needs row provenance, and deleting parent demo rows must not cascade into untracked user-owned child rows.

Business Impact (AI generated)

This protects customer app versions, channels, devices, deploy history, build rows, and analytics from being removed during onboarding/demo resets. It reduces support and restoration risk after demo onboarding, while keeping demo data resettable for sales/onboarding flows.

Test Plan (AI generated)

  • bun lint
  • bun lint:backend
  • bunx eslint tests/onboarding-demo-reset.test.ts
  • bun typecheck
  • bun run supabase:db:reset
  • bun run supabase:with-env -- bunx vitest run tests/onboarding-demo-reset.test.ts
  • bun run supabase:with-env -- bunx vitest run tests/build_time_tracking.test.ts tests/cron_stat_org.test.ts tests/private-error-cases.test.ts
  • bun test:backend

Generated with AI

Summary by CodeRabbit

  • New Features

    • Safer, transactional demo app seeding that reliably provisions demo content in one operation.
    • Provenance-backed demo tracking so demo rows are explicitly recorded and cleared.
  • Bug Fixes

    • More robust demo reset that preserves real app data, prevents unsafe cascades, and refreshes app rollups.
    • Improved cache/metrics clearing tied to demo resets.
  • Tests

    • New integration tests validating demo seed/reset behaviors and safety guards.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 19, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c1f14805-9f4c-4786-a276-1f73a43af7db

📥 Commits

Reviewing files that changed from the base of the PR and between e2295c3 and 4a496fa.

📒 Files selected for processing (1)
  • tests/onboarding-demo-reset.test.ts

📝 Walkthrough

Walkthrough

This PR replaces onboarding demo reset with provenance tracking. A new migration introduces an onboarding_demo_data table and guarded reset functions that safely delete only tracked demo rows while preventing cascades into production data. Demo app seeding is refactored to use a single Postgres transaction with bulk inserts instead of incremental Supabase operations. Comprehensive tests validate reset behavior, guards, legacy cleanup, and authorization.

Changes

Safe Onboarding Demo Data Reset

Layer / File(s) Summary
Provenance-based demo reset infrastructure
supabase/migrations/20260519123613_safe_demo_data_reset.sql
New onboarding_demo_data table with RLS tracks which rows belong to demo seeding. track_onboarding_demo_data() records provenance, claim_legacy_onboarding_demo_data() backfills legacy demo markers, and reset_onboarding_demo_app_data() safely deletes only tracked rows with multiple guards against cascading into untracked data. Legacy app processing loop and rollup refresh helper complete the infrastructure. clear_onboarding_app_data() wrappers retain compatibility.
Demo seeding refactored to transactional approach
supabase/functions/_backend/public/app/demo.ts
Imports PG client utilities. Adds system demo version helpers to distinguish placeholder versions. Implements seedOnboardingDemoDataInTransaction() for bulk Postgres inserts of all demo data (versions, manifest, channels, deploys, devices, daily metrics, builds) via jsonb_to_recordset, linking insertions via track_onboarding_demo_data, and explicit COMMIT/ROLLBACK with error handling. createDemoApp() prepares row-data arrays in memory and calls the transaction seeder once instead of multiple Supabase upserts.
Integration test coverage
tests/onboarding-demo-reset.test.ts
Test setup with helpers for app generation and relation tracking. Main test seeds mixed real/demo data, runs reset, and asserts preservation of real rows and removal of tracked demo rows. Guard tests verify: system placeholders are preserved, reset rejects when version metrics are non-nullable, cascade is prevented into untracked channels, legacy demo markers are correctly identified and removed, naming conflicts are handled, onboarding completion only clears tracked data, and role-based authorization restricts reset/track/claim functions to service_role only.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant ResetFunc as reset_onboarding_demo_app_data
  participant LegacyClaim as claim_legacy_onboarding_demo_data
  participant ProvenanceTable as onboarding_demo_data
  participant DataTables as demo data tables
  
  Client->>ResetFunc: call with app_uuid
  ResetFunc->>LegacyClaim: rebuild legacy provenance
  LegacyClaim->>ProvenanceTable: upsert legacy row markers
  ResetFunc->>DataTables: check guards (cascade safety)
  DataTables-->>ResetFunc: assert no untracked cascade
  ResetFunc->>DataTables: delete tracked demo rows
  ResetFunc->>ProvenanceTable: remove tracked rows
  ResetFunc-->>Client: reset complete
Loading
sequenceDiagram
  participant createDemoApp as createDemoApp()
  participant seedTxn as seedOnboardingDemoDataInTransaction()
  participant pgClient as Postgres Client
  participant tracking as track_onboarding_demo_data
  
  createDemoApp->>createDemoApp: generate demoSeedId + prepare version rows
  createDemoApp->>createDemoApp: prepare manifest/channel/deploy rows
  createDemoApp->>createDemoApp: prepare device/daily chart rows
  createDemoApp->>createDemoApp: prepare build request rows
  createDemoApp->>seedTxn: pass all row arrays + demoSeedId
  seedTxn->>pgClient: BEGIN transaction
  seedTxn->>pgClient: bulk insert all data via jsonb_to_recordset
  seedTxn->>tracking: call track_onboarding_demo_data for each relation
  seedTxn->>pgClient: clear app_metrics_cache for org
  seedTxn->>pgClient: COMMIT or ROLLBACK
  seedTxn-->>createDemoApp: return insert counts
  createDemoApp-->>createDemoApp: log transaction committed summary
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • Cap-go/capgo#2296: Both PRs modify the onboarding data-deletion and reset paths; main PR introduces provenance-based clear_onboarding_app_data() behavior while the related PR adjusts guard conditions and reverts the onboarding completion trigger.
  • Cap-go/capgo#2291: Both PRs touch onboarding cleanup—PR #2291 originally introduced clear_onboarding_app_data and onboarding-completion triggers, while the main PR replaces the reset implementation with provenance tracking while retaining clear_onboarding_app_data as a compatibility wrapper.

Poem

🐰 Demo data, tracked with care,
Provenance rows floating there,
Postgres txns, bulk and fast,
Guards ensure no data blast,
Tests hop through each safety cast.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title '[codex] Make onboarding demo reset provenance-based' clearly summarizes the main change—implementing provenance-based demo reset instead of app-wide cleanup, which is the core objective across all file changes.
Description check ✅ Passed The PR description includes a Summary section with the key changes and motivation. However, it lacks the 'Test plan' section explicitly required by the template—though test details appear within the Summary rather than in a dedicated section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/safe-demo-reset

Comment @coderabbitai help to get the list of available commands and usage tips.

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented May 19, 2026

Merging this PR will not alter performance

✅ 43 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing codex/safe-demo-reset (4a496fa) with main (7632d62)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@riderx riderx marked this pull request as ready for review May 19, 2026 13:35
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@riderx riderx marked this pull request as draft May 19, 2026 22:35
Comment thread supabase/functions/_backend/public/app/demo.ts Fixed
@riderx riderx marked this pull request as ready for review May 20, 2026 01:29
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/onboarding-demo-reset.test.ts (1)

63-64: ⚡ Quick win

Consider using it.concurrent() for parallel test execution.

Per coding guidelines, tests should use it.concurrent() instead of it() to run tests in parallel within the same file for faster CI/CD. Each test creates isolated data with unique app IDs, so they can safely run concurrently.

-describe('onboarding demo reset', () => {
-  it('deletes tracked demo rows while preserving real app data', async () => {
+describe('onboarding demo reset', () => {
+  it.concurrent('deletes tracked demo rows while preserving real app data', async () => {

Apply the same change to all other it() calls in this file (lines 325, 399, 459, 539, 635, 727, 775).

As per coding guidelines: "use it.concurrent() instead of it() to run tests in parallel within the same file for faster CI/CD"

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/onboarding-demo-reset.test.ts` around lines 63 - 64, Replace all
synchronous test declarations using it(...) with concurrent test declarations
it.concurrent(...) in this file so the tests run in parallel; specifically
update the top-level test starting with the "onboarding demo reset" describe and
every other it() call referenced (the tests beginning around lines 63, 325, 399,
459, 539, 635, 727, 775) by changing the call site to use it.concurrent and
keeping the same async test bodies and names (no other logic changes required).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/onboarding-demo-reset.test.ts`:
- Around line 63-64: Replace all synchronous test declarations using it(...)
with concurrent test declarations it.concurrent(...) in this file so the tests
run in parallel; specifically update the top-level test starting with the
"onboarding demo reset" describe and every other it() call referenced (the tests
beginning around lines 63, 325, 399, 459, 539, 635, 727, 775) by changing the
call site to use it.concurrent and keeping the same async test bodies and names
(no other logic changes required).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e2dce202-4597-4dee-acc8-fbc563053081

📥 Commits

Reviewing files that changed from the base of the PR and between 7632d62 and 0a6e28e.

📒 Files selected for processing (8)
  • cli/README.md
  • cli/package.json
  • cli/webdocs/build.mdx
  • package.json
  • supabase/functions/_backend/public/app/demo.ts
  • supabase/functions/_backend/utils/version.ts
  • supabase/migrations/20260519123613_safe_demo_data_reset.sql
  • tests/onboarding-demo-reset.test.ts
💤 Files with no reviewable changes (2)
  • cli/README.md
  • cli/webdocs/build.mdx

@riderx
Copy link
Copy Markdown
Member Author

riderx commented May 20, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 20, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/onboarding-demo-reset.test.ts (1)

583-610: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Make the legacy build markers unique and match migration patterns.

These two it.concurrent() tests still insert the same builder_job_id and upload_session_key literals. Reusing fixed seed identifiers breaks the parallel-safe test-data rule and can become flaky if build_requests constrains either field or matches on them outside app_id. The migration's demo data tracking uses strict LIKE patterns ('demo-job-%' and 'demo-session-%'), so unique values must preserve these prefixes.

Suggested change
-        'demo-job-legacy',
-        'demo-session-legacy',
+        `demo-job-onboarding-demo-reset-${randomUUID()}`,
+        `demo-session-onboarding-demo-reset-${randomUUID()}`,

Also applies to: 675-702

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/onboarding-demo-reset.test.ts` around lines 583 - 610, The test inserts
reuse fixed literals for builder_job_id and upload_session_key
("demo-job-legacy" and "demo-session-legacy") which breaks parallel-safe
test-data; change the values generated in the executeSQL call so they remain
prefixed with "demo-job-" and "demo-session-" but are unique per test run (e.g.
append appId, a timestamp or random suffix). Update the INSERT call used by the
it.concurrent() tests (the executeSQL invocation that sets builder_job_id and
upload_session_key) to build unique strings while preserving the required LIKE
prefixes so the migration patterns ('demo-job-%' and 'demo-session-%') still
match.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tests/onboarding-demo-reset.test.ts`:
- Around line 583-610: The test inserts reuse fixed literals for builder_job_id
and upload_session_key ("demo-job-legacy" and "demo-session-legacy") which
breaks parallel-safe test-data; change the values generated in the executeSQL
call so they remain prefixed with "demo-job-" and "demo-session-" but are unique
per test run (e.g. append appId, a timestamp or random suffix). Update the
INSERT call used by the it.concurrent() tests (the executeSQL invocation that
sets builder_job_id and upload_session_key) to build unique strings while
preserving the required LIKE prefixes so the migration patterns ('demo-job-%'
and 'demo-session-%') still match.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 26c272fa-64ec-450b-aa75-3238c87cd6e8

📥 Commits

Reviewing files that changed from the base of the PR and between 0a6e28e and e2295c3.

📒 Files selected for processing (2)
  • supabase/functions/_backend/public/app/demo.ts
  • tests/onboarding-demo-reset.test.ts
💤 Files with no reviewable changes (1)
  • supabase/functions/_backend/public/app/demo.ts

@sonarqubecloud
Copy link
Copy Markdown

@riderx riderx merged commit 7712cd5 into main May 20, 2026
42 checks passed
@riderx riderx deleted the codex/safe-demo-reset branch May 20, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant