Skip to content

chore(migration/001): convert legacy configuration to the new format#641

Merged
DarianM merged 44 commits intomainfrom
migration-cli
Apr 20, 2026
Merged

chore(migration/001): convert legacy configuration to the new format#641
DarianM merged 44 commits intomainfrom
migration-cli

Conversation

@DarianM
Copy link
Copy Markdown
Member

@DarianM DarianM commented Mar 26, 2026

Summary

one-time CLI to migrate legacy S3 config data to the new format

  1. Adds a migrations CLI that reads wallet configs from the legacy S3 prefix, converts them to the new shape, writes them to the new prefix.
  • wallets already present in the new prefix are cleaned up (legacy key deleted) and counted as skipped.
  1. Adds migrations/s3/ as a standalone S3 client package wrapping client-s3, shared by the CLI.

  2. Includes a dry-run mode (--dry-run) that simulates all S3 writes/deletes without touching real data, and writes its own separate log file.

  3. Each run produces a timestamped log file under migrations recording successful, skipped, and failed wallets.

  4. Migration packages are excluded from the root pnpm workspace — they live under their own migrations/pnpm-workspace.yaml and are installed separately (cd migrations && pnpm install), so CI is not affected

@DarianM DarianM linked an issue Mar 26, 2026 that may be closed by this pull request
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 26, 2026

Deployment results

Worker Alias URL Outcome
API - 56949bae success
CDN - 60e41f9f success
App - 23485356 success

Logs #24672982760

@DarianM DarianM marked this pull request as ready for review March 31, 2026 11:39
Comment thread migrations/001-config-s3-structure/index.ts Outdated
Comment thread migrations/s3/index.ts
Comment thread pnpm-workspace.yaml Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
@DarianM DarianM requested a review from sidvishnoi April 5, 2026 15:54
@DarianM DarianM changed the title feat(/config-migration): add migration service to convert legacy configuration to the new format feat(migration): add service to convert legacy configuration to the new format Apr 15, 2026
@DarianM DarianM requested a review from sidvishnoi April 16, 2026 13:24
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/pnpm-workspace.yaml
Comment on lines +6 to +7
const LEGACY_PREFIX = '20250717-dev'
const NEW_PREFIX = '20260305-dev'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still going through to understand changes, but should these be environment variables or stored in a different way or is this something only going to be ran once and no further change would be required?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're specific to each migration.

@DarianM DarianM requested a review from sidvishnoi April 16, 2026 21:34
Comment thread migrations/001-config-s3-structure/__tests__/index.test.ts
Comment thread migrations/001-config-s3-structure/index.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migrate-cli.ts Outdated
Comment thread migrations/001-config-s3-structure/migration-log.ts Outdated
@DarianM DarianM requested a review from sidvishnoi April 17, 2026 09:13
Comment thread migrations/001-config-s3-structure/index.ts Outdated
Copy link
Copy Markdown
Member

@sidvishnoi sidvishnoi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Let's do a screen share when you run this migration.

@sidvishnoi sidvishnoi changed the title feat(migration): add service to convert legacy configuration to the new format feat(migration/001): convert legacy configuration to the new format Apr 20, 2026
@sidvishnoi sidvishnoi changed the title feat(migration/001): convert legacy configuration to the new format chore(migration/001): convert legacy configuration to the new format Apr 20, 2026
@DarianM DarianM merged commit 666b85c into main Apr 20, 2026
4 checks passed
@DarianM DarianM deleted the migration-cli branch April 20, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: Create utility for migrating stored data formats - Phase 4

3 participants