Skip to content

fix(migrations): add migration lock and batch DB updates#484

Merged
ReenigneArcher merged 2 commits intomasterfrom
fix/rank-migrations
Feb 15, 2026
Merged

fix(migrations): add migration lock and batch DB updates#484
ReenigneArcher merged 2 commits intomasterfrom
fix/rank-migrations

Conversation

@ReenigneArcher
Copy link
Member

Description

Introduce a global MIGRATION_LOCK to prevent concurrent migrations and refactor Reddit/Mee6 import flows into internal methods. Migrations now disable database.GIT_ENABLED and restore its original state on completion or error. Reddit migration applies updates in a single transactional context by building an existing-users map and performing inserts/updates with progress logging. Mee6 migration collects updates asynchronously and applies them in batched transactions, with improved error handling and final DB sync.

Screenshot

Issues Fixed or Closed

Roadmap Issues

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Unit tests have been added or updated for any new or modified functionality

AI Usage

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

Introduce a global MIGRATION_LOCK to prevent concurrent migrations and refactor Reddit/Mee6 import flows into internal methods. Migrations now disable database.GIT_ENABLED and restore its original state on completion or error. Reddit migration applies updates in a single transactional context by building an existing-users map and performing inserts/updates with progress logging. Mee6 migration collects updates asynchronously and applies them in batched transactions, with improved error handling and final DB sync.
@codecov
Copy link

codecov bot commented Feb 15, 2026

Codecov Report

❌ Patch coverage is 35.29412% with 88 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.96%. Comparing base (d56d2ad) to head (c90533d).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/common/rank.py 35.29% 88 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #484      +/-   ##
==========================================
+ Coverage   59.45%   59.96%   +0.51%     
==========================================
  Files          24       24              
  Lines        1761     1806      +45     
==========================================
+ Hits         1047     1083      +36     
- Misses        714      723       +9     
Files with missing lines Coverage Δ
src/common/rank.py 58.27% <35.29%> (+4.19%) ⬆️

Extract reusable helpers and simplify migration flows for Reddit and Mee6 imports. Added _process_reddit_item and _update_reddit_rank_database to centralize author handling, XP aggregation, DB upsert logic and progress logging; this reduces duplication and improves error handling. For Mee6, added _fetch_mee6_page (with timeout/error handling) and _process_mee6_batch, and changed the migration to fetch pages and apply updates in batches, reducing DB churn. Also updated final stats to use derived counts and restored original Git state before final sync. Overall aims: clearer structure, fewer nested try/except blocks, better logging, and more efficient DB writes.
@sonarqubecloud
Copy link

@ReenigneArcher ReenigneArcher merged commit 74105a1 into master Feb 15, 2026
16 of 18 checks passed
@ReenigneArcher ReenigneArcher deleted the fix/rank-migrations branch February 15, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant