Skip to content

Switch to AsyncPgConnection and clean up tests#103

Merged
leynos merged 3 commits intomainfrom
codex/update-database-backend-to-support-postgresql
Jun 12, 2025
Merged

Switch to AsyncPgConnection and clean up tests#103
leynos merged 3 commits intomainfrom
codex/update-database-backend-to-support-postgresql

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Jun 12, 2025

Summary

  • use AsyncPgConnection for the Postgres backend
  • pass only the database URL to run migrations for Postgres
  • provide apply_migrations helper for tests
  • deduplicate category list tests using a new helper

Testing

  • cargo clippy -- -D warnings
  • cargo test --quiet
  • cargo clippy --no-default-features --features postgres -- -D warnings
  • cargo build --no-default-features --features postgres
  • markdownlint docs/supporting-both-sqlite3-and-postgresql-in-diesel.md
  • nixie docs/supporting-both-sqlite3-and-postgresql-in-diesel.md

https://chatgpt.com/codex/tasks/task_e_684a0e7bd68c8322b29a4143e9166965

Summary by Sourcery

Switch the Postgres backend to use async connections and revamp migration routines, and streamline tests by deduplicating migration and TCP framing logic through new helpers.

New Features:

  • Add an apply_migrations helper to unify running embedded migrations across SQLite and PostgreSQL
  • Introduce a list_categories helper in news category tests to centralize TCP framing and response decoding

Enhancements:

  • Switch PostgreSQL backend to use diesel_async::AsyncPgConnection
  • Refactor run_migrations to accept only the database URL for PostgreSQL and spawn blocking tasks for migration
  • Consolidate migration invocation in both main code and test utilities under conditional compilation
  • Clean up duplicated migration and framing setup in news category tests using new helpers

Documentation:

  • Update documentation to explain using diesel_async for queries and migration handling differences between backends

Tests:

  • Refactor news category tests to use apply_migrations and list_categories helpers
  • Add cfg guards around SQLite-only tests in the database module

Summary by CodeRabbit

  • Documentation
    • Clarified how database migrations are handled asynchronously for both PostgreSQL and SQLite backends.
  • Refactor
    • Introduced a unified migration function to streamline migration calls across PostgreSQL and SQLite backends.
    • Refactored test code to reduce duplication by adding a helper function for listing news categories and centralising request/response handling.
    • Updated migration calls throughout the codebase to use the new unified migration function with additional parameters.

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Jun 12, 2025

Reviewer's Guide

This PR transitions the Postgres backend to use diesel_async’s AsyncPgConnection, restructures migration logic into a unified helper, and cleans up repetitive test setup by introducing a shared socket/ui helper and migration runner.

Sequence Diagram for Updated Postgres Migration Process

sequenceDiagram
    participant AppLogic as "Application Logic (e.g., setup_database)"
    participant PgMigrations as "run_migrations (Postgres)"
    participant BlkTask as "tokio::task::spawn_blocking"
    participant SyncPgConn as "PgConnection (sync, temporary)"
    participant PgDB as "PostgreSQL Database"

    AppLogic->>+PgMigrations: run_migrations(database_url)
    PgMigrations->>+BlkTask: spawn_blocking_task()
    BlkTask->>+SyncPgConn: PgConnection::establish(database_url)
    SyncPgConn-->>-BlkTask: sync_connection
    BlkTask->>+SyncPgConn: sync_connection.run_pending_migrations(MIGRATIONS)
    SyncPgConn->>PgDB: Execute SQL Migrations
    DB-->>SyncPgConn: Migration Result
    SyncPgConn-->>-BlkTask: Migration Result
    BlkTask-->>-PgMigrations: Task Result
    PgMigrations-->>-AppLogic: QueryResult
Loading

File-Level Changes

Change Details Files
Switch Postgres to AsyncPgConnection and refactor its migration runner
  • Change DbConnection type to AsyncPgConnection under the postgres feature
  • Replace synchronous SyncConnectionWrapper PgConnection with AsyncPgConnection
  • Implement run_migrations(database_url) for Postgres by spawning a blocking PgConnection to run migrations
src/db.rs
Introduce apply_migrations helper to unify migration logic
  • Add apply_migrations in test-util to dispatch run_migrations based on backend
  • Replace direct run_migrations calls in tests and with_db with apply_migrations
  • Pass only database URL for Postgres migrations
test-util/src/lib.rs
tests/news_categories.rs
Deduplicate socket framing and response parsing in tests
  • Extract TCP handshake, request framing, and response decoding into list_categories helper
  • Replace repetitive inline socket code in all news_categories tests with calls to list_categories
tests/news_categories.rs
Update CLI and setup routines to use new migration functions
  • Use run_migrations(&database_url) for Postgres and run_migrations(&mut conn) for SQLite in main.rs
  • Guard calls with cfg(feature) to select correct signature
src/main.rs
Document migration approach for diesel_async
  • Add note explaining use of diesel_async for queries and the need to spawn a blocking PgConnection for migrations
  • Clarify difference in MigrationHarness usage between SQLite and Postgres
docs/supporting-both-sqlite3-and-postgresql-in-diesel.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 12, 2025

Walkthrough

This update refactors database migration handling to provide backend-specific logic for SQLite and PostgreSQL, including changes to function signatures, type aliases, and conditional compilation. Test utilities and documentation are updated to clarify and abstract migration application, and test code is refactored to reduce duplication via helper functions.

Changes

File(s) Change Summary
src/db.rs Simplified run_migrations for Postgres by removing unused parameter; introduced async apply_migrations to unify migration calls across backends with conditional compilation; updated test compilation and error mapping.
src/main.rs Replaced calls to run_migrations with apply_migrations, passing both mutable connection and database URL; updated imports accordingly.
test-util/src/lib.rs Replaced run_migrations with apply_migrations in with_db function; removed conditional compilation blocks for migrations; minor formatting cleanup.
tests/news_categories.rs Added list_categories helper to centralise TCP connection and request handling; replaced direct migration calls with apply_migrations; removed duplicated setup code.
docs/supporting-both-sqlite3-and-postgresql-in-diesel.md Added explanation about using diesel_async for queries and synchronous migration handling via blocking tasks and wrappers for PostgreSQL and SQLite respectively.

Sequence Diagram(s)

sequenceDiagram
    participant App as Application
    participant DB as Database (SQLite/Postgres)
    participant Diesel as diesel_async/diesel
    participant OS as OS Thread/Blocking Task

    App->>DB: Request to run migrations
    alt PostgreSQL
        App->>OS: Spawn blocking task
        OS->>Diesel: Establish synchronous PgConnection
        Diesel->>DB: Run migrations synchronously
        OS->>App: Return migration result
    else SQLite
        App->>Diesel: Use SyncConnectionWrapper
        Diesel->>DB: Run migrations via wrapper
        Diesel->>App: Return migration result
    end
Loading

Possibly related PRs

Suggested reviewers

  • codescene-delta-analysis

Poem

In burrows deep, migrations run,
SQLite and Postgres—now both are fun!
Async or sync, we bridge the divide,
With helpers and wrappers, side by side.
Refactored tests, clean and bright—
The codebase hops along, just right!
🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 38be326 and 26a7e05.

📒 Files selected for processing (4)
  • src/db.rs (5 hunks)
  • src/main.rs (3 hunks)
  • test-util/src/lib.rs (2 hunks)
  • tests/news_categories.rs (10 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • test-util/src/lib.rs
  • src/main.rs
  • src/db.rs
  • tests/news_categories.rs
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: coverage
✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Health Improved (1 files improve in Code Health)

Gates Failed
Prevent hotspot decline (1 hotspot with String Heavy Function Arguments)
Enforce advisory code health rules (1 file with String Heavy Function Arguments)

Gates Passed
4 Quality Gates Passed

See analysis details in CodeScene

Reason for failure
Prevent hotspot decline Violations Code Health Impact
lib.rs 1 rule in this hotspot 10.00 → 9.69 Suppress
Enforce advisory code health rules Violations Code Health Impact
lib.rs 1 advisory rule 10.00 → 9.69 Suppress
View Improvements
File Code Health Impact Categories Improved
news_categories.rs 8.65 → 10.00 Large Method

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
src/main.rs (2)

95-103: Consider re-using the new apply_migrations helper to avoid duplicated cfg-gated calls.

apply_migrations already hides the backend differences; wiring it into the runtime code (not only tests) would remove these duplicated #[cfg] blocks and keep the public migration API in one place.


195-199: Same duplication here – could be collapsed via apply_migrations.

Applying the helper in setup_database would leave just:

apply_migrations(&mut conn, database).await?;

and the cfg branches disappear.

tests/news_categories.rs (1)

16-59: Add socket timeouts to prevent hanging tests.

If the server fails to reply, the read_exact calls will block indefinitely, leaving the test stuck. Setting small read/write timeouts on the TcpStream (or wrapping the I/O in std::io::Read/Write timeouts) makes the failure mode explicit and speeds up CI diagnostics.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bdf14d6 and 97bd973.

📒 Files selected for processing (5)
  • docs/supporting-both-sqlite3-and-postgresql-in-diesel.md (1 hunks)
  • src/db.rs (7 hunks)
  • src/main.rs (2 hunks)
  • test-util/src/lib.rs (3 hunks)
  • tests/news_categories.rs (9 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
tests/news_categories.rs (3)
src/db.rs (3)
  • create_bundle (306-315)
  • create_category (290-299)
  • conn (411-411)
src/transaction.rs (6)
  • encode_params (468-486)
  • decode_params (408-443)
  • from_bytes (89-99)
  • new (117-130)
  • new (292-298)
  • new (356-363)
test-util/src/lib.rs (4)
  • apply_migrations (207-212)
  • handshake (177-194)
  • port (144-146)
  • start_with_setup (107-141)
🔇 Additional comments (2)
docs/supporting-both-sqlite3-and-postgresql-in-diesel.md (1)

193-197: Documentation note looks accurate – nothing to amend.

The added clarification about spawning a blocking task for Postgres migrations is consistent with the implementation in src/db.rs.

src/db.rs (1)

67-85: Postgres migration path looks correct – no issues spotted.

spawn_blocking correctly propagates synchronous Diesel errors back to the async context; the returned QueryResult<()> is forwarded as the function result.

Comment thread test-util/src/lib.rs Outdated
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @leynos - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Health Improved (1 files improve in Code Health)

Gates Passed
6 Quality Gates Passed

See analysis details in CodeScene

View Improvements
File Code Health Impact Categories Improved
news_categories.rs 8.65 → 10.00 Large Method

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Health Improved (1 files improve in Code Health)

Gates Passed
6 Quality Gates Passed

See analysis details in CodeScene

View Improvements
File Code Health Impact Categories Improved
news_categories.rs 8.61 → 10.00 Large Method

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

@leynos leynos merged commit cfcb378 into main Jun 12, 2025
2 of 5 checks passed
@leynos leynos deleted the codex/update-database-backend-to-support-postgresql branch June 12, 2025 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant