Skip to content

Enable Postgres in tests#92

Merged
leynos merged 4 commits intomainfrom
codex/gate-diesel-dependencies-in-test-util
Jun 11, 2025
Merged

Enable Postgres in tests#92
leynos merged 4 commits intomainfrom
codex/gate-diesel-dependencies-in-test-util

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Jun 11, 2025

Summary

  • gate test-util Diesel deps behind features
  • support Postgres in TestServer
  • update integration tests to use connection URLs
  • forward main crate features to test-util

Testing

  • cargo clippy -- -D warnings
  • cargo test --no-run
  • cargo test

https://chatgpt.com/codex/tasks/task_e_6849897e63488322afad1078d5a80215

Summary by Sourcery

Enable optional PostgreSQL in test utilities by feature-gating Diesel dependencies, using database URLs in TestServer, and aligning feature flags across crates

New Features:

  • Add optional PostgreSQL support in TestServer behind a new postgres feature

Enhancements:

  • Switch TestServer and test-util setup functions to use database URL strings instead of file paths
  • Gate Diesel and Diesel-async dependencies in test-util behind separate sqlite and postgres features
  • Propagate primary crate feature flags (sqlite and postgres) to test-util for consistent configuration

Build:

  • Revise Cargo.toml in test-util, main, and validator crates to disable default features and define sqlite and postgres features for dependency gating

Tests:

  • Update integration tests to use db_url() and establish connections via URL strings

Summary by CodeRabbit

  • New Features

    • Added support for PostgreSQL in test utilities, enabling tests to run against both SQLite and PostgreSQL databases.
  • Refactor

    • Unified database handling in tests to use database URLs instead of file paths.
    • Updated test server and utility APIs to reflect the new database URL approach.
  • Chores

    • Updated dependency configurations for improved feature management and compatibility across crates.
    • Adjusted development dependencies to better control enabled features.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Failed
Prevent hotspot decline (1 hotspot with Large Method)
Enforce advisory code health rules (1 file with Large Method)

Gates Passed
4 Quality Gates Passed

See analysis details in CodeScene

Reason for failure
Prevent hotspot decline Violations Code Health Impact
lib.rs 1 rule in this hotspot 10.00 → 9.56 Suppress
Enforce advisory code health rules Violations Code Health Impact
lib.rs 1 advisory rule 10.00 → 9.56 Suppress

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 11, 2025

Warning

Rate limit exceeded

@leynos has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 13 minutes and 17 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 864e3f8 and 7020f23.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (1)
  • test-util/Cargo.toml (1 hunks)

Walkthrough

The changes introduce optional PostgreSQL support alongside SQLite for testing utilities, refactor database setup and server launch logic, and update dependency features in several Cargo.toml files. The test utility API now uses database URLs instead of file paths, and related tests are updated accordingly. No changes to exported or public entities in production code.

Changes

File(s) Summary
Cargo.toml, validator/Cargo.toml Updated test-util and mxd dependencies: added feature gating, disabled default features, and adjusted feature sets for test/dev dependencies.
test-util/Cargo.toml Refactored dependencies and features: added optional PostgreSQL support, reorganised Diesel features, and structured feature groups.
test-util/src/lib.rs Refactored TestServer: unified database URL handling, added PostgreSQL support, split setup logic, and updated public API signatures.
tests/news_articles.rs, tests/news_categories.rs Updated tests to use database URLs instead of file paths when establishing database connections.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Test Code
    participant TestServer as TestServer
    participant DB as Database (SQLite/PostgreSQL)
    participant Server as Application Server

    Test->>TestServer: start_with_setup(manifest_path, setup_fn)
    alt Feature: sqlite
        TestServer->>DB: setup_sqlite()
        DB-->>TestServer: sqlite db_url
    else Feature: postgres
        TestServer->>DB: setup_postgres()
        DB-->>TestServer: postgres db_url
    end
    TestServer->>Server: build_server_command(db_url, port)
    TestServer->>Server: Start server process
    TestServer->>Server: wait_for_server()
    Server-->>TestServer: Ready (listening)
    TestServer-->>Test: TestServer instance with db_url
    Test->>DB: Establish connection using db_url
Loading

Poem

A bunny hops with code anew,
Now Postgres joins the test-bed crew!
URLs replace the paths of old,
Features toggled, dependencies bold.
With every hop, the tests expand—
Both SQLite and Postgres at hand!
🐇✨


🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Passed
6 Quality Gates Passed

See analysis details in CodeScene

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Passed
6 Quality Gates Passed

See analysis details in CodeScene

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (1)
Cargo.toml (1)

39-45: ⚠️ Potential issue

test-util/* cannot be referenced from a feature list while it is only a dev-dependency

test-util is declared exclusively under [dev-dependencies], yet the feature definitions attempt to enable
"test-util/postgres" and "test-util/sqlite".
Cargo will fail when downstream crates enable the postgres or sqlite feature because dev-dependencies are not built for normal compilation units.

Recommended fix:

  1. Promote test-util to an optional normal dependency:
[dependencies]
+test-util = { path = "test-util", optional = true, default-features = false }
  1. Remove the test-util entry from [dev-dependencies] (or keep it without duplication).

This keeps the crate buildable for external consumers while still compiling test-util for tests.

Also applies to: 46-52

♻️ Duplicate comments (1)
test-util/src/lib.rs (1)

107-141: ⚠️ Potential issue

Fix compilation errors with feature combinations.

The current implementation has several issues:

  1. When both sqlite and postgres features are enabled, db_url is defined twice
  2. The pg variable is used unconditionally on line 139 but only defined when postgres feature is enabled
  3. The variable shadowing pattern on lines 122-123 is confusing

Additionally, this method remains above the complexity threshold (79 lines vs 70 threshold).

 pub fn start_with_setup<F>(
     manifest_path: &str,
     setup: F,
 ) -> Result<Self, Box<dyn std::error::Error>>
 where
     F: FnOnce(&str) -> Result<(), Box<dyn std::error::Error>>,
 {
     let temp = TempDir::new()?;
 
-    #[cfg(feature = "sqlite")]
-    let db_url = setup_sqlite(&temp, setup)?;
-
-    #[cfg(feature = "postgres")]
-    let (db_url, mut pg) = setup_postgres(setup)?;
-
-    #[cfg(feature = "postgres")]
-    let db_url = db_url;
+    #[cfg(all(feature = "sqlite", not(feature = "postgres")))]
+    let (db_url, pg) = {
+        let url = setup_sqlite(&temp, setup)?;
+        (url, None)
+    };
+
+    #[cfg(feature = "postgres")]
+    let (db_url, pg) = {
+        let (url, postgres) = setup_postgres(setup)?;
+        (url, Some(postgres))
+    };
+
+    #[cfg(not(any(feature = "sqlite", feature = "postgres")))]
+    compile_error!("Either 'sqlite' or 'postgres' feature must be enabled");
 
     let socket = TcpListener::bind("127.0.0.1:0")?;
     let port = socket.local_addr()?.port();
     drop(socket);
 
     let mut child = build_server_command(manifest_path, port, &db_url).spawn()?;
 
     wait_for_server(&mut child)?;
 
     Ok(Self {
         child,
         port,
         db_url,
         _temp: temp,
-        #[cfg(feature = "postgres")]
-        pg: Some(pg),
+        #[cfg(feature = "postgres")]
+        pg,
+        #[cfg(not(feature = "postgres"))]
+        _pg: std::marker::PhantomData,
     })
 }

Also update the struct definition:

 pub struct TestServer {
     child: Child,
     port: u16,
     db_url: String,
     _temp: TempDir,
     #[cfg(feature = "postgres")]
     pg: Option<PostgreSQL>,
+    #[cfg(not(feature = "postgres"))]
+    _pg: std::marker::PhantomData<()>,
 }
🧹 Nitpick comments (3)
tests/news_categories.rs (1)

19-25: Minor: favour the existing with_db helper to avoid spinning up ad-hoc runtimes

Repeatedly creating a new tokio::Runtime inside each test adds a few hundred ms per test and obscures intent.
The test_util::with_db helper (already used elsewhere) keeps the style uniform and avoids boilerplate.

No action is required for correctness, but consider refactoring:

TestServer::start_with_setup("./Cargo.toml", |db| {
    with_db(db, |conn| {
        Box::pin(async move {
            run_migrations(conn).await?;
            // …
        })
    })
})

Also applies to: 107-112, 195-200, 247-253, 313-318

test-util/src/lib.rs (2)

28-36: Remove redundant path-to-string conversions.

The function converts the path to string twice with different error handling methods.

 fn setup_sqlite<F>(temp: &TempDir, setup: F) -> Result<String, Box<dyn std::error::Error>>
 where
     F: FnOnce(&str) -> Result<(), Box<dyn std::error::Error>>,
 {
     let path = temp.path().join("mxd.db");
-    setup(path.to_str().expect("db path utf8"))?;
-    Ok(path.to_str().unwrap().to_owned())
+    let db_path = path.to_str().ok_or("db path is not valid UTF-8")?;
+    setup(db_path)?;
+    Ok(db_path.to_owned())
 }

165-169: Consider the runtime creation overhead in Drop.

Creating a new Tokio runtime in the Drop implementation adds overhead and could potentially cause issues in some contexts. However, this may be acceptable for test utilities.

Consider documenting this behaviour or exploring alternatives such as:

  • Using a shared runtime for PostgreSQL operations
  • Making the cleanup synchronous if possible
  • Documenting that this struct should not be used in async contexts where runtime creation could be problematic
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3a5ca34 and 864e3f8.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • Cargo.toml (1 hunks)
  • test-util/Cargo.toml (1 hunks)
  • test-util/src/lib.rs (7 hunks)
  • tests/news_articles.rs (1 hunks)
  • tests/news_categories.rs (5 hunks)
  • validator/Cargo.toml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: coverage
🔇 Additional comments (6)
tests/news_articles.rs (1)

256-259: Possible String/&str mismatch – double-check db_url() return type

DbConnection::establish expects a &str.
If TestServer::db_url() returns a String, passing it by value will not auto-coerce and the line will fail to compile; you’d need &server.db_url() (or return &str from db_url()).

-        let mut conn = DbConnection::establish(server.db_url()).await?;
+        let mut conn = DbConnection::establish(&server.db_url()).await?;

Please run cargo check to verify the actual signature.

validator/Cargo.toml (1)

12-15: LGTM – aligns mxd usage with top-level feature gating
No issues spotted.

test-util/src/lib.rs (4)

8-26: LGTM! Clean feature-gated PostgreSQL support.

The struct refactoring to use db_url instead of db_path properly generalises the database connection handling for both SQLite and PostgreSQL.


56-98: Excellent refactoring into helper functions.

The extraction of wait_for_server and build_server_command improves code organisation and readability whilst maintaining the original functionality.


148-151: Clean API migration to database URLs.

The consistent change from file paths to database URLs across all public APIs properly supports both SQLite and PostgreSQL connection strings.

Also applies to: 206-206, 221-221, 270-270


38-54:

Details

✅ Verification successful

Consider the implications of creating a new runtime.

Creating a new Tokio runtime with block_on could panic if this function is called from within an existing async runtime. However, this may be acceptable for test utilities that are typically called from synchronous test contexts.


🏁 Script executed:

#!/bin/bash
# Check if setup_postgres is ever called from async contexts
rg -A 5 'setup_postgres|start_with_setup' --type rust

Length of output: 5695


No runtime conflicts detected
All invocations of setup_postgres (via TestServer::start_with_setup) occur in synchronous test functions, so the dedicated Tokio runtime and ​block_on​ call will never run inside an existing async context. No changes are required.

Comment thread Cargo.toml
Comment on lines 58 to 60
[dev-dependencies]
test-util = { path = "test-util" }
test-util = { path = "test-util", default-features = false }
postgresql_embedded = { version = "0.18.5", features = ["tokio"] }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Duplicate declaration after promoting test-util

After the change above, remember to delete the current [dev-dependencies] entry to avoid version/feature mismatches.

-[dev-dependencies]
-test-util = { path = "test-util", default-features = false }
+## entry removed – now in [dependencies] as optional

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In Cargo.toml around lines 58 to 60, there is a duplicate declaration of the
[dev-dependencies] section after promoting the test-util dependency. To fix
this, remove the existing [dev-dependencies] block that contains the test-util
and postgresql_embedded entries to avoid conflicts or mismatches in versions and
features.

Comment thread test-util/Cargo.toml
Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Passed
6 Quality Gates Passed

See analysis details in CodeScene

Quality Gate Profile: Pay Down Tech Debt
Want more control? Customize Code Health rules or catch issues early with our IDE extension and CLI tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant