Skip to content

Refactor BDD harness to per-scenario TestWorld with rstest-bdd#251

Merged
leynos merged 46 commits intomainfrom
terragon/migrate-netsuke-to-rstest-bdd-lt13bh
Dec 29, 2025
Merged

Refactor BDD harness to per-scenario TestWorld with rstest-bdd#251
leynos merged 46 commits intomainfrom
terragon/migrate-netsuke-to-rstest-bdd-lt13bh

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Dec 21, 2025

Summary

  • Refactors the BDD harness to a per-scenario TestWorld, aligned with rstest-bdd 0.3.2, and expands the dedicated tests/bdd suite.
  • Introduces centralized fixtures, helpers, and domain types under tests/bdd (fixtures, helpers, types.rs) and wires them into the step modules (CLI, manifest, IR, ninja, process, stdlib, etc.).
  • Moves scaffolding utilities under tests/bdd/fixtures and switches step signatures to consume shared world helpers/fixtures.
  • Extends the harness with modular step definitions and a centralized TestWorld-based lifecycle for per-scenario isolation.
  • Updates CI and toolchain handling to improve determinism (single-threaded execution by default, nightly toolchain support) and wires it into the workflow.
  • Documentation and references updated to reflect the shared harness approach and per-scenario fixtures (adr-002 and rstest-bdd usage guide).
  • Task references preserved for traceability.

Changes

  • Shared BDD harness and world lifecycle

    • Added tests/bdd/fixtures/mod.rs providing a per-scenario TestWorld and utilities for building and tearing down world state; implemented Drop semantics for environment restoration and HTTP server lifecycle management.
    • Added world fixture registration (world) to provide a fresh TestWorld per scenario.
    • Added RefCell-based helper trait (RefCellOptionExt) to simplify Option-wrapped fixtures inside steps.
  • Refactored BDD steps to consume shared helpers

    • Reworked key BDD step modules (cli.rs, ir.rs, manifest.rs, manifest_command.rs, ninja.rs, process.rs, stdlib/*) to rely on TestWorld and shared fixtures rather than bespoke scaffolding.
    • Introduced or updated steps to use shared types/utilities from tests/bdd/fixtures, reducing duplication and improving consistency across domains (CLI, manifest, IR, ninja, stdlib, filesystem, etc.).
  • Project structure and fixtures

    • Added tests/bdd/fixtures for a centralized world and related utilities to support per-scenario isolation and easier extension in future phases.
    • Added tests/bdd/steps/mod.rs and tests/bdd/steps/cli.rs, tests/bdd/steps/ir.rs, tests/bdd/steps/manifest/mod.rs, tests/bdd/steps/manifest/targets.rs, tests/bdd/steps/manifest_command.rs, tests/bdd/steps/ninja.rs, tests/bdd/steps/process.rs, tests/bdd/steps/stdlib/* modules to align with shared harness usage.
    • Added tests/bdd/helpers/{mod.rs, assertions.rs, parse_store.rs} and tests/bdd/mod.rs to wire up shared helpers and fixtures.
    • Introduced tests/bdd/types.rs to define domain types used across steps.
    • Added tests/bdd/tests and wiring to enable a dedicated BDD suite; migrated relevant fs, manifest, CLI, IR, ninja, process, stdlib steps into the new layout.
  • CI, toolchains, and test execution

    • Updated CI workflow to run with a matrix that includes nightly toolchain and sets RUSTUP_TOOLCHAIN to the selected toolchain; CI is configured to default to single-threaded test execution for environment isolation.
    • Updated Makefile/CI references to reflect single-threaded execution by default.
  • Documentation and references

    • Docs ADR (adr-002) and rstest-bdd user guide updated to reflect the shared TestWorld harness and per-scenario fixtures.
    • Documentation references updated to reflect migration toward rstest-bdd with per-scenario fixtures.
  • Compatibility and breaking changes

    • The migration moves from per-module scaffolding to a shared TestWorld-based harness; some step signatures and access patterns have been updated to consume fixtures from the shared world.
  • Notes for maintainers

    • The shared world pattern simplifies adding new steps and domains; future phases will extend TestWorld with domain-specific fixtures as needed.

Task

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Dec 21, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Summarise the migration from cucumber to rstest-bdd, add TestWorld and many BDD step modules, introduce PATH override plumbing for stdlib which lookups, refactor manifest target expansion to evaluate static when clauses, update CI toolchain matrix and test deps, and route tracing logs to stderr. (48 words)

Changes

Cohort / File(s) Summary
CI / Build
\.github/workflows/ci.yml, Cargo.toml
Add nightly matrix include and set RUSTUP_TOOLCHAIN from matrix; mark experimental nightly as continue-on-error; replace cucumber dev-dep with rstest-bdd/rstest-bdd-macros and add predicates.
Docs / ADR / User guide
docs/adr-002-replace-cucumber-with-rstest-bdd.md, docs/rstest-bdd-users-guide.md
Record decision to adopt rstest-bdd v0.3.2, migration notes, bump MSRV to Rust 1.89.0 and edition 2024; document rstest-bdd server/CLI changes and ScenarioMetadata API additions.
Logging
src/main.rs, tests/logging_stderr_tests.rs
Configure tracing subscriber to write to stderr via io::stderr; add integration test asserting stderr logging.
Manifest expansion
src/manifest/expand.rs
Change expand_target to take mut map; add eval_when to evaluate static when (template/Jinja or expr); update when_allows to accept optional iteration context; add tests for static when behaviour.
Stdlib PATH override plumbing
src/stdlib/config.rs, src/stdlib/register.rs, src/stdlib/which/mod.rs, src/stdlib/which/cache.rs, src/stdlib/which/env.rs, src/stdlib/which/lookup/tests.rs
Add path_override to StdlibConfig/WhichConfig/WhichResolver; thread override into resolver; extend EnvSnapshot::capture to accept a path override; update tests to pass explicit PATH values instead of mutating process env.
Which resolver internals
src/stdlib/which/cache.rs, src/stdlib/which/env.rs
Add path_override: Option<OsString> field, accept override in constructors, prefer override over environment PATH when resolving; update signatures accordingly.
Registration wiring
src/stdlib/register.rs, src/stdlib/which/mod.rs
Update WhichConfig::new signature to include path_override; forward config.path_override into WhichResolver::new at registration sites.
BDD test infra — entry, fixtures, helpers, types
tests/bdd_tests.rs, tests/bdd/mod.rs, tests/bdd/fixtures/mod.rs, tests/bdd/helpers/*, tests/bdd/types.rs
Add TestWorld fixture, helpers (assertions, parse_store), many typed newtypes for BDD parameters, register scenarios via scenarios!, and wire TestWorld into generated tests.
BDD step implementations (new)
tests/bdd/steps/mod.rs, tests/bdd/steps/cli.rs, tests/bdd/steps/fs.rs, tests/bdd/steps/ir.rs, tests/bdd/steps/manifest/*, tests/bdd/steps/manifest_command.rs, tests/bdd/steps/ninja.rs, tests/bdd/steps/process.rs, tests/bdd/steps/stdlib/*
Add comprehensive rstest-bdd step modules for CLI, FS, IR, manifest parsing/targets, manifest_command, ninja, process, and extensive stdlib step families (assertions, config, parsing, policy, rendering, types, workspace).
Cucumber test harness removed
tests/cucumber.rs, tests/steps/*, tests/steps/stdlib_steps/*
Remove old Cucumber runner, CliWorld, and all cucumber-based step modules and helper types (many test files deleted).
Test features & data
tests/data/jinja_is.yml, tests/data/jinja_is_missing.yml, tests/features/*, tests/features_unix/*
Promote nested recipe.command to top-level command in manifest fixtures; add missing-path manifest; update feature wording to match rstest-bdd phrases; convert some Scenario Outlines to concrete Scenarios.
BDD helpers & manifest assertions
tests/bdd/helpers/*, tests/bdd/steps/manifest/helpers.rs
Add assertion helpers (assert_optional_contains, assert_slot_contains), parse outcome store helper, and typed manifest assertion utilities with contextual error messages.
Misc test refactors
tests/std_filter_tests/which_filter_common.rs, various tests/steps/* removals
Add contextual error on directory creation; update tests to use explicit PATH overrides and the new TestWorld API; remove many legacy cucumber step files replaced by rstest-bdd equivalents.

Sequence Diagram(s)

sequenceDiagram
    %% Styling for clarity
    rect rgb(240,248,255)
    participant Runner as Test Runner / Scenario
    participant World as TestWorld
    participant Stdlib as Stdlib (StdlibConfig)
    participant Which as WhichResolver
    participant Env as EnvSnapshot
    participant FS as Filesystem / PATH
    end

    Runner->>World: instantiate TestWorld
    Runner->>Stdlib: request render (template + optional path)
    Stdlib->>Stdlib: build StdlibConfig (includes path_override)
    Stdlib->>Which: create WhichConfig(path_override)
    Which->>Which: WhichResolver::new(path_override)
    Stdlib->>Which: resolve helper command
    Which->>Env: EnvSnapshot::capture(path_override?)
    Env->>FS: read PATH (prefer path_override)
    FS-->>Env: PATH entries
    Env-->>Which: resolved path entries
    Which-->>Stdlib: resolved executable path or not found
    Stdlib-->>World: store render output or error
    World-->>Runner: assertion/verification step
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~55 minutes

Possibly related issues

Possibly related PRs

Poem

✨ Swap the steps, let fixtures sing,
TestWorld rises, rstest-bdd takes wing.
PATHs diverted, templates learn to say,
Nightly flags blink in CI’s array.
Logs to stderr — ready for the day.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check ✅ Passed The title accurately summarises the main change: refactoring the BDD harness to use per-scenario TestWorld with rstest-bdd.
Description check ✅ Passed The description provides comprehensive detail on the refactoring scope, touching on fixtures, helpers, types, step modules, CI/toolchain updates, and documentation changes—all directly related to the changeset.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch terragon/migrate-netsuke-to-rstest-bdd-lt13bh

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/cli.rs

Comment on file

//! Step definitions for CLI parsing scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 93.3% of all arguments to its 23 functions are strings. The threshold for string arguments is 39.0%

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on file

//! Step definitions for manifest parsing scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 58.7% of all arguments to its 40 functions are strings. The threshold for string arguments is 39.0%

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/stdlib/rendering.rs

Comment on file

//! Render-related stdlib step helpers used by BDD scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 84.6% of all arguments to its 9 functions are strings. The threshold for string arguments is 39.0%

@coderabbitai

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

@leynos leynos changed the title Plan migration from cucumber to rstest-bdd Migrate tests from cucumber to rstest-bdd Dec 21, 2025
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos leynos changed the title Migrate tests from cucumber to rstest-bdd Plan migration from cucumber to rstest-bdd Dec 21, 2025
@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on lines +619 to +643

fn target_rule_is(index: usize, rule: String) -> Result<()> {
    let rule = RuleName::new(rule);
    ensure!(index > 0, "target index is 1-based");
    with_world(|world| {
        let result = world.manifest.with_ref(|m| {
            let target = m
                .targets
                .get(index - 1)
                .with_context(|| format!("missing target {index}"))?;
            match &target.recipe {
                Recipe::Rule { rule: actual } => {
                    let actual_str = get_string_from_string_or_list(actual, "rule")?;
                    ensure!(
                        actual_str == rule.as_str(),
                        "expected target {index} rule '{}', got '{actual_str}'",
                        rule
                    );
                    Ok(())
                }
                other => bail!("Expected rule recipe, got: {other:?}"),
            }
        });
        result.context("manifest has not been parsed")?
    })
}

❌ New issue: Code Duplication
The module contains 14 functions with similar structure: first_target_name,manifest_has_macros,manifest_has_targets,target_command_n and 10 more functions

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on file

//! Step definitions for manifest parsing scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 56.5% of all arguments to its 40 functions are strings. The threshold for string arguments is 39.0%

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 21, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/stdlib/rendering.rs

Comment on file

//! Render-related stdlib step helpers used by BDD scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 42.1% of all arguments to its 15 functions are strings. The threshold for string arguments is 39.0%

@coderabbitai

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 23, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on file

//! Step definitions for manifest parsing scenarios.

❌ New issue: String Heavy Function Arguments
In this module, 39.2% of all arguments to its 52 functions are strings. The threshold for string arguments is 39.0%

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 23, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on lines +80 to +87

fn assert_target_name_eq(target_index: usize, actual: &str, expected: &TargetName) -> Result<()> {
    ensure!(
        actual == expected.as_str(),
        "expected target {target_index} name '{}', got '{actual}'",
        expected
    );
    Ok(())
}

❌ New issue: Code Duplication
The module contains 5 functions with similar structure: assert_macro_signature_eq,assert_target_command_eq,assert_target_name_eq,assert_target_rule_eq and 1 more functions

@coderabbitai

This comment was marked as resolved.

@coderabbitai

This comment was marked as resolved.

@leynos leynos changed the title Plan migration from cucumber to rstest-bdd Migrate tests from cucumber to rstest-bdd with new harness Dec 23, 2025
codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 23, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest.rs

Comment on lines +82 to +101

fn assert_string_eq<T>(
    context: &str,
    index: Option<usize>,
    field: &str,
    actual: &str,
    expected: &T,
) -> Result<()>
where
    T: AsRef<str> + std::fmt::Display,
{
    let location = match index {
        Some(idx) => format!("{context} {idx}"),
        None => context.to_string(),
    };
    ensure!(
        actual == expected.as_ref(),
        "expected {location} {field} '{expected}', got '{actual}'"
    );
    Ok(())
}

❌ New issue: Excess Number of Function Arguments
assert_string_eq has 5 arguments, max arguments = 4

@coderabbitai

This comment was marked as resolved.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos leynos changed the title Migrate tests from cucumber to rstest-bdd with new harness Migrate tests to rstest-bdd with per-scenario TestWorld harness Dec 23, 2025
@leynos leynos changed the title Migrate BDD harness to per-scenario TestWorld with rstest-bdd Refactor BDD steps into shared TestWorld helpers with rstest-bdd Dec 29, 2025
Add function-level #[expect(...)] attributes in BDD test step modules to suppress clippy lints caused by rstest-bdd macro-generated code. This cleans up file-wide blanket allows, making lint management more targeted and maintainable.

Also apply minor code cleanups for style and clarity across BDD steps, such as using method references, trimming strings before processing, passing references, and simplifying code formatting.

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 29, 2025

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Dec 29, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/bdd/steps/stdlib/parsing.rs (1)

7-10: Add doc comments to public functions.

Both parse_iso_timestamp and parse_expected_offset are pub(crate) but lack /// doc comments, whilst the private helpers (parse_sign, parse_optional_component) are documented. Document the public API for cargo doc generation and maintainability.

🔎 Proposed doc comments
+/// Parse an ISO8601 timestamp string into an `OffsetDateTime`.
+///
+/// # Errors
+///
+/// Returns an error if the input does not conform to ISO8601 format.
 pub(crate) fn parse_iso_timestamp(raw: &str) -> Result<OffsetDateTime> {
     OffsetDateTime::parse(raw, &Iso8601::DEFAULT)
         .with_context(|| format!("parse ISO8601 timestamp from '{raw}'"))
 }
+/// Parse a UTC offset string such as `+00:00`, `-05:30`, or `Z`.
+///
+/// Supports hours, optional minutes, and optional seconds components.
+/// The special case-insensitive token `"Z"` is recognized as UTC.
+///
+/// # Errors
+///
+/// Returns an error if the format is invalid or components are out of range.
 pub(crate) fn parse_expected_offset(raw: &str) -> Result<UtcOffset> {
     let trimmed = raw.trim();

Also applies to: 37-63

📜 Review details

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 087aeaf and eee965b.

📒 Files selected for processing (15)
  • tests/bdd/steps/cli.rs
  • tests/bdd/steps/ir.rs
  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/manifest_command.rs
  • tests/bdd/steps/mod.rs
  • tests/bdd/steps/ninja.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/stdlib/assertions.rs
  • tests/bdd/steps/stdlib/parsing.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/stdlib/rendering.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/types.rs
  • tests/logging_stderr_tests.rs
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed in Rust code.
Fix any warnings emitted during Rust tests in the code itself rather than silencing them.
In Rust, extract meaningfully named helper functions when a function is too long, adhering to separation of concerns and CQRS.
In Rust, group related parameters in meaningfully named structs when a function has too many parameters.
In Rust, consider using Arc to reduce data returned when a function is returning a large error.
Every Rust module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs in Rust using Rustdoc comments (///) so documentation can be generated with cargo doc.
In Rust, prefer immutable data and avoid unnecessary mut bindings.
In Rust, handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code in Rust unless absolutely necessary, and document any usage clearly.
In Rust, place function attributes after doc comments.
In Rust, do not use return in single-line functions.
In Rust, use predicate functions for conditional criteria with more than two branches.
Lints in Rust must not be silenced except as a last resort.
Lint rule suppressions in Rust must be tightly scoped and include a clear reason.
In Rust, prefer expect over allow.
In Rust tests, use rstest fixtures for shared setup.
In Rust tests, replace duplicated tests with #[rstest(...)] parameterized cases.
In Rust, prefer mockall for mocks and stubs.
In Rust, use concat!() to combine long string literals rather than escaping newlines with a backslash.
In Rust, prefer single line versions of functions where appropriate (e.g., pub fn new(id: u64) -> Self { Self(id) } instead of multi-line).
Use NewTypes in Rust to model domain values and eliminate 'integer soup'. Reach for newt-hype when introducing many homogeneous wrappers; add small shims for string-backed wrappers. For path-centric wrappers implement AsRef<Path> along...

Files:

  • tests/logging_stderr_tests.rs
  • tests/bdd/steps/manifest_command.rs
  • tests/bdd/steps/ir.rs
  • tests/bdd/steps/mod.rs
  • tests/bdd/steps/ninja.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/stdlib/parsing.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/steps/cli.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/stdlib/rendering.rs
  • tests/bdd/types.rs
  • tests/bdd/steps/stdlib/assertions.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cognitive complexity of functions no more than 9.

  • Adhere to single responsibility and CQRS
  • Place function attributes after doc comments.
  • Do not use return in single-line functions.
  • Move conditionals with >2 branches into a predicate function.
  • Avoid unsafe unless absolutely necessary.
  • Every module must begin with a //! doc comment that explains the module's purpose and utility.
  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar
  • Lints must not be silenced except as a last resort.
    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.
  • Use rstest fixtures for shared setup and to avoid repetition between tests.
  • Replace duplicated tests with #[rstest(...)] parameterised cases.
  • Prefer mockall for mocks/stubs.
  • Prefer .expect() over .unwrap() in tests.
  • .expect() and .unwrap() are forbidden outside of tests. Errors must be propagated.
  • Ensure that any API or behavioural changes are reflected in the documentation in docs/
  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/
  • Files must not exceed 400 lines in length
    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such
    • For testing of functionality depending upon environment variables, dependency injection and...

Files:

  • tests/logging_stderr_tests.rs
  • tests/bdd/steps/manifest_command.rs
  • tests/bdd/steps/ir.rs
  • tests/bdd/steps/mod.rs
  • tests/bdd/steps/ninja.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/stdlib/parsing.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/steps/cli.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/stdlib/rendering.rs
  • tests/bdd/types.rs
  • tests/bdd/steps/stdlib/assertions.rs
🧬 Code graph analysis (5)
tests/bdd/steps/ir.rs (3)
tests/bdd/helpers/parse_store.rs (1)
  • store_parse_outcome (36-51)
src/manifest/mod.rs (1)
  • from_path (158-160)
src/ir/from_manifest.rs (1)
  • from_manifest (24-35)
tests/bdd/steps/manifest/targets.rs (1)
tests/bdd/steps/manifest/mod.rs (2)
  • get_string_from_string_or_list (41-61)
  • with_manifest_error_context (30-39)
tests/bdd/steps/manifest/mod.rs (3)
tests/bdd/helpers/parse_store.rs (1)
  • store_parse_outcome (36-51)
test_support/src/error.rs (1)
  • display_error_chain (17-27)
src/manifest/mod.rs (1)
  • from_path (158-160)
tests/bdd/steps/process.rs (5)
test_support/src/lib.rs (2)
  • ensure_manifest_exists (195-214)
  • fake_ninja (96-128)
tests/bdd/fixtures/mod.rs (1)
  • world (167-169)
test_support/src/env.rs (2)
  • prepend_dir_to_path (196-210)
  • override_ninja_env (255-266)
test_support/src/check_ninja.rs (1)
  • fake_ninja_check_build_file (82-110)
src/runner/process/mod.rs (1)
  • run_ninja (151-160)
tests/bdd/steps/stdlib/policy.rs (2)
tests/bdd/fixtures/mod.rs (1)
  • world (167-169)
src/stdlib/network/mod.rs (1)
  • policy (360-360)
🔇 Additional comments (43)
tests/logging_stderr_tests.rs (1)

1-24: LGTM! Clean integration test with proper idioms.

The test correctly verifies that error logs are written to stderr. The implementation is idiomatic, using assert_cmd and predicates effectively, and follows all applicable coding guidelines. The module and function doc comments are clear and follow en-GB conventions. Previous issues (formatting, unnecessary mut binding, manual output parsing) have been resolved.

tests/bdd/steps/stdlib/parsing.rs (2)

1-2: LGTM: module documentation is clear and accurate.

The module-level doc comment correctly describes the parsing helpers for timestamp and offset formats, and the removal of host-extraction mention aligns with the migration of those concerns to centralized fixtures.


37-63: LGTM: variable rename improves clarity.

The rename to trimmed is semantically accurate, and the logic correctly uses the trimmed value for parsing whilst preserving the original raw input in error messages for debugging context.

tests/bdd/steps/cli.rs (7)

16-23: LGTM!

The apply_cli helper cleanly composes token building, parsing with normalisation, and outcome storage. Error handling is delegated appropriately to store_parse_outcome.


62-74: LGTM!

The build_token_list function now uses shlex::split for shell-like argument handling with a silent fallback to whitespace splitting, correctly addressing the previous review feedback. The documentation clearly explains the behaviour.


91-125: LGTM!

The ExpectedCommand enum provides a clean abstraction for command verification. The #[expect] attribute is correctly scoped to the function with an accurate reason explaining why const evaluation is not possible.


137-149: LGTM!

The verify_job_count function now uses the simplified pattern that separates data retrieval from assertion, as suggested in the previous review.


276-287: LGTM!

The lint suppression is correctly scoped with a clear reason and a FIXME link for tracking. The step function properly converts the string parameter to the CliArgs newtype at the boundary before delegating to the typed helper.


293-366: LGTM!

The Then step functions follow a consistent pattern: accept string parameters from rstest-bdd, convert to semantic newtypes at the boundary, and delegate to typed verification helpers. Error propagation is clean via direct Result returns.


67-74: No action required. The BDD feature file already contains comprehensive scenarios validating the shlex::split implementation with quoted and space-containing arguments, including single quotes, double quotes, escaped spaces, and multiple quoted arguments.

tests/bdd/steps/stdlib/policy.rs (3)

1-6: LGTM: module doc and imports are well structured.

The module doc correctly uses en-GB spelling and clearly explains the module's purpose. Imports are clean and organized.


24-36: LGTM: step functions correctly propagate errors.

Both allow_scheme and allow_host follow single responsibility, return Result<()>, and use ? to propagate errors from the underlying policy methods. The duplication in the take/modify/store pattern is addressed in the earlier comment.


38-54: LGTM: lint suppression is appropriately scoped and documented.

The #[expect] attribute on default_deny is narrowly scoped, includes a clear reason, and references the upstream issue. Both default_deny and block_host correctly manage policy state.

tests/bdd/steps/process.rs (4)

41-75: Clean error handling and clear phase separation.

The past review comment has been properly addressed. The function now has three well-separated phases: extract temp path, extract CLI manifest path, resolve and update. Error messages are explicit and use proper context.

The ?? pattern on line 62 is idiomatic for unwrapping Option<Result<T, E>> and provides clear error messages for both the None and Err cases.


103-191: LGTM: Given steps follow BDD patterns with proper guards.

The Given steps provide clear, focused setup actions. Unix-specific steps are properly guarded with #[cfg(unix)], error handling is consistent throughout, and helper reuse (via install_test_ninja) reduces duplication effectively.


193-244: LGTM: When steps with justified lint suppression.

The lint suppression at lines 197–200 is properly scoped with #[expect], includes a clear reason referencing an upstream rstest-bdd issue, and the if-let-else pattern is necessary to handle the program_path lifetime correctly.

Unix-specific subcommand execution is appropriately guarded with #[cfg(unix)].


246-287: LGTM: Then steps provide clear assertions.

The assertion steps use ensure! with descriptive messages and proper error context. Fragment matching via contains() is appropriate for BDD scenario verification.

tests/bdd/steps/stdlib/rendering.rs (4)

1-12: Module structure and imports are well-organized.

The module doc comment is present, newtypes are imported from the centralized types.rs, and the separation of concerns between helpers and step definitions is clear. Previous review comments about consolidating newtypes have been addressed.


18-87: Configuration extraction and rendering helpers are well-structured.

The RenderConfig struct groups related parameters as per coding guidelines. The render_template_with_context function handles multiple optional configurations cleanly with proper error contexts. The pattern of extracting config before applying avoids borrowing issues.


114-167: ContextSource enum provides clean abstraction for context value sources.

The enum pattern with key() and get_value() methods eliminates duplication across rendering steps while maintaining type safety. The render_with_context_source helper composes these cleanly.


173-223: Step definitions follow the established pattern correctly.

Each step function:

  1. Has narrowly-scoped #[expect] with reason and FIXME link
  2. Converts String parameters to newtypes at the step boundary
  3. Delegates to typed internal helpers
  4. Returns Result<()>

This aligns with the PR objectives and coding guidelines.

tests/bdd/steps/stdlib/workspace.rs (6)

1-21: Imports and constants are well-organized.

The concat!() macro for LINES_FIXTURE follows the coding guideline to use concat!() for combining long string literals rather than escaping newlines.


26-52: ensure_workspace correctly manages workspace lifecycle.

The function:

  • Uses early return for idempotency
  • Handles cross-platform symlink differences with #[cfg] attributes
  • Stores the TempDir handle in world to prevent premature cleanup
  • Provides proper error contexts throughout

54-71: compile_and_register_helper uses generics to reduce duplication.

The generic approach allows reuse across different helper compilation functions while maintaining type safety.


163-205: Cross-platform executable handling is comprehensive.

The functions resolve_executable_path, executable_script, and mark_executable correctly handle platform differences:

  • Windows: adds .cmd extension, uses batch script syntax, skips chmod
  • Unix: uses shell script syntax, sets 0o755 permissions

265-278: Environment variable mutation correctly uses test_support::env::set_var with proper locking.

The function correctly:

  • Tracks previous values for restoration via world.track_env_var
  • Sets both HOME and USERPROFILE on Windows
  • Wraps unsafe environment access with EnvLock serialisation to prevent concurrent mutation

The test_support::env::set_var implementation acquires a shared guard before calling the underlying std::env::set_var, ensuring thread-safe mutation as required by Rust 2024.


211-263: Given step definitions follow the established pattern and require no lint suppressions.

Each step converts String parameters to newtypes at the boundary (HelperName::from(), TemplatePath::new(), FileContents::new(), PathEntries::new()), delegates to typed internal helpers, and returns Result<()> for proper error propagation.

The absence of #[expect(clippy::shadow_reuse)] is correct. Unlike @when steps in rendering.rs, @given steps do not trigger this warning. This difference stems from how the rstest-bdd macro generates wrapper code for each step type—a known issue documented in leynos/rstest-bdd#381.

tests/bdd/steps/stdlib/assertions.rs (8)

1-28: Module imports and initial assertion are correct.

The assert_stdlib_output function properly uses .get() with .context() for error handling, avoiding panics.


30-47: Helper functions correctly handle output retrieval.

The stdlib_output function properly handles both success and error cases via stdlib_output_or_error, and the as_deref() usage correctly converts Option<String> to Option<&str>.


49-81: Purity assertions correctly consolidated into assert_purity_state.

The helper function eliminates duplication between impure and pure checks, and the public step functions delegate cleanly. This addresses the past review comment.


83-110: Fetch cache assertion handles path construction correctly.

The function properly:

  • Parses the URL with error context
  • Constructs the cache path relative to workspace root
  • Uses try_exists rather than assuming existence

112-147: Path and executable assertions are implemented correctly.

The assertions properly delegate to shared helpers and provide clear error messages. The resolve_executable_path call ensures cross-platform compatibility for executable paths.


149-177: Timestamp assertions handle parsing and validation correctly.

The 5-second tolerance for UTC timestamp validation is reasonable, and offset parsing uses the dedicated parsing module functions.


179-216: File content assertions include proper validation.

The assert_stdlib_output_file_uniform function correctly validates that the expected pattern is both:

  1. A single character (line 196)
  2. A single byte (line 201)

This handles potential multi-byte UTF-8 characters appropriately, ensuring the uniform byte check is valid.


218-232: Final assertion completes the file correctly.

The assert_stdlib_output_equals_text function properly retrieves both the expected text and file content, comparing them with a clear error message.

tests/bdd/steps/mod.rs (1)

1-27: LGTM!

The module documentation is clear, and the lint suppression approach now complies with coding guidelines. The file-wide #[expect] attributes have been removed (as confirmed in past review), and function-level suppressions with FIXME tracking are appropriately documented.

tests/bdd/steps/manifest/mod.rs (1)

1-281: LGTM!

The manifest step definitions follow best practices:

  • Function-level #[expect] attributes are properly scoped with FIXME tracking
  • All step functions use Result and ? for error propagation
  • Newtypes are applied at step boundaries as intended
  • Helper functions have clear purposes and appropriate documentation
tests/bdd/steps/ninja.rs (1)

1-144: LGTM!

The Ninja step definitions demonstrate excellent practices:

  • All helpers use Result and appropriate error context
  • The shlex::split optional result is properly converted to Result (line 41)
  • Index 2 usage is now well-documented (lines 123-126) per past review
  • No lint suppressions needed, no panicking code paths
tests/bdd/steps/ir.rs (1)

1-145: LGTM!

The IR step definitions demonstrate excellent refactoring outcomes:

  • Generic assert_graph_collection_count helper eliminates duplication (lines 25-44)
  • Duplicate step handlers consolidated into single function with multiple annotations (lines 72-74), addressing past review
  • compile_manifest_impl now returns () instead of Result<()> (line 139), addressing past review
  • All error handling uses Result and ? appropriately
tests/bdd/steps/manifest_command.rs (1)

1-186: LGTM!

The manifest command step definitions demonstrate that past review feedback has been addressed:

  • The trivial check_file_exists helper was inlined (line 53 now calls path.exists() directly)
  • store_run_result ordering improved (line 87 clones stderr early for use in both branches)
  • All step functions use newtypes at boundaries and proper error propagation
  • Function-scoped #[expect] attributes are appropriately applied
tests/bdd/steps/manifest/targets.rs (3)

22-47: Helper types are well-designed.

FieldLocation provides excellent abstraction for assertion error messages, using const fn for construction and clear formatting logic. The pattern of grouping location context improves readability over passing individual parameters.


49-83: Excellent macro-based abstraction for reducing boilerplate.

The combination of generic assert_string_eq and the define_assertion! macro effectively eliminates duplication whilst maintaining type safety and clear error messages. The macro invocations generate five distinct assertion functions with minimal code.


128-142: with_target provides excellent generic abstraction.

The helper function is well-designed:

  • Generic over return type enables flexible usage
  • 1-based index validated explicitly (line 133)
  • Consistent error context chaining via with_manifest_error_context
  • Closure-based API keeps target access logic centralised

Comment thread tests/bdd/steps/cli.rs
Comment thread tests/bdd/steps/cli.rs
Comment thread tests/bdd/steps/manifest/targets.rs
Comment thread tests/bdd/steps/process.rs
Comment thread tests/bdd/steps/stdlib/policy.rs Outdated
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/types.rs
Comment thread tests/bdd/types.rs
…et access helpers

- Moved domain-specific typed assertions and target helper functions
  from `targets.rs` to a new `helpers.rs` module.
- Updated `mod.rs` to include the new `helpers` module.
- Simplified `targets.rs` by importing helper functions from `helpers`.
- This reduces duplication and improves code organization and error context
  for BDD step definitions related to manifest targets.

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
codescene-delta-analysis[bot]

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

♻️ Duplicate comments (3)
tests/bdd/steps/process.rs (2)

22-41: LGTM!

The UTF-8 validation now uses explicit to_str().ok_or_else() with a clear error message, addressing the previous review concern. The guard lifecycle management is correct.


43-77: LGTM!

The refactored implementation separates concerns clearly with explicit comments for each step. The ok_or_else(...)?? pattern at line 64 correctly handles the Option<Result<...>> return type from with_ref.

tests/bdd/steps/stdlib/policy.rs (1)

61-64: Apply the same simplifications as allow_scheme and allow_host.

The Ok(p.block_host(host)?) pattern can be simplified to just p.block_host(host), and the host parameter should be converted to a HostName newtype at the step boundary for consistency with the broader refactoring pattern.

🔎 Proposed simplification
 #[given("the stdlib network policy blocks host {host:string}")]
 pub(crate) fn block_host(world: &TestWorld, host: &str) -> Result<()> {
-    modify_policy(world, |p| Ok(p.block_host(host)?))
+    modify_policy(world, |p| p.block_host(host))
 }
📜 Review details

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between eee965b and f176535.

📒 Files selected for processing (8)
  • tests/bdd/steps/cli.rs
  • tests/bdd/steps/manifest/helpers.rs
  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/types.rs
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed in Rust code.
Fix any warnings emitted during Rust tests in the code itself rather than silencing them.
In Rust, extract meaningfully named helper functions when a function is too long, adhering to separation of concerns and CQRS.
In Rust, group related parameters in meaningfully named structs when a function has too many parameters.
In Rust, consider using Arc to reduce data returned when a function is returning a large error.
Every Rust module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs in Rust using Rustdoc comments (///) so documentation can be generated with cargo doc.
In Rust, prefer immutable data and avoid unnecessary mut bindings.
In Rust, handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code in Rust unless absolutely necessary, and document any usage clearly.
In Rust, place function attributes after doc comments.
In Rust, do not use return in single-line functions.
In Rust, use predicate functions for conditional criteria with more than two branches.
Lints in Rust must not be silenced except as a last resort.
Lint rule suppressions in Rust must be tightly scoped and include a clear reason.
In Rust, prefer expect over allow.
In Rust tests, use rstest fixtures for shared setup.
In Rust tests, replace duplicated tests with #[rstest(...)] parameterized cases.
In Rust, prefer mockall for mocks and stubs.
In Rust, use concat!() to combine long string literals rather than escaping newlines with a backslash.
In Rust, prefer single line versions of functions where appropriate (e.g., pub fn new(id: u64) -> Self { Self(id) } instead of multi-line).
Use NewTypes in Rust to model domain values and eliminate 'integer soup'. Reach for newt-hype when introducing many homogeneous wrappers; add small shims for string-backed wrappers. For path-centric wrappers implement AsRef<Path> along...

Files:

  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/steps/manifest/helpers.rs
  • tests/bdd/types.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/cli.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cognitive complexity of functions no more than 9.

  • Adhere to single responsibility and CQRS
  • Place function attributes after doc comments.
  • Do not use return in single-line functions.
  • Move conditionals with >2 branches into a predicate function.
  • Avoid unsafe unless absolutely necessary.
  • Every module must begin with a //! doc comment that explains the module's purpose and utility.
  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar
  • Lints must not be silenced except as a last resort.
    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.
  • Use rstest fixtures for shared setup and to avoid repetition between tests.
  • Replace duplicated tests with #[rstest(...)] parameterised cases.
  • Prefer mockall for mocks/stubs.
  • Prefer .expect() over .unwrap() in tests.
  • .expect() and .unwrap() are forbidden outside of tests. Errors must be propagated.
  • Ensure that any API or behavioural changes are reflected in the documentation in docs/
  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/
  • Files must not exceed 400 lines in length
    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such
    • For testing of functionality depending upon environment variables, dependency injection and...

Files:

  • tests/bdd/steps/manifest/mod.rs
  • tests/bdd/steps/process.rs
  • tests/bdd/steps/manifest/targets.rs
  • tests/bdd/steps/stdlib/workspace.rs
  • tests/bdd/steps/manifest/helpers.rs
  • tests/bdd/types.rs
  • tests/bdd/steps/stdlib/policy.rs
  • tests/bdd/steps/cli.rs
🧠 Learnings (1)
📚 Learning: 2025-08-08T22:52:32.915Z
Learnt from: leynos
Repo: leynos/netsuke PR: 82
File: tests/support/mod.rs:151-155
Timestamp: 2025-08-08T22:52:32.915Z
Learning: In Rust code, when encountering the pattern:
```rust
#[allow(unfulfilled_lint_expectations, reason = "...")]
#[expect(dead_code, reason = "...")]
```
Recommend replacing it with:
```rust
#[cfg_attr(not(any(...)), expect(dead_code, reason = "..."))]
```
This uses conditional compilation to apply the `expect` attribute only when needed, avoiding the forbidden `#[allow]` attribute while properly handling cross-crate test utility usage scenarios.

Applied to files:

  • tests/bdd/types.rs
🧬 Code graph analysis (5)
tests/bdd/steps/process.rs (5)
test_support/src/lib.rs (2)
  • ensure_manifest_exists (195-214)
  • fake_ninja (96-128)
tests/bdd/fixtures/mod.rs (1)
  • world (167-169)
test_support/src/env.rs (2)
  • prepend_dir_to_path (196-210)
  • override_ninja_env (255-266)
test_support/src/check_ninja.rs (1)
  • fake_ninja_check_build_file (82-110)
src/runner/process/mod.rs (1)
  • run_ninja (151-160)
tests/bdd/steps/manifest/helpers.rs (2)
tests/bdd/steps/manifest/mod.rs (1)
  • with_manifest_error_context (32-41)
tests/bdd/fixtures/mod.rs (1)
  • world (167-169)
tests/bdd/types.rs (2)
tests/bdd/steps/stdlib/types.rs (1)
  • as_str (18-20)
src/runner/mod.rs (1)
  • into_string (42-44)
tests/bdd/steps/stdlib/policy.rs (3)
tests/bdd/fixtures/mod.rs (1)
  • world (167-169)
src/stdlib/network/mod.rs (1)
  • policy (360-360)
src/stdlib/network/policy/mod.rs (1)
  • deny_all_hosts (226-229)
tests/bdd/steps/cli.rs (5)
tests/bdd/helpers/parse_store.rs (1)
  • store_parse_outcome (36-51)
tests/bdd/fixtures/mod.rs (3)
  • world (167-169)
  • is_some (181-181)
  • is_some (199-201)
src/cli_policy.rs (1)
  • network_policy (47-65)
tests/bdd/types.rs (6)
  • new (368-370)
  • value (373-375)
  • as_path (89-91)
  • as_path (137-139)
  • as_str (298-300)
  • as_str (344-349)
src/stdlib/network/policy/mod.rs (1)
  • evaluate (286-319)
🔍 Remote MCP

Let me search for more specific information about the PR's technical concerns and rstest-bdd capabilities.## Summary of Additional Context for PR Review

Based on my search for relevant information, here is the key context that will help with reviewing this pull request:

rstest-bdd Framework Capabilities

rstest-bdd brings BDD to Rust without requiring adoption of a bespoke runner or monolithic "world" object. The key architectural decision in this PR aligns well with rstest-bdd's design philosophy: rstest-bdd provides first-class fixtures that can be shared between unit, integration, and BDD tests.

Critical advantage for this PR: Since the generated test is an rstest, fixture dependency graph Just Works™, which directly supports the PR's goal of per-scenario TestWorld isolation through rstest fixtures.

Fixture Injection and Per-Test Isolation

The core idea is that you can inject test dependencies by passing them as test arguments, and a fixture can be injected by another fixture. This enables the PR's centralized TestWorld approach where each scenario gets a fresh fixture instance automatically.

Rust 1.89.0 and MSRV Context

The PR updates the Rust version to 1.89.0. Rust 1.89.0 was released on August 7, 2025 and stabilizes features including repr128, unpredictable_function_pointer_comparisons expansion, and avx512/kl/widekl target features for x86. The PR's MSRV bump from 1.75 to 1.89.0 is significant; reviewers should verify all new code uses only stable features available in 1.89.0.

Interior Mutability via RefCell

The PR's heavy use of RefCell<Option<T>> for TestWorld fields is a standard Rust pattern for shared test state. This design allows step definitions to mutate world state without taking mutable references, enabling the flexibility needed for BDD scenarios where steps are registered globally and called in sequence.

Compile-Time Step Registration

.feature files are processed at compile time; scenarios are static, and step definitions must be known at compile time (no dynamic registration). This supports the PR's transition from cucumber's dynamic registration to rstest-bdd's static, compile-time approach visible in the new tests/bdd_tests.rs entrypoint.

Key Technical Points for Review

  1. TestWorld as Central Fixture: The new tests/bdd/fixtures/TestWorld struct properly leverages rstest's per-test fixture isolation, eliminating the shared mutable state issues of the old CliWorld.

  2. Type-Safe Step Parameters: The new tests/bdd/types.rs domain newtypes (CliArgs, TargetName, etc.) address the reviewers' repeated feedback about replacing bare String parameters with semantic types at the step boundary.

  3. Toolchain Alignment: The PR updates documentation to reflect migration from nightly dependency to stable toolchain support (while CI validates both stable and MSRV), though you should verify the docs in docs/adr-002 and docs/rstest-bdd-users-guide.md are internally consistent on this point.

  4. PATH Override Mechanism: Changes to src/stdlib/which/ modules introduce explicit path override parameters flowing through WhichResolver, EnvSnapshot::capture, and WhichConfig::new, enabling per-instance path overrides without global mutation.

⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: release / build-macos (x86_64-apple-darwin, macos-15-intel, macos-x86_64, macos-x86_64) / Build macos artefacts
  • GitHub Check: release / build-linux (x86_64-unknown-linux-gnu, linux-amd64, amd64) / Build linux artefacts
  • GitHub Check: release / build-linux (aarch64-unknown-linux-gnu, linux-arm64, arm64) / Build linux artefacts
  • GitHub Check: release / build-windows (aarch64-pc-windows-msvc, arm64, arm64, windows-aarch64) / Build windows artefacts
  • GitHub Check: release / build-windows (x86_64-pc-windows-msvc, amd64, x64, windows-x86_64) / Build windows artefacts
  • GitHub Check: release / build-macos (aarch64-apple-darwin, macos-15, macos-arm64, macos-aarch64) / Build macos artefacts
  • GitHub Check: build-test (ubuntu-latest, stable)
  • GitHub Check: build-test (ubuntu-latest, 1.89.0)
🔇 Additional comments (24)
tests/bdd/steps/process.rs (8)

1-16: LGTM!

Module doc comment present, imports are well-organized, and dependencies are appropriate for the BDD step definitions.


79-89: LGTM!

Simple delegation with clear error propagation.


91-103: LGTM!

Clear and concise result recording pattern.


109-165: LGTM!

Given steps are well-structured with appropriate platform-specific guards. The no_ninja step correctly simulates a missing executable by referencing a non-existent path.


167-193: LGTM!

The build_dir_exists step correctly creates a directory named build.ninja to test the error path when the build file is actually a directory.


199-222: LGTM!

The #[expect] attribute is correctly scoped with a clear reason and includes a FIXME link for tracking. The if-let-else pattern provides better readability for path resolution.


224-246: LGTM!

Good extraction of run_subcommand helper to avoid duplication between run_clean and run_graph. The {e:#} format preserves the error chain.


252-289: LGTM!

Then steps provide clear assertions with descriptive error messages. The ensure! macro is appropriate for BDD step validation.

tests/bdd/steps/stdlib/policy.rs (3)

1-1: Module documentation is clear and compliant.

The module-level doc comment meets the requirement to explain the module's purpose and utility.


12-35: Helper extraction successfully eliminates duplication.

The modify_policy and modify_policy_infallible helpers consolidate the repeated pattern of take-modify-store across all step functions, directly addressing the previous review feedback. The generic closure approach provides flexibility whilst maintaining type safety.


51-59: Lint suppression is properly scoped and documented.

The #[expect(clippy::unnecessary_wraps)] is correctly scoped to this single function, includes a clear reason referencing the rstest-bdd macro constraint, and provides a FIXME link to the upstream issue. This follows the strict lint guidelines.

tests/bdd/steps/manifest/targets.rs (1)

1-251: Past review concern about file length appears resolved.

The past review flagged that this file exceeded 400 lines (421 lines). The provided code shows the file is now 251 lines, which complies with the 400-line guideline. The suggested modularisation into helpers.rs (for helper types and assertion utilities) has been implemented, as evidenced by the imports from super::helpers on lines 6-11.

tests/bdd/steps/manifest/helpers.rs (3)

1-193: Excellent use of macro-based code generation to reduce duplication.

The define_assertion! macro (lines 68-78) effectively generates typed assertion wrappers around the generic assert_string_eq function, eliminating repetitive boilerplate whilst maintaining type safety and clear error messages. The invocations on lines 80-84 create five assertion functions with consistent signatures and behaviour.


19-48: FieldLocation struct provides good error context.

The FieldLocation struct captures location information (context, optional index, field name) and the format_prefix method (lines 42-47) formats it appropriately for error messages. The use of const fn for with_index (line 29) enables compile-time construction where possible.


130-143: with_target encapsulates the 1-based indexing pattern effectively.

The with_target helper (lines 130-143) consolidates the common pattern of:

  1. Validating 1-based index
  2. Accessing manifest via with_ref
  3. Retrieving target with bounds check
  4. Applying a closure
  5. Wrapping result with manifest error context

This reduces duplication across step functions in targets.rs and ensures consistent error handling.

tests/bdd/steps/cli.rs (5)

1-5: Module documentation properly expanded.

The module doc now explains both purpose and utility, integrating with TestWorld as required by coding guidelines.


20-60: Helper functions are well-structured.

Proper error context and Result usage throughout.


95-274: Verification helpers are well-implemented.

The typed verification approach with ExpectedCommand enum and individual verify functions provides good separation of concerns. Past review feedback has been addressed: job count verification simplified, lint reason corrected, and bail! used consistently.


280-370: Step definitions follow BDD best practices.

Proper newtype conversion at step boundaries, consistent Result handling, and appropriate lint suppression for the rstest-bdd macro constraint.


71-78: Test coverage for quoted and space-containing arguments already exists in the BDD scenarios. The cli.feature file includes explicit cases for single-quoted arguments (line 90–94), double-quoted arguments (line 96–100), escaped spaces (line 102–106), and multiple quoted arguments (line 108–112). The shlex code path is fully exercised by these scenarios. No action is required.

Likely an incorrect or invalid review comment.

tests/bdd/types.rs (4)

82-103: PathString properly implements path-centric API.

The AsRef<Path> trait implementation was added as requested in previous reviews, complementing as_path() and to_path_buf().


135-152: ManifestPath implements complete path-centric API.

The AsRef<Path> trait and to_path_buf() method were added as requested in previous reviews, providing the standard path wrapper interface.


249-356: Domain-specific type extensions are well-designed.

Appropriate helper methods for each domain: FileContents::as_bytes(), TokenList::to_vec(), and the HelperName distinction using &'static str. The ContentName enum with its as_str() and Display implementation is particularly clean.


362-382: JobCount properly models non-string domain value.

Appropriate use of a non-tuple struct with semantic constructor and accessor, with proper derives for comparison.

Comment thread tests/bdd/steps/stdlib/policy.rs
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/steps/stdlib/workspace.rs
Comment thread tests/bdd/types.rs
Comment thread tests/bdd/types.rs
@leynos
Copy link
Copy Markdown
Owner Author

leynos commented Dec 29, 2025

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix:

tests/bdd/steps/manifest/helpers.rs

Comment on lines +182 to +193

pub(super) fn assert_macro_count(world: &TestWorld, expected: usize) -> Result<()> {
    let count = world
        .manifest
        .with_ref(|m| m.macros.len())
        .context("manifest has not been parsed");
    let actual = with_manifest_error_context(world, count)?;
    ensure!(
        actual == expected,
        "expected manifest to have {expected} macros, got {actual}"
    );
    Ok(())
}

❌ New issue: Code Duplication
The module contains 2 functions with similar structure: assert_macro_count,assert_target_count

@coderabbitai

This comment was marked as resolved.

…orkspace helpers

- Introduce a generic function `assert_manifest_count` to handle manifest collection count assertions,
  replacing duplicated code for targets and macros in BDD test steps.
- Simplify `assert_target_count` and `assert_macro_count` by delegating to the new generic function.
- Add detailed documentation and caching logic to `ensure_workspace` for stdlib test workspace setup.
- Add utility functions `resolve_template_path` and `resolve_executable_path` with platform-specific handling.
- Minor cleanup in BDD type wrappers documentation with improved clippy expectations and allowance reasons.

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
@leynos leynos changed the title Refactor BDD steps into shared TestWorld helpers with rstest-bdd Address review comments; run tests for rstest-bdd harness Dec 29, 2025
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking as resolved: documentation added to clarify design choice.

- Added newtype wrappers Scheme and HostName to bdd/types.rs for URL scheme and host name
- Updated policy manipulation functions to use these newtypes for improved type safety and clarity
- Modified error handling in modify_policy to accept any error convertible to anyhow::Error
- Refactored allow_scheme, allow_host, and block_host steps to use the newtypes instead of raw strings

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
@leynos leynos changed the title Address review comments; run tests for rstest-bdd harness Migrate BDD harness to per-scenario TestWorld with shared fixtures Dec 29, 2025
…] for API completeness

Replaced instances of #[allow(dead_code)] with #[expect(dead_code)] annotations in the newtype and path wrappers. This change improves lint clarity by specifying that dead code warnings are expected and justified due to varying usage across type instantiations rather than simply allowing them.

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
@leynos leynos changed the title Migrate BDD harness to per-scenario TestWorld with shared fixtures Migrate BDD harness to per-scenario TestWorld with rstest-bdd Dec 29, 2025
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4980d4a and de9d491.

📒 Files selected for processing (1)
  • tests/bdd/types.rs
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Clippy warnings MUST be disallowed in Rust code.
Fix any warnings emitted during Rust tests in the code itself rather than silencing them.
In Rust, extract meaningfully named helper functions when a function is too long, adhering to separation of concerns and CQRS.
In Rust, group related parameters in meaningfully named structs when a function has too many parameters.
In Rust, consider using Arc to reduce data returned when a function is returning a large error.
Every Rust module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs in Rust using Rustdoc comments (///) so documentation can be generated with cargo doc.
In Rust, prefer immutable data and avoid unnecessary mut bindings.
In Rust, handle errors with the Result type instead of panicking where feasible.
Avoid unsafe code in Rust unless absolutely necessary, and document any usage clearly.
In Rust, place function attributes after doc comments.
In Rust, do not use return in single-line functions.
In Rust, use predicate functions for conditional criteria with more than two branches.
Lints in Rust must not be silenced except as a last resort.
Lint rule suppressions in Rust must be tightly scoped and include a clear reason.
In Rust, prefer expect over allow.
In Rust tests, use rstest fixtures for shared setup.
In Rust tests, replace duplicated tests with #[rstest(...)] parameterized cases.
In Rust, prefer mockall for mocks and stubs.
In Rust, use concat!() to combine long string literals rather than escaping newlines with a backslash.
In Rust, prefer single line versions of functions where appropriate (e.g., pub fn new(id: u64) -> Self { Self(id) } instead of multi-line).
Use NewTypes in Rust to model domain values and eliminate 'integer soup'. Reach for newt-hype when introducing many homogeneous wrappers; add small shims for string-backed wrappers. For path-centric wrappers implement AsRef<Path> along...

Files:

  • tests/bdd/types.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cognitive complexity of functions no more than 9.

  • Adhere to single responsibility and CQRS
  • Place function attributes after doc comments.
  • Do not use return in single-line functions.
  • Move conditionals with >2 branches into a predicate function.
  • Avoid unsafe unless absolutely necessary.
  • Every module must begin with a //! doc comment that explains the module's purpose and utility.
  • Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar
  • Lints must not be silenced except as a last resort.
    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.
  • Use rstest fixtures for shared setup and to avoid repetition between tests.
  • Replace duplicated tests with #[rstest(...)] parameterised cases.
  • Prefer mockall for mocks/stubs.
  • Prefer .expect() over .unwrap() in tests.
  • .expect() and .unwrap() are forbidden outside of tests. Errors must be propagated.
  • Ensure that any API or behavioural changes are reflected in the documentation in docs/
  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/
  • Files must not exceed 400 lines in length
    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such
    • For testing of functionality depending upon environment variables, dependency injection and...

Files:

  • tests/bdd/types.rs
🧬 Code graph analysis (1)
tests/bdd/types.rs (2)
src/manifest/expand.rs (1)
  • as_str (148-152)
src/runner/mod.rs (1)
  • into_string (42-44)
🪛 GitHub Actions: CI
tests/bdd/types.rs

[error] 28-28: this lint expectation is unfulfilled.

🪛 GitHub Check: build-test (ubuntu-latest, 1.89.0)
tests/bdd/types.rs

[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 34-34:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled

🪛 GitHub Check: build-test (ubuntu-latest, nightly, true)
tests/bdd/types.rs

[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 34-34:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled

🪛 GitHub Check: build-test (ubuntu-latest, stable)
tests/bdd/types.rs

[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 34-34:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled


[failure] 28-28:
this lint expectation is unfulfilled

🔍 Remote MCP

Based on my search of the repository documentation, here is the key additional context relevant to reviewing this pull request:

Summary of Additional Context Found

rstest-bdd Library Information

rstest-bdd brings BDD to Rust without requiring a bespoke runner or monolithic "world" object, instead building on rstest's excellent fixture and parametrization model. The framework allows scenarios to be executed with cargo test, enables first-class fixtures shared between unit and BDD tests, and provides ergonomic step definitions.

Migration Motivation and Related Work

The Deepwiki search confirmed that the repository currently uses a three-tier testing strategy with:

  • rstest for fixture-based unit tests
  • cucumber-rs for BDD with the CliWorld struct managing state
  • insta for snapshot testing

The PR's migration from cucumber-rs with CliWorld to rstest-bdd with TestWorld fixtures directly addresses the architectural goal of per-scenario isolation mentioned in the PR summary.

Test Framework Capabilities

Steps are registered at compile time into a global registry, at runtime the framework matches each Gherkin line against the registry and invokes the correct function, and if a step has no matching implementation you get a compile error.

Related Architecture from Deepwiki

The repository's overall architecture uses a six-stage pipeline for transformations:

  1. File ingestion
  2. YAML parsing
  3. Template expansion (foreach/when)
  4. Deserialization and string rendering
  5. IR generation and validation
  6. Ninja synthesis and execution

This context helps validate that the BDD infrastructure changes (this PR) are foundational to supporting the full pipeline testing strategy documented in the design.

⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: release / build-windows (x86_64-pc-windows-msvc, amd64, x64, windows-x86_64) / Build windows artefacts
  • GitHub Check: release / build-linux (x86_64-unknown-linux-gnu, linux-amd64, amd64) / Build linux artefacts
  • GitHub Check: release / build-windows (aarch64-pc-windows-msvc, arm64, arm64, windows-aarch64) / Build windows artefacts
  • GitHub Check: release / build-macos (x86_64-apple-darwin, macos-15-intel, macos-x86_64, macos-x86_64) / Build macos artefacts
🔇 Additional comments (1)
tests/bdd/types.rs (1)

134-137: The #[expect(dead_code)] attribute at line 134 is correct—ManifestPath::to_path_buf() is genuinely unused throughout the codebase. The method exists for API completeness as a path wrapper, and the expectation is narrowly scoped with a clear reason. No changes required.

Comment thread tests/bdd/types.rs Outdated
Comment thread tests/bdd/types.rs
Comment thread tests/bdd/types.rs
…] in macro-generated code

The macro-generated newtypes provide complete APIs, but some methods are unused depending on instantiation. Since #[expect(dead_code)] requires the lint to fire and its usage varies, it was replaced with #[allow(dead_code)] to suppress warnings more consistently. Additionally, a module-level expect for clippy::allow_attributes was added to justify using #[allow(dead_code] within the macro.

Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
@leynos leynos changed the title Migrate BDD harness to per-scenario TestWorld with rstest-bdd Refactor BDD harness to per-scenario TestWorld with rstest-bdd Dec 29, 2025
@leynos leynos merged commit 8bcee3d into main Dec 29, 2025
15 checks passed
@leynos leynos deleted the terragon/migrate-netsuke-to-rstest-bdd-lt13bh branch December 29, 2025 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant