ADR-001: Replace Cucumber with rstest-bdd for BDD tests#408
Conversation
Add Architecture Decision Record 001 proposing replacement of the Cucumber behavioural testing framework with rstest-bdd. The ADR details context, rationale, approach, consequences, and a phased roadmap for migrating tests to rstest-bdd to unify test execution under cargo test, improve fixture reuse, and reduce maintenance overhead. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
Reviewer's GuideAdds ADR-001 documenting the migration of Wireframe’s behavioural tests from the cucumber crate to rstest-bdd, including rationale, migration approach, consequences, and a phased roadmap for adopting rstest-bdd and retiring Cucumber. Sequence diagram for running behavioural tests under cargo test with rstest-bddsequenceDiagram
actor Developer
participant CargoTest
participant RstestBddMacros
participant RstestBdd
participant RstestFixtures
participant ScenarioState
participant ApplicationUnderTest
Developer->>CargoTest: invoke_cargo_test_for_bdd_suite
CargoTest->>RstestBddMacros: expand_scenario_and_step_macros
RstestBddMacros-->>CargoTest: generated_test_functions_with_compile_time_validation
CargoTest->>RstestBdd: run_generated_scenarios
loop for_each_scenario
RstestBdd->>RstestFixtures: instantiate_required_fixtures
RstestFixtures-->>RstestBdd: fixture_instances
RstestBdd->>ScenarioState: initialize_per_scenario_state_if_needed
ScenarioState-->>RstestBdd: scenario_state_instance
RstestBdd->>ApplicationUnderTest: execute_given_when_then_steps_using_fixtures_and_state
ApplicationUnderTest-->>RstestBdd: step_results
RstestBdd-->>CargoTest: report_scenario_pass_fail
end
CargoTest-->>Developer: aggregated_test_results_for_bdd_and_other_suites
Flow diagram for the three-phase migration roadmap from Cucumber to rstest-bddflowchart TD
Start["Start migration ADR_001"] --> P1["Phase_1 establish_rstest_bdd_foundation"]
subgraph Phase_1
P1 --> P1_Toolchain["Task Add_rstest_bdd_and_rstest_bdd_macros_dev_dependencies_with_compile_time_validation"]
P1_Toolchain --> P1_Layout["Task Create_tests_bdd_layout_and_smoke_scenario_under_cargo_test"]
P1_Layout --> P1_Docs["Task Document_new_BDD_test_layout_and_invocation_in_testing_guides"]
end
P1_Docs --> P2["Phase_2 migrate_existing_scenarios"]
subgraph Phase_2
P2 --> P2_Worlds["Task Convert_Cucumber_worlds_to_rstest_fixtures_or_ScenarioState"]
P2_Worlds --> P2_Helpers["Task Move_shared_helpers_into_wireframe_testing_where_needed"]
P2_Helpers --> P2_Steps["Task Translate_step_definitions_to_given_when_then_functions"]
P2_Steps --> P2_Validate["Task Ensure_migrated_scenarios_pass_under_cargo_test_and_coverage_matches"]
end
P2_Validate --> P3["Phase_3 retire_Cucumber"]
subgraph Phase_3
P3 --> P3_DeleteCode["Task Delete_tests_cucumber_rs_and_Cucumber_specific_world_modules_and_dependency"]
P3_DeleteCode --> P3_CI["Task Update_CI_pipelines_to_drop_Cucumber_target_and_use_cargo_test"]
P3_CI --> P3_Docs["Task Replace_or_retire_Cucumber_documentation"]
end
P3_Docs --> End["Cucumber_retired_rstest_bdd_is_primary_BDD_framework"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Summary by CodeRabbitRelease Notes
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughDocument ADR-001 that specifies migrating behavioural tests from Cucumber to rstest-bdd, preserving feature files, replacing world/step bindings with rstest-bdd fixtures and ScenarioState, and outlining a phased roadmap (foundation, migration, retirement) with consequences and CI changes. (47 words) Changes
Sequence Diagram(s)(Skipped — changes are documentation and do not introduce new runtime control flow requiring multi-component sequence diagrams.) Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes
Possibly related issues
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsErrors were encountered while retrieving linked issues. Errors (1)
Comment |
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
- Adjust capitalization in headings for consistency - Improve paragraph formatting and line wrapping - Clarify terminology such as "Behaviour-Driven Development (BDD)" - Correct link formatting in footnote These changes improve readability and maintain style conventions in the ADR document. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
|
@sourcery-ai review |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Since the ADR calls out the shift from async Cucumber worlds to synchronous rstest-bdd steps, it would be useful to briefly describe the intended pattern for handling existing async operations (e.g., using a test runtime, helper layers, or explicit async boundaries) so teams have a clear reference design during migration.
- Consider adding a short rollback/mitigation section to the ADR that explains what happens if rstest-bdd’s current limitations (like wildcard steps) or other gaps block migration, so there’s an explicit plan if parity cannot be reached as expected.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Since the ADR calls out the shift from async Cucumber worlds to synchronous rstest-bdd steps, it would be useful to briefly describe the intended pattern for handling existing async operations (e.g., using a test runtime, helper layers, or explicit async boundaries) so teams have a clear reference design during migration.
- Consider adding a short rollback/mitigation section to the ADR that explains what happens if rstest-bdd’s current limitations (like wildcard steps) or other gaps block migration, so there’s an explicit plan if parity cannot be reached as expected.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
docs/adr-001-replace-cucumber-with-rstest-bdd.md
🧰 Additional context used
📓 Path-based instructions (3)
docs/**/*.md
📄 CodeRabbit inference engine (docs/contents.md)
Follow the documentation style guide conventions when writing project documentation
docs/**/*.md: Use British English based on the Oxford English Dictionary (en-GB-oxendict), including: -ize suffix in words like 'realize' and 'organization', -lyse suffix in words like 'analyse', -our suffix in words like 'colour', -re suffix in words like 'centre', double 'l' in words like 'cancelled', maintain 'e' in words like 'likeable', -ogue suffix in words like 'catalogue'
The word 'outwith' is acceptable in British English documentation
Keep US spelling when used in an API (for example, 'color')
Use the Oxford comma in documentation: 'ships, planes, and hovercraft' where it aids comprehension
Treat company names as collective nouns: 'Lille Industries are expanding'
Write headings in sentence case in documentation
Use Markdown headings (#, ##, ###, and so on) in order without skipping levels
Follow markdownlint recommendations for Markdown files
Always provide a language identifier for fenced code blocks in documentation; use 'plaintext' for non-code text
Use '-' as the first level bullet and renumber lists when items change in documentation
Prefer inline links using 'text' or angle brackets around the URL in documentation
Ensure blank lines before and after bulleted lists and fenced blocks in documentation
Ensure tables have a delimiter line below the header row in documentation
Expand any uncommon acronym on first use in documentation (for example, Continuous Integration (CI))
Wrap paragraphs at 80 columns in documentation
Wrap code at 120 columns in documentation
Do not wrap tables in documentation
Use footnotes referenced with '[^label]' in documentation
Include Mermaid diagrams where they add clarity to documentation
When embedding figures in documentation, use '' and provide brief alt text describing the content
Add a short description before each Mermaid diagram so screen readers can understand itWrite official documentation with se...
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
docs/**/*.{rs,md}
📄 CodeRabbit inference engine (docs/rust-doctest-dry-guide.md)
Every doctest should validate the public API of a crate from the perspective of an external user, treating each documentation test as a separate temporary crate that imports the library as an external dependency
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
**/*.md
📄 CodeRabbit inference engine (AGENTS.md)
**/*.md: Validate Markdown files usingmake markdownlint.
Runmake fmtafter any documentation changes to format all Markdown files and fix table markup.
Validate Mermaid diagrams in Markdown files by runningmake nixie.
Markdown paragraphs and bullet points must be wrapped at 80 columns.
Code blocks in Markdown must be wrapped at 120 columns.
Markdown tables and headings must not be wrapped.
Use dashes (-) for list bullets in Markdown.
Use GitHub-flavoured Markdown footnotes ([^1]) for references and footnotes in Markdown.
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
⚙️ CodeRabbit configuration file
**/*.md: * Avoid 2nd person or 1st person pronouns ("I", "you", "we")
- Use en-GB-oxendict (-ize / -yse / -our) spelling and grammar
- Headings must not be wrapped.
- Documents must start with a level 1 heading
- Headings must correctly increase or decrease by no more than one level at a time
- Use GitHub-flavoured Markdown style for footnotes and endnotes.
- Numbered footnotes must be numbered by order of appearance in the document.
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
🪛 LanguageTool
docs/adr-001-replace-cucumber-with-rstest-bdd.md
[uncategorized] ~63-~63: Use a comma before “and” if it connects two independent clauses (unless they are closely connected and short).
Context: ...dencies only after parity is established and behavioural coverage is fully represent...
(COMMA_COMPOUND_SENTENCE_2)
[uncategorized] ~94-~94: Possible missing comma found.
Context: ...rstest fixture or ScenarioState struct and move shared helpers into `wireframe...
(AI_HYDRA_LEO_MISSING_COMMA)
🔍 Remote MCP
Let me try a web search to gather additional context about rstest-bdd and the migration approach:Excellent context found on rstest-bdd. Now let me search for additional information on Cucumber-rs and BDD testing approaches in Rust to provide better comparative context:Perfect! I now have comprehensive context about both rstest-bdd and Cucumber-rs. Let me compile the relevant findings for the PR review.
Additional Context for PR Review
Key Framework Characteristics
rstest-bdd brings the collaborative clarity of BDD to Rust without asking you to adopt a bespoke runner or a monolithic "world" object. It executes scenarios with cargo test and allows you to share rstest fixtures between unit, integration, and BDD tests. It provides ergonomic step definitions with #[given], #[when], #[then] macros with typed placeholders and explicit #[scenario] binding from test code to a named scenario.
Cucumber-rs (Current Approach) is a native Rust implementation of the Cucumber testing framework with no external test runners or dependencies. However, Cucumber assumes involvement of non-technical members on a project and provides human-readable syntax via the Gherkin language. Cucumber-rs requires a custom runner and uses a monolithic World struct for state management.
Advantages of rstest-bdd for the Migration
-
Test Harness Alignment: Execute scenarios with cargo test - aligns with Rust's standard test infrastructure instead of requiring a dedicated runner.
-
Fixture Reuse: Share rstest fixtures between unit, integration, and BDD tests - enables better integration with existing test infrastructure and reduces boilerplate.
-
Compile-Time Validation: Support for compile-time and strict compile-time validation features - prevents missing or incorrect step bindings at compile time.
-
Design Philosophy: rstest-bdd offers a different trade-off for teams already invested in rstest and cargo test - fits teams valuing integrated Rust tooling over cross-platform BDD standardization.
Known Limitations to Consider
-
IDE navigation from Gherkin to Rust may require tooling support - potential friction for developers used to better IDE integration with Cucumber.
-
Registry implementation relies on platform features; use no-inventory if needed - may require workarounds on certain platforms.
Migration Path Validation
The ADR's approach aligns with rstest-bdd's design: Feature parity includes Scenario Outlines, Background, data tables, and docstrings - supporting preservation of existing .feature files. The phased roadmap (foundation → migration → retirement) reduces execution risk by enabling parallel operation during transition.
[::web_search::]
⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Sourcery review
- GitHub Check: Sourcery review
|
@coderabbitai is it fair to say, re the following, that we should postpone the migration until after the forthcoming async support in rstest-bdd is implemented?
|
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 5112 🌐 Web query: 💡 Result: Short answer: As of Dec 2025, rstest-bdd is still an alpha crate (0.1.x) and supports async tests via runtime feature flags (tokio / async-std) — it relies on rstest’s async support (runtime‑agnostic/future helpers). Use the crate feature (tokio or async-std) and rstest’s #[future]/#[awt] helpers or the chosen async test attribute. [1][2][3] Sources
🌐 Web query: 💡 Result: Yes — rstest-bdd supports async steps. Minimal example (browser/WebDriver style) — async fixtures, async step defs, and an async scenario test: Code: References: rstest-bdd docs with async step & scenario examples. [1][2] Sources
(If you want a non-WebDriver example or an explanation of #[future]/#[from] usage, tell me which runtime — tokio/async-std — and I’ll adapt the snippet.) @leynos: No, you should not postpone the migration—async support already exists in rstest-bdd. My web search reveals that rstest-bdd supports async step definitions through rstest's async infrastructure. The framework provides runtime-agnostic async support via feature flags ( Current Async Support Patternuse rstest::fixture;
use rstest_bdd::{scenario, given, when, then};
#[fixture]
async fn some_async_fixture() -> Result<SomeResource> {
// async setup
}
#[scenario(path = "features/example.feature", name = "Some scenario")]
#[tokio::test]
async fn test_scenario(#[future] some_async_fixture: SomeResource) {
// scenario executes here
}
#[given("some precondition")]
async fn async_given_step(fixture: &mut SomeResource) -> Result<()> {
// async operations here
Ok(())
}
#[when(r#"some action with "(.*)""#)]
async fn async_when_step(fixture: &mut SomeResource, param: String) -> Result<()> {
// async operations here
Ok(())
}Recommended Updates to the ADRThe "Consequences" section currently states that steps will move to "synchronous functions", which is inaccurate. Update it to:
This means your existing async Cucumber steps can translate directly to async rstest-bdd steps without the refactoring overhead the ADR currently anticipates. Proceed with the migration—the async support is production-ready. |
|
@coderabbitai speaking as the lead (and only, apart from my robot helpers) dev on |
|
Tip For best results, initiate chat on the files or code changes. @leynos: I apologise for the error—thank you for the correction. As the maintainer, you're obviously the authoritative source on rstest-bdd's capabilities. Given that async support is planned but not yet implemented, let me reconsider your original question: Yes, it would be fair to postpone the migration until async support lands, especially if your existing Cucumber scenarios rely heavily on async operations. ConsiderationsArguments for postponing:
Arguments for proceeding now:
RecommendationUpdate the ADR's "Consequences" section to explicitly state:
Since you control both the migration timeline and the rstest-bdd roadmap, you're well-positioned to coordinate them optimally. ✏️ Learnings added
|
- Document that rstest-bdd async step support is planned but unavailable. - Recommend aligning migration timing with async support release to avoid rewrites. - Highlight necessity to refactor or defer async behavior during migration. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
Refine wording and structure in ADR-001 to clarify migration plan from Cucumber to rstest-bdd BDD framework. Enhance roadmap phases, detail handling of async steps, and add mitigation and rollback strategies to guide behavioural test suite transition. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
docs/adr-001-replace-cucumber-with-rstest-bdd.md
🧰 Additional context used
📓 Path-based instructions (3)
docs/**/*.md
📄 CodeRabbit inference engine (docs/contents.md)
Follow the documentation style guide conventions when writing project documentation
docs/**/*.md: Use British English based on the Oxford English Dictionary (en-GB-oxendict), including: -ize suffix in words like 'realize' and 'organization', -lyse suffix in words like 'analyse', -our suffix in words like 'colour', -re suffix in words like 'centre', double 'l' in words like 'cancelled', maintain 'e' in words like 'likeable', -ogue suffix in words like 'catalogue'
The word 'outwith' is acceptable in British English documentation
Keep US spelling when used in an API (for example, 'color')
Use the Oxford comma in documentation: 'ships, planes, and hovercraft' where it aids comprehension
Treat company names as collective nouns: 'Lille Industries are expanding'
Write headings in sentence case in documentation
Use Markdown headings (#, ##, ###, and so on) in order without skipping levels
Follow markdownlint recommendations for Markdown files
Always provide a language identifier for fenced code blocks in documentation; use 'plaintext' for non-code text
Use '-' as the first level bullet and renumber lists when items change in documentation
Prefer inline links using 'text' or angle brackets around the URL in documentation
Ensure blank lines before and after bulleted lists and fenced blocks in documentation
Ensure tables have a delimiter line below the header row in documentation
Expand any uncommon acronym on first use in documentation (for example, Continuous Integration (CI))
Wrap paragraphs at 80 columns in documentation
Wrap code at 120 columns in documentation
Do not wrap tables in documentation
Use footnotes referenced with '[^label]' in documentation
Include Mermaid diagrams where they add clarity to documentation
When embedding figures in documentation, use '
Add a short description before each Mermaid diagram so screen readers can understand itWrite official documentation with se...
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
docs/**/*.{rs,md}
📄 CodeRabbit inference engine (docs/rust-doctest-dry-guide.md)
Every doctest should validate the public API of a crate from the perspective of an external user, treating each documentation test as a separate temporary crate that imports the library as an external dependency
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
**/*.md
📄 CodeRabbit inference engine (AGENTS.md)
**/*.md: Validate Markdown files usingmake markdownlint.
Runmake fmtafter any documentation changes to format all Markdown files and fix table markup.
Validate Mermaid diagrams in Markdown files by runningmake nixie.
Markdown paragraphs and bullet points must be wrapped at 80 columns.
Code blocks in Markdown must be wrapped at 120 columns.
Markdown tables and headings must not be wrapped.
Use dashes (-) for list bullets in Markdown.
Use GitHub-flavoured Markdown footnotes ([^1]) for references and footnotes in Markdown.
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
⚙️ CodeRabbit configuration file
**/*.md: * Avoid 2nd person or 1st person pronouns ("I", "you", "we")
- Use en-GB-oxendict (-ize / -yse / -our) spelling and grammar
- Headings must not be wrapped.
- Documents must start with a level 1 heading
- Headings must correctly increase or decrease by no more than one level at a time
- Use GitHub-flavoured Markdown style for footnotes and endnotes.
- Numbered footnotes must be numbered by order of appearance in the document.
Files:
docs/adr-001-replace-cucumber-with-rstest-bdd.md
🔍 Remote MCP Deepwiki
Summary of additional, review-relevant facts found about rstest-bdd (concise):
- Architecture & registration
- Step defs compile to wrappers that call inventory::submit!(Step); runtime discovers steps via inventory::iter and builds STEP_MAP for fast lookup.
- Macros & generated tests
- Procedural macros: #[given]/#[when]/#[then] (step wrappers) and #[scenario] (parses .feature at compile time and emits #[rstest] tests). Scenario Outline → generated #[case] attributes per Examples row.
- Fixture model & StepContext
- Generated tests are #[rstest] functions; fixtures from rstest are inserted into a StepContext and injected into step functions by name/type at runtime. Step return values can be boxed and, if they match exactly one fixture TypeId, override that fixture for subsequent steps.
- Pattern/placeholder handling
- Patterns use {name} or {name:Type}; rstest-bdd-patterns compiles these to anchored regexes; typed hints (u32, i32, f64, etc.) map to specialized regex fragments and captured strings are parsed via FromStr.
- Compile-time validation
- Optional features: compile-time-validation (emit warnings for missing/ambiguous steps) and strict-compile-time-validation (treat missing/ambiguous steps as compile errors). Validation is crate-scoped and may not see cross-crate step defs.
- Data tables, docstrings, ScenarioState
- Supports data tables (Vec<Vec>) and strong-typed Rows via #[derive(DataTableRow)], docstrings passed as String, and ScenarioState/Slot with #[derive(ScenarioState)] to manage mutable per-scenario state.
- Diagnostics & tooling
- diagnostics feature (default) enables serde-based registry dumps and the cargo-bdd CLI (steps/unused/duplicates) which runs test binaries with --dump-steps to collect registry JSON.
- Async tests
- Generated tests can be async by annotating the scenario test function with #[tokio::test] or #[async_std::test]; documentation shows tokio/async-std integration via feature flags (enable appropriate rstest features).
Cited source:
- Deepwiki repository docs for rstest-bdd (overview, macros, users guide, design, pattern crate, datatable, ScenarioState, diagnostics)
⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Sourcery review
🔇 Additional comments (1)
docs/adr-001-replace-cucumber-with-rstest-bdd.md (1)
78-84: Correct rstest-bdd wildcard-step claim.Lines 82–84 incorrectly state that rstest-bdd "currently lacks support for wildcard
*steps". In fact, rstest-bdd supports the Gherkin standard asterisk (*) as a step keyword shorthand (equivalent to Given/When/Then/And), and matches it against registered step patterns normally. Remove this unsupported limitation claim from the migration roadmap. Verify whether any advanced wildcard pattern features (beyond standard Gherkin asterisk support) are relevant to your existing scenarios, and if so, document only those constraints.Likely an incorrect or invalid review comment.
1 participant
Summary
This PR adds ADR-001 documenting the migration from Cucumber to rstest-bdd for Wireframe's behavioural tests. It outlines the rationale, approach, consequences, and a phased roadmap for migrating to the standard Rust test harness while preserving existing feature files.
Changes
Rationale
Approach
Consequences
Roadmap
Phase 1: Establish the rstest-bdd foundation
Phase 2: Migrate existing scenarios
Phase 3: Retire Cucumber
References
🌿 Generated by Terry
ℹ️ Tag @terragon-labs to ask questions and address PR feedback
📎 Task: https://www.terragonlabs.com/task/8bda5c9b-873a-4c8c-adad-7ecc83bafc31
Summary by Sourcery
Documentation: