Skip to content

Add validation tests#10

Merged
leynos merged 4 commits intomainfrom
codex/add-tests-for-mbake-validate-and-cargo-clippy
Jul 12, 2025
Merged

Add validation tests#10
leynos merged 4 commits intomainfrom
codex/add-tests-for-mbake-validate-and-cargo-clippy

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Jul 12, 2025

Summary

  • add test for mbake validate on generated Makefile
  • add test for cargo clippy on generated template
  • document new checks in testing guide

Testing

  • pip install -r requirements.txt
  • pytest -q (fails: lint error from cargo clippy)

https://chatgpt.com/codex/tasks/task_e_68727965f70883229e0c9e6f8b00cd50

Summary by Sourcery

Add tests to validate the generated Makefile with mbake and enforce cargo clippy linting on the rendered project, and update the testing documentation accordingly.

Documentation:

  • Update testing guide to document Makefile validation and Clippy lint checks

Tests:

  • Add test for validating generated Makefile with mbake
  • Add test to enforce cargo clippy linting on generated template

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Jul 12, 2025

Reviewer's Guide

This PR extends the test suite by adding two end-to-end validation tests for generated projects—one ensuring the Makefile passes mbake validate, another enforcing cargo clippy errors on warnings—and updates the testing guide to document these new checks.

Sequence diagram for test execution with mbake validate and cargo clippy

sequenceDiagram
    actor User
    participant Pytest
    participant GeneratedProject
    User->>Pytest: Run pytest
    Pytest->>GeneratedProject: Run mbake validate
    GeneratedProject-->>Pytest: mbake validate result
    Pytest->>GeneratedProject: Run cargo clippy
    GeneratedProject-->>Pytest: cargo clippy result
    Pytest-->>User: Display test results
Loading

File-Level Changes

Change Details Files
Add Makefile validation test
  • Define test_makefile_validates using pytest-copier fixture
  • Generate project with copier.copy and check for Makefile existence
  • Run mbake validate Makefile in the generated project
tests/test_template.py
Add Clippy lint test
  • Define test_clippy_runs using pytest-copier fixture
  • Generate project with copier.copy for clippy example
  • Run cargo clippy --all-targets --all-features -- -D warnings
tests/test_template.py
Update testing documentation with new checks
  • Adjust plugin description formatting
  • Add mention of mbake validate Makefile check
  • Add mention of running cargo clippy with warnings as errors
docs/testing.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jul 12, 2025

Summary by CodeRabbit

  • Documentation

    • Improved testing documentation with clearer formatting and additional details on the verification process.
    • Updated README to specify requirement for Copier version 9.0 or later and clarified installation instructions.
  • Tests

    • Updated tests to use UTC for the licence year.
    • Added tests to validate the generated Makefile and ensure Clippy runs with warnings as errors.
  • Chores

    • Increased minimum required Copier version to 9.0 in configuration.
  • Style

    • Refined Clippy lint configuration for greater clarity and control.

Summary by CodeRabbit

  • Documentation
    • Improved formatting and expanded details in the testing documentation, including clearer explanations of test steps and additional information about Makefile validation and linting.
  • Tests
    • Added tests to verify that the generated Makefile is valid and that Rust linter checks pass with warnings treated as errors.
  • Template
    • Modified Cargo.toml.jinja to specify the pedantic lint configuration with an explicit priority alongside the warning level.
  • Configuration
    • Raised Copier minimum version requirement to 9.0 for compatibility.

Walkthrough

Update the documentation to clarify and expand on the testing process, specifically regarding Makefile validation and Rust linting. Add two new test functions: one to validate the generated Makefile with mbake, and another to run cargo clippy on the rendered project. Remove an unused import from the test suite. Adjust the Clippy lint configuration in the Cargo.toml template to include a priority setting. Raise the Copier tool minimum version requirement to 9.0 in configuration and README.

Changes

File(s) Change Summary
docs/testing.md Improve formatting, clarify pytest-copier usage, and document Makefile and clippy testing.
tests/test_template.py Add test_makefile_validates and test_clippy_runs functions; remove unused subprocess import; use UTC for year.
template/Cargo.toml.jinja Change Clippy pedantic lint config from string to table with level and priority fields.
README.md, copier.yaml Raise Copier minimum version requirement to 9.0 for compatibility.

Sequence Diagram(s)

sequenceDiagram
    participant Tester
    participant Copier
    participant Project
    participant MakefileValidator as mbake
    participant Clippy as cargo_clippy

    Tester->>Copier: Render template with parameters
    Copier->>Project: Create project files
    Tester->>Project: Check for Makefile
    Tester->>MakefileValidator: Validate Makefile
    MakefileValidator-->>Tester: Validation result

    Tester->>Copier: Render template with clippy params
    Copier->>Project: Create project files
    Tester->>Clippy: Run cargo clippy (all targets, all features, warnings as errors)
    Clippy-->>Tester: Linting result
Loading

Possibly related PRs

  • Add Makefile scaffold #6: Adds tests for Makefile validation and cargo clippy linting, directly relating to the new test functions and documentation updates in this PR.
  • Update Makefile tests mxd#161: Modifies Makefile test targets and documents make test, complementing the new Makefile validation and linting tests here.

Poem

In the land of code where templates grow,
Tests now check what Makefiles know.
Clippy inspects with a warning’s glare,
Ensuring Rust code is handled with care.
With docs refreshed and tests anew,
The project shines, robust and true!
🦀🛠️


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e1e5686 and 80e5996.

📒 Files selected for processing (1)
  • tests/test_template.py (4 hunks)
🔇 Additional comments (4)
tests/test_template.py (4)

4-4: Import change addresses timezone concerns appropriately.

The addition of UTC to the datetime import resolves the static analysis warning about timezone-aware datetime usage.


21-21: Timezone-aware datetime usage implemented correctly.

The consistent use of datetime.now(tz=UTC).year across all test functions properly addresses the previous timezone concerns and ensures deterministic behaviour.

Also applies to: 36-36, 51-51, 66-66, 80-80


60-71: Makefile validation test implementation is sound.

The test correctly validates the generated Makefile using mbake validate. The assertion ensures the Makefile exists before attempting validation, which provides clear failure messaging.


74-84: Verify Clippy configuration in the project template

Ensure the generated project enforces Clippy warnings and resolves existing lint failures:

  • Confirm the template defines Clippy rules in either .cargo/config.toml (e.g.
    [build] rustflags = ["-D warnings"]) or in Cargo.toml under [package.metadata.clippy].
  • Remove any lingering TODO/FIXME comments related to linting in the template source.
  • Generate a fresh project and run:
    cargo clippy --all-targets --all-features -- -D warnings
    Verify it exits cleanly with zero warnings or errors.
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch codex/add-tests-for-mbake-validate-and-cargo-clippy

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai auto-generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @leynos - I've reviewed your changes - here's some feedback:

  • Add mbake to the test dependencies (e.g. requirements.txt) so the mbake validate command can run without manual setup.
  • Either document or guard the Clippy test to ensure the Rust toolchain and cargo clippy are available in CI, or skip it when they’re missing to avoid unexpected failures.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Add mbake to the test dependencies (e.g. requirements.txt) so the `mbake validate` command can run without manual setup.
- Either document or guard the Clippy test to ensure the Rust toolchain and `cargo clippy` are available in CI, or skip it when they’re missing to avoid unexpected failures.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8682bd4 and f4b01a6.

📒 Files selected for processing (2)
  • docs/testing.md (1 hunks)
  • tests/test_template.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.md

Instructions used from:

Sources:
⚙️ CodeRabbit Configuration File

🪛 LanguageTool
docs/testing.md

[style] ~7-~7: Consider shortening or rephrasing this to strengthen your wording.
Context: ...renders correctly. Run the tests after making changes to the template: ```bash pip install -r r...

(MAKE_CHANGES)

🪛 Ruff (0.11.9)
tests/test_template.py

66-66: datetime.datetime.now() called without a tz argument

(DTZ005)


80-80: datetime.datetime.now() called without a tz argument

(DTZ005)

🔇 Additional comments (4)
docs/testing.md (2)

4-5: LGTM!

The reformatting improves readability whilst maintaining the same information.


14-15: LGTM!

The addition clearly describes the validation process and aligns with the new test functions added in the PR.

tests/test_template.py (2)

60-71: LGTM!

The test correctly validates the generated Makefile using mbake validate and follows the established pattern of the existing test functions.


74-84: LGTM!

The test properly runs cargo clippy with appropriate flags to treat warnings as errors, which aligns with the PR objectives for validation testing.

Comment thread tests/test_template.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f0cd07a and e1e5686.

📒 Files selected for processing (2)
  • README.md (1 hunks)
  • copier.yaml (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.md

Instructions used from:

Sources:
⚙️ CodeRabbit Configuration File

🔇 Additional comments (1)
copier.yaml (1)

1-1: Synchronise CI and dependency constraints with the new minimum Copier version

Raising _min_copier_version to 9.0 is sensible, yet the project’s CI workflow and any lock-files (e.g. requirements*.txt) still need to guarantee copier>=9. Otherwise, pipelines may pick up an older cached 8.x build and violate the constraint at runtime.
Update all automated installation steps to pin copier>=9 explicitly.

Comment thread README.md
Comment on lines +7 to 12
The template requires **Copier 9.0** or later to avoid incompatibilities.

## How to use

1. Install Copier: `pip install copier`.
1. Install Copier 9.0 or later: `pip install copier`.
2. Run `copier copy gh:leynos/agent-template-rust <destination>`.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enforce the ≥9.0 requirement in the installation command

The prose mandates Copier 9+, but the command pip install copier can still resolve to an older cached wheel when a developer already has Copier 7 installed. Replace the instruction with an explicit version specifier to guarantee compliance.

-1. Install Copier 9.0 or later: `pip install copier`.
+1. Install Copier 9.0 or later: `pip install "copier>=9"`.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
The template requires **Copier 9.0** or later to avoid incompatibilities.
## How to use
1. Install Copier: `pip install copier`.
1. Install Copier 9.0 or later: `pip install copier`.
2. Run `copier copy gh:leynos/agent-template-rust <destination>`.
The template requires **Copier 9.0** or later to avoid incompatibilities.
## How to use
1. Install Copier 9.0 or later: `pip install "copier>=9"`.
2. Run `copier copy gh:leynos/agent-template-rust <destination>`.
🤖 Prompt for AI Agents
In README.md around lines 7 to 12, the installation command for Copier does not
explicitly specify the minimum required version 9.0, which can lead to
installing an older version if cached. Update the pip install command to include
the version specifier "copier>=9.0" to enforce installing Copier version 9.0 or
later.

@leynos leynos merged commit 6e329b7 into main Jul 12, 2025
1 check passed
@leynos leynos deleted the codex/add-tests-for-mbake-validate-and-cargo-clippy branch July 12, 2025 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant