Skip to content

Include doctests in coverage targets#63

Merged
leynos merged 3 commits intomainfrom
codex/add-doctests-flag-to-test-cov-targets
Aug 13, 2025
Merged

Include doctests in coverage targets#63
leynos merged 3 commits intomainfrom
codex/add-doctests-flag-to-test-cov-targets

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Aug 13, 2025

Summary

  • include doctest runs in test-cov coverage checks
  • collect doctest coverage in test-cov-lcov outputs

Testing

  • make fmt
  • make lint
  • make test

https://chatgpt.com/codex/tasks/task_e_689cd9916ec483229b5144153746c36f

Summary by Sourcery

Add the --doctests flag to both test-cov and test-cov-lcov Makefile targets to ensure doctests are included in coverage reports

Enhancements:

  • Include doctest runs in the workspace-wide coverage checks
  • Collect doctest coverage in the LCOV output for the test-cov-lcov target

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Aug 13, 2025

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The Makefile coverage targets now include doctest runs by adding the --doctests flag to both text-summary and LCOV invocations, ensuring documentation tests are included in coverage metrics.

File-Level Changes

Change Details Files
Enable doctest coverage for text-summary reports
  • Add --doctests flag to the cargo llvm-cov invocation in the test-cov target
Makefile
Enable doctest coverage for LCOV outputs
  • Add --doctests flag to the cargo llvm-cov invocation in the test-cov-lcov target
Makefile

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Aug 13, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Warning

Rate limit exceeded

@leynos has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 8 minutes and 25 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 4107592 and 88a2b4e.

📒 Files selected for processing (1)
  • Makefile (1 hunks)

Summary by CodeRabbit

  • Tests
    • Coverage tasks now include documentation tests, counted in both summary and LCOV reports.
    • Coverage thresholds apply with documentation tests included.
    • Command names and existing flags remain unchanged.
    • Developers will observe updated coverage metrics locally and in CI.
    • No changes to user-facing features or interfaces.

Walkthrough

Add the --doctests flag to cargo-llvm-cov in the test-cov and test-cov-lcov Makefile targets, enabling doctest coverage reporting without altering other flags or target names.

Changes

Cohort / File(s) Summary
Coverage Make targets
Makefile
Add --doctests to cargo-llvm-cov in test-cov (summary/text) and test-cov-lcov (LCOV output). No other changes.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

  • Add coverage test target #58: Modify the same Makefile coverage targets to include --doctests in cargo-llvm-cov, directly overlapping in scope.

Poem

Flip the switch for docs to test,
Let snippets join the coverage quest.
Flags aligned, the targets sing,
LCOV writes its tidy ring.
Code and docs now stride as one—
Green bars rise; the job is done.

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch codex/add-doctests-flag-to-test-cov-targets

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 2 comments

Edit Code Review Bot Settings | Greptile

Comment thread Makefile Outdated
Comment thread Makefile Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4df0b66 and 4107592.

📒 Files selected for processing (1)
  • Makefile (1 hunks)
🧰 Additional context used
🪛 GitHub Actions: CI
Makefile

[error] 44-44: Step 'make lint' failed. Command 'cargo clippy --all-targets --all-features -- -D warnings' exited with code 101. Error: Unrecognized option: 'workspace' during rustc invocation.

🔍 Additional research (Context7)

Summary of relevant context found:

  • PR changes: adds --doctests to cargo-llvm-cov invocations used by Makefile coverage targets test-cov and test-cov-lcov (collects doctest coverage; no API/public changes). Test commands exercised: make fmt, make lint, make test. Related PR: #58 modifies same Makefile targets.,

  • Repo/CI/dev context: project uses a Makefile wrapping Cargo targets (coverage generation in CI), comprehensive test harness (unit + BDD) and tooling around coverage generation/upload in CI — relevant to assessing impact of adding --doctests to coverage commands.

Sources used:,

🔇 Additional comments (2)
Makefile (2)

33-33: Enable doctest coverage correctly.

Adding --doctests to both coverage targets is the right switch for cargo-llvm-cov and aligns with the PR goal.

Also applies to: 38-38


33-33: Ignore RUSTFLAGS export leak
Validated Makefile recipe indentation (tabs on lines 29, 33, 38) and found only local RUSTFLAGS="-D warnings" assignments. No export RUSTFLAGS anywhere in the repo. The --workspace flag is correctly passed to cargo llvm-cov, not to rustc. The CI error “Unrecognized option: ‘workspace’” indicates an outdated cargo-llvm-cov version. Upgrade the plugin in your CI environment to a release that supports --workspace.

Likely an incorrect or invalid review comment.

Comment thread Makefile Outdated
Comment thread Makefile Outdated
leynos and others added 2 commits August 13, 2025 22:03
…ecipe commands.

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ecipe commands.

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Bot Settings | Greptile

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Bot Settings | Greptile

@leynos leynos merged commit 1687452 into main Aug 13, 2025
1 of 2 checks passed
@leynos leynos deleted the codex/add-doctests-flag-to-test-cov-targets branch August 13, 2025 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant