feat: Move comprehensive coverage check workflow before approved#9482
Conversation
Created a dedicated workflow that runs code coverage before PR approval: 🚀 Coverage Runs Early: - Triggers: Push to branches + PR events (opened, sync, ready_for_review) - Smart filtering: Only runs when backend code changes - Fast feedback: Unit tests only for quick coverage results 📊 Comprehensive Reporting: - CodeCov integration with proper flags and naming - PR comments with coverage status and links - Workflow summary with coverage percentage - Coverage artifacts (XML + HTML) saved for review ⚡ Intelligent Execution: - Path filtering: src/backend/**, pyproject.toml, uv.lock - Branch filtering: main, develop, feature/**, fix/**, hotfix/** - Draft protection: Skips draft PRs - Dynamic naming: Different names for push vs PR contexts 🎯 Benefits: - Developers get immediate coverage feedback on push - Reviewers see coverage context during PR review - Coverage issues caught before approval, not after - Continuous monitoring of coverage trends across branches This replaces the previous "coverage after approval" approach with "coverage before approval" - exactly what was requested! 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughAdds a new GitHub Actions workflow to run backend test coverage on pushes/PRs, upload results to Codecov, comment coverage on PRs, enforce a 35% minimum threshold, and publish coverage artifacts. Includes a dependent summary job that reflects the coverage job result. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Dev as Developer
participant GH as GitHub
participant Runner as Actions Runner
participant Tests as Pytest+Coverage
participant CC as Codecov
participant PRBot as PR Comment Bot
participant Art as Artifacts
Dev->>GH: push / PR / workflow_dispatch
GH-->>Runner: Start coverage job (unless draft PR)
Runner->>Runner: Checkout repo (full history)
Runner->>Runner: Setup Python 3.10 + uv, sync deps
Runner->>Runner: Generate .coveragerc (script)
Runner->>Tests: Run pytest with coverage
Tests-->>Runner: coverage.xml, htmlcov
Runner->>CC: Upload coverage (always)
Runner->>Art: Upload artifacts (coverage.xml, htmlcov)
Runner->>PRBot: Upsert PR coverage comment
alt Coverage >= 35%
Runner-->>GH: Job success
else Coverage < 35% or parse error
Runner-->>GH: Job failure with summary note
end
GH-->>Runner: Start coverage-summary job (always)
Runner-->>GH: Mirror result of coverage job
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
github-actions
Bot
added
enhancement
- Coverage now runs on ANY push to ANY branch - Coverage runs on ANY PR with ANY changes - No path filtering - runs regardless of what files changed - No branch filtering - runs on all branches - Ensures coverage runs on every PR as requested
github-actions
Bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (7)
.github/workflows/coverage-check.yml (7)
178-193: Treat skipped (draft PR) coverage as neutral instead of failing the summary jobWith the current script, when the coverage job is skipped (draft PR), this job exits 1, turning the check red. That’s noisy for drafts.
Adjust the script to pass on “skipped”:
- name: Coverage Check Result run: | - if [[ "${{ needs.coverage.result }}" == "success" ]]; then + if [[ "${{ needs.coverage.result }}" == "skipped" ]]; then + echo "ℹ️ Coverage check skipped (likely a draft PR)." + exit 0 + elif [[ "${{ needs.coverage.result }}" == "success" ]]; then echo "✅ Coverage check passed" exit 0 else echo "❌ Coverage check failed or was skipped" echo "Check the coverage job logs for details" exit 1 fiAlternatively, set
continue-on-error: ${{ needs.coverage.result == 'skipped' }}on this step.
48-54: Run the config generator within the project environment and fail fastIf scripts/generate_coverage_config.py imports project deps, calling system python may fail. Also, printing a backend-relative .coveragerc reads better if you generate it from that directory. Add strict shell flags to surface errors.
Apply:
- - name: Generate dynamic coverage configuration - run: | - echo "Generating dynamic coverage configuration..." - python3 scripts/generate_coverage_config.py - echo "Generated .coveragerc with the following exclusions:" - cat src/backend/.coveragerc + - name: Generate dynamic coverage configuration + run: | + set -euo pipefail + echo "Generating dynamic coverage configuration..." + cd src/backend + # Run under the same env used for tests + uv run python ../scripts/generate_coverage_config.py + echo "Generated .coveragerc with the following exclusions:" + cat .coveragercIf the script actually lives under src/backend/scripts/, adjust the path accordingly.
59-66: Use the configured test path and keep pathing consistentYou defined PYTEST_RUN_PATH; use it to avoid drift. Since you cd into src/backend, PYTEST_RUN_PATH should be relative to that dir (e.g., tests/unit).
Apply:
- uv run python -m pytest tests/unit \ + uv run python -m pytest "$PYTEST_RUN_PATH" \Note: This pairs with updating env.PYTEST_RUN_PATH to "tests/unit" (see earlier env diff).
3-7: Reduce noise by scoping workflow triggers to relevant pathsRunning on every change can waste CI minutes. Restrict to backend and workflow files.
Example:
on: push: + paths: + - 'src/backend/**' + - '.github/workflows/coverage-check.yml' + - 'scripts/generate_coverage_config.py' pull_request: types: [opened, synchronize, ready_for_review] + paths: + - 'src/backend/**' + - '.github/workflows/coverage-check.yml' + - 'scripts/generate_coverage_config.py'Tune paths as needed (e.g., include lockfiles).
3-13: Avoid duplicate/blocked runs with concurrency controlTo auto-cancel outdated runs on the same ref (useful on PR push bursts), add workflow-level concurrency.
Insert:
on: push: pull_request: types: [opened, synchronize, ready_for_review] workflow_dispatch: inputs: ref: description: "Branch or commit to check coverage for" required: false type: string + +concurrency: + group: coverage-${{ github.event_name }}-${{ github.ref }} + cancel-in-progress: true
69-78: Optional: Skip Codecov token for public reposIf this repo is public, Codecov usually doesn’t require a token on GitHub Actions. Dropping it reduces secret surface area. Keep it if you rely on private uploads.
No diff provided since repo visibility may vary; remove the token line if safe:
with: # token: ${{ secrets.CODECOV_TOKEN }}
193-193: Add a trailing newlineYAMLlint flagged a missing newline at EOF.
Add a newline after the last line.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
.github/workflows/coverage-check.yml(1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
.github/workflows/coverage-check.yml
[error] 193-193: no new line character at the end of file
(new-line-at-end-of-file)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Code Coverage Check
- GitHub Check: Code Coverage Check
| env: | ||
| POETRY_VERSION: "1.8.2" | ||
| PYTEST_RUN_PATH: "src/backend/tests" | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | ||
|
|
There was a problem hiding this comment.
Remove unused secrets from global env to avoid unnecessary exposure
OPENAI_API_KEY and ANTHROPIC_API_KEY aren’t used anywhere in this workflow. Keeping secrets in the job-wide env makes them available to all third‑party actions (checkout, codecov, peter‑evans), which is an avoidable risk. Also, POETRY_VERSION is defined but Poetry isn’t used.
Apply this diff to drop unused env entries:
env:
- POETRY_VERSION: "1.8.2"
- PYTEST_RUN_PATH: "src/backend/tests"
- OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+ PYTEST_RUN_PATH: "tests/unit"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| env: | |
| POETRY_VERSION: "1.8.2" | |
| PYTEST_RUN_PATH: "src/backend/tests" | |
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | |
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | |
| env: | |
| PYTEST_RUN_PATH: "tests/unit" |
🤖 Prompt for AI Agents
.github/workflows/coverage-check.yml around lines 14 to 19: remove the unused
global env entries POETRY_VERSION, OPENAI_API_KEY and ANTHROPIC_API_KEY because
Poetry isn’t used and the secrets aren’t required by this workflow; update the
env block to only include variables actually used by the job (or remove the env
section entirely if empty), and if those secrets are needed for a specific step
in the future, pass them as step-level env or inputs instead of job-level to
limit exposure.
| coverage: | ||
| name: Code Coverage Check | ||
| runs-on: ubuntu-latest | ||
| if: github.event_name != 'pull_request' || github.event.pull_request.draft == false | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Harden GITHUB_TOKEN permissions for least privilege
By default, jobs run with broad write permissions. This workflow only needs read access plus PR comment write in the coverage job; the summary job needs read only.
Add job-level permissions:
jobs:
coverage:
name: Code Coverage Check
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
+ permissions:
+ contents: read
+ pull-requests: writeAnd for the summary job:
coverage-summary:
name: Coverage Summary
runs-on: ubuntu-latest
needs: coverage
if: always()
+ permissions:
+ contents: read📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| coverage: | |
| name: Code Coverage Check | |
| runs-on: ubuntu-latest | |
| if: github.event_name != 'pull_request' || github.event.pull_request.draft == false | |
| jobs: | |
| coverage: | |
| name: Code Coverage Check | |
| runs-on: ubuntu-latest | |
| if: github.event_name != 'pull_request' || github.event.pull_request.draft == false | |
| permissions: | |
| contents: read | |
| pull-requests: write | |
| coverage-summary: | |
| name: Coverage Summary | |
| runs-on: ubuntu-latest | |
| needs: coverage | |
| if: always() | |
| permissions: | |
| contents: read |
🤖 Prompt for AI Agents
.github/workflows/coverage-check.yml lines 21-25: the workflow currently runs
with default broad token permissions; tighten least-privilege by adding
job-level permissions — for the coverage job set permissions: contents: read and
pull-requests: write (so the job can read repo content and post PR comments
only), and for the summary job set permissions: contents: read (read-only); add
these permissions blocks under each job definition to override the default.
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ inputs.ref || github.ref }} | ||
| fetch-depth: 0 # Fetch full history for better coverage diff |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Pin third‑party actions to commit SHAs
Using floating tags (v3/v4/v5/v6) risks supply-chain surprises. Pin to specific commit SHAs for checkout, setup-python, setup-uv, codecov, upload-artifact, and peter-evans actions.
Please look up the current recommended SHAs and replace versions, e.g.:
Also applies to: 34-37, 39-42, 69-78, 132-140, 142-149, 151-176
🌐 Web query:
Provide the latest stable commit SHAs for:
- actions/checkout@v4
- actions/setup-python@v5
- astral-sh/setup-uv@v6
- codecov/codecov-action@v5
- actions/upload-artifact@v4
- peter-evans/find-comment@v3
- peter-evans/create-or-update-comment@v4
💡 Result:
Here are the latest stable commit SHAs for the specified GitHub Actions as of August 21, 2025. These SHAs refer to the HEAD of the respective version tags (e.g., v4, v5, etc.) at this time — so they point to the latest commit used when you reference, for example, actions/checkout@v4 in your workflow.
1. actions/checkout@v4
SHA: 8f4b7f7655e3ff386d8ab5379747be048c05837c
2. actions/setup-python@v5
SHA: d0a8e5e2f289e22fcd361b0975256801e123e998
3. astral-sh/setup-uv@v6
SHA: a80d4f0aa181de25e26c8a6a89f1bfe847d3d8d9
4. codecov/codecov-action@v5
SHA: 19b11d6a7b6d945bfc664669dec9108c8ecd5028
5. actions/upload-artifact@v4
SHA: e6f3feb5860b9c8349ca01c0cfc544fe4e104a41
6. peter-evans/find-comment@v3
SHA: b9770c8f9327fad4b90e4c0b022b49319173f1c5
7. peter-evans/create-or-update-comment@v4
SHA: b401b1595a3bc07044adad6d86002671ed20e9fa
How to use in your workflow for extra security:
uses: actions/checkout@8f4b7f7655e3ff386d8ab5379747be048c05837cNote:
- Double-check the SHAs, as they may change with new releases.
- To always lock to the latest stable version at this date, reference the SHA directly.
- To get the most current, trustworthy SHAs, it’s best to inspect each repository’s tags and the commit they point to.
If you need SHA updates for other actions or a different date, let me know!
Pin third-party actions to specific commit SHAs
To mitigate supply-chain risks, replace all floating version tags with their corresponding commit SHAs in .github/workflows/coverage-check.yml. At minimum, update the following actions:
actions/checkout@8f4b7f7655e3ff386d8ab5379747be048c05837cactions/setup-python@d0a8e5e2f289e22fcd361b0975256801e123e998astral-sh/setup-uv@a80d4f0aa181de25e26c8a6a89f1bfe847d3d8d9codecov/codecov-action@19b11d6a7b6d945bfc664669dec9108c8ecd5028actions/upload-artifact@e6f3feb5860b9c8349ca01c0cfc544fe4e104a41peter-evans/find-comment@b9770c8f9327fad4b90e4c0b022b49319173f1c5peter-evans/create-or-update-comment@b401b1595a3bc07044adad6d86002671ed20e9fa
Please apply these updates across all referenced ranges in this workflow:
- Lines 28–31
- Lines 34–37
- Lines 39–42
- Lines 69–78
- Lines 132–140
- Lines 142–149
- Lines 151–176
🤖 Prompt for AI Agents
.github/workflows/coverage-check.yml lines 28-31, 34-37, 39-42, 69-78, 132-140,
142-149, 151-176: several third-party GitHub Actions are referenced with
floating/tags; replace each action reference with the exact commit SHA provided
(actions/checkout -> 8f4b7f7655e3ff386d8ab5379747be048c05837c,
actions/setup-python -> d0a8e5e2f289e22fcd361b0975256801e123e998,
astral-sh/setup-uv -> a80d4f0aa181de25e26c8a6a89f1bfe847d3d8d9,
codecov/codecov-action -> 19b11d6a7b6d945bfc664669dec9108c8ecd5028,
actions/upload-artifact -> e6f3feb5860b9c8349ca01c0cfc544fe4e104a41,
peter-evans/find-comment -> b9770c8f9327fad4b90e4c0b022b49319173f1c5,
peter-evans/create-or-update-comment ->
b401b1595a3bc07044adad6d86002671ed20e9fa), updating every occurrence in the
listed line ranges so the workflow pins to those commit SHAs; ensure the syntax
remains owner/repo@<sha> and run a quick workflow lint or local parse to confirm
no typos.
| echo "✅ **Coverage Check**: Passed" >> $GITHUB_STEP_SUMMARY | ||
| else | ||
| echo "❌ **Coverage Check**: Below threshold" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "💡 **Tip**: Add tests for new code to improve coverage" >> $GITHUB_STEP_SUMMARY | ||
| fi |
There was a problem hiding this comment.
Coverage threshold isn’t enforced; the job never fails when below threshold
The shell if/else only echoes messages. Even when the Python snippet exits 1 for low coverage, the step continues and succeeds. That means the coverage job ends “success,” and downstream logic reports Pass.
Make the step fail when below threshold:
if python3 -c "
...
"; then
echo "✅ **Coverage Check**: Passed" >> $GITHUB_STEP_SUMMARY
else
echo "❌ **Coverage Check**: Below threshold" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "💡 **Tip**: Add tests for new code to improve coverage" >> $GITHUB_STEP_SUMMARY
+ exit 1
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| echo "✅ **Coverage Check**: Passed" >> $GITHUB_STEP_SUMMARY | |
| else | |
| echo "❌ **Coverage Check**: Below threshold" >> $GITHUB_STEP_SUMMARY | |
| echo "" >> $GITHUB_STEP_SUMMARY | |
| echo "💡 **Tip**: Add tests for new code to improve coverage" >> $GITHUB_STEP_SUMMARY | |
| fi | |
| echo "✅ **Coverage Check**: Passed" >> $GITHUB_STEP_SUMMARY | |
| else | |
| echo "❌ **Coverage Check**: Below threshold" >> $GITHUB_STEP_SUMMARY | |
| echo "" >> $GITHUB_STEP_SUMMARY | |
| echo "💡 **Tip**: Add tests for new code to improve coverage" >> $GITHUB_STEP_SUMMARY | |
| exit 1 | |
| fi |
🤖 Prompt for AI Agents
.github/workflows/coverage-check.yml around lines 121 to 126: the else branch
only appends messages to GITHUB_STEP_SUMMARY so the job doesn't fail even when
coverage is below threshold; after writing the summary lines for the failing
case, explicitly exit with a non-zero status (e.g., exit 1) to cause the step to
fail and propagate the failure to the workflow.
- Remove 'lgtm' label requirement from CI trigger - Run tests immediately on PR opened/synchronized events - Add ci.yml to path filters to trigger tests when workflow changes - Coverage and tests now run before approval for early feedback 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
github-actions
Bot
added
enhancement
Frontend Unit Test Coverage ReportCoverage Summary
Unit Test Results
|
Codecov Report✅ All modified and coverable lines are covered by tests. ❌ Your project status has failed because the head coverage (5.81%) is below the target coverage (10.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #9482 +/- ##
=======================================
Coverage 34.59% 34.60%
=======================================
Files 1209 1209
Lines 57115 57115
Branches 5419 5419
=======================================
+ Hits 19760 19765 +5
+ Misses 37211 37206 -5
Partials 144 144
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
github-actions
Bot
added
enhancement
carlosrcoelho
requested review from
italojohnny,
jordanrfrazier and
ogabrielluiz
| default: "['3.10']" | ||
| pull_request: | ||
| types: [synchronize, labeled] | ||
| types: [opened, synchronize] |
There was a problem hiding this comment.
Should we use [opened, synchronize, labeled] is it required? with 3 ?
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
Yukiyukiyeah
changed the title
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
|
3 participants
This pull request adds a comprehensive GitHub Actions workflow for automated code coverage checks on the backend. The workflow runs on pushes and pull requests, installs dependencies, generates a dynamic coverage configuration, runs tests, uploads results to CodeCov, and provides coverage summaries and feedback directly in PR comments.
Continuous Integration & Coverage Automation:
.github/workflows/coverage-check.ymlto automate backend code coverage checks on pushes and pull requests for relevant branches and paths.uv, sets up Python 3.10, and generates a dynamic.coveragercconfiguration for flexible test exclusions.Feedback & Reporting:
Summary by CodeRabbit