[Due for payment 2025-11-19] Improve CodeCov comment formatting #74374
Description
Background
We implemented CodeCov to help with tracking code coverage and give actionable feedback to contributors working on our repo.
Problem
The comment CodeCov gives is noisy and contains unnecessary information that isn't possible to adjust via the configuration file. CodeCov's engineers have also rejected PRs on their open-source repo to allow us to configure the comment to our liking. When the comment is noisy and contains unnecessary information it becomes less useful and more likely to be ignored. This leads to worse coverage and more deploy blockers in our application.
Solution
Create a GitHub action that runs when CodeCov comments on PRs that will update the comment to our desired formatting.
Specifically it should have these attributes:
- it should only contain table that lists
Coverage Δ, not thePatch % - it should not have the "New features to boost your workflow" section at the bottom
- If the comment contains any ⬇️ emojis it should show the following text:
❌ Looks like you've decreased code coverage for some files. Please write tests to increase, or at least maintain, the existing level of code coverage. See our documentation [here](https://github.com/Expensify/App/blob/main/contributingGuides/CodeCov.md) for how to interpret this table. - Otherwise, it should say
✅ Changes either increased or maintained existing code coverage, great job! - Sometimes CodeCov comments that all lines are covered by existing tests. It shouldn't based on our configuration, but when it does we should leave that unchanged. Example 3 will show that.
Example 1
Current Format
Codecov Report
❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| src/libs/ReportUtils.ts | 50.00% | 2 Missing |
| src/libs/Navigation/Navigation.ts | 0.00% | 1 Missing |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| src/libs/Navigation/Navigation.ts | 56.43% <0.00%> (+22.09%) |
⬆️ |
| src/libs/ReportUtils.ts | 75.10% <50.00%> (+15.88%) |
⬆️ |
... and 169 files with indirect coverage changes
🚀 New features to boost your workflow:
- ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
Desired Format
Codecov Report
✅ Changes either increased or maintained existing code coverage, great job!
| Files with missing lines | Coverage Δ | |
|---|---|---|
| src/libs/Navigation/Navigation.ts | 56.43% <0.00%> (+22.09%) |
⬆️ |
| src/libs/ReportUtils.ts | 75.10% <50.00%> (+15.88%) |
⬆️ |
... and 169 files with indirect coverage changes
Example 2
Current Format
Codecov Report
❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| src/pages/workspace/WorkspaceMembersPage.tsx | 0.00% | 4 Missing |
| ...s/workspace/members/WorkspaceMemberDetailsPage.tsx | 0.00% | 2 Missing |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| ...s/workspace/members/WorkspaceMemberDetailsPage.tsx | 0.00% <0.00%> (ø) |
|
| src/pages/workspace/WorkspaceMembersPage.tsx | 75.31% <0.00%> (-0.97%) |
⬇️ |
... and 218 files with indirect coverage changes
🚀 New features to boost your workflow:
- ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
Desired format
Codecov Report
❌ Looks like you've decreased code coverage for some files. Please write tests to increase, or at least maintain, the existing level of code coverage. See our documentation here for how to interpret this table.
| Files with missing lines | Coverage Δ | |
|---|---|---|
| ...s/workspace/members/WorkspaceMemberDetailsPage.tsx | 0.00% <0.00%> (ø) |
|
| src/pages/workspace/WorkspaceMembersPage.tsx | 75.31% <0.00%> (-0.97%) |
⬇️ |
... and 218 files with indirect coverage changes
Example 3
Current Format
Codecov Report
✅ All modified and coverable lines are covered by tests.
see 7 files with indirect coverage changes
🚀 New features to boost your workflow:
- ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
Desired Format
Codecov Report
✅ All modified and coverable lines are covered by tests.
see 7 files with indirect coverage changes
Issue Owner
Current Issue Owner: @abzokhattabMetadata
Metadata
Assignees
Labels
Type
Projects
Status