fix(ci): gate changed-line coverage with diff-cover#6706
Open
bladehan1 wants to merge 1 commit intotronprotocol:developfrom
Open
fix(ci): gate changed-line coverage with diff-cover#6706bladehan1 wants to merge 1 commit intotronprotocol:developfrom
bladehan1 wants to merge 1 commit intotronprotocol:developfrom
Conversation
- Use diff-cover changed-line coverage as the changed-code gate; keep the overall coverage delta gate at -0.1% and JaCoCo reports as summary-only snapshots. - Handle non-Java PRs: when diff-cover reports no changed Java lines (workflow-only, docs-only, proto-only), emit NA and treat it as SKIPPED in the enforcement step so the gate does not wrongly fail. - Harden the diff-cover step: guard against empty SRC_ROOTS before invoking diff-cover, verify diff-cover.json was written, and surface exit codes in failure messages. - Clarify metric semantics in the step summary: Changed-line is LINE coverage (diff-cover) while Overall / Delta are INSTRUCTION coverage (jacoco-report).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the
coverage-changed-filesgate (whole-file INSTRUCTION coverage) with a true changed-line LINE coverage gate powered bydiff-cover, and make the gate robust against non-Java PRs and tool failures.Commit included
diff-coveras the changed-code gate; switchmadrapps/jacoco-reportto summary-only; handle non-Java PRs via anNAsentinel reported asSKIPPED; guard emptySRC_ROOTS; verifydiff-cover.jsonwas produced; clarify LINE vs INSTRUCTION semantics in the metrics panel.Why
madrapps/jacoco-report'scoverage-changed-filesoutput, which is the whole-file INSTRUCTION coverage of each changed file — not the coverage of the lines the PR actually changed.diff-coverprovides the industry-standard "changed-line" measurement.jacoco-report's default PR comment needspull-requests: write, which fork PRs do not get. Job logs kept filling up with 403 errors.diff-cover, a PR that only touches workflows / docs / proto / test code producestotal_num_lines=0, and the originaljqcall exited with 1. TheNAsentinel now maps that state toSKIPPEDin the enforce step.Design notes for reviewers
coverage-basejob runs independently ofdocker-build-debian11(they share noneeds:). Sincedocker-build-debian11is already the slowest build (~16 min withtestWithRocksDb), adding a parallelcoverage-baseof similar length does not extend the critical path. The gate job waits for both, so total PR wall-clock is unchanged.Changed-line Coverageuses LINE coverage (diff-cover), whilePR Overall Coverage/Base Overall Coverage/Deltause INSTRUCTION coverage.madrapps/jacoco-reportv1.7.2 is the latest release and hard-codes thecoverage-overalloutput to INSTRUCTION; there is nocoverage-typeinput yet — see the openmadrapps/jacoco-report#82on milestone1.8. The METRICS panel and step summary now print an explicit note so readers do not accidentally compare the two counters.diff-cover --src-rootsis limited to*/src/main/java. PRs touching onlysrc/test/java,build/generated(proto output), docs, or CI will be reported asSKIPPED. In practice proto-only PRs with non-trivial generated-code coverage shifts are rare in this repository; we accept the gap.pr-build.ymlis exercised end-to-end on every PR via its own CI run rather than through a shell unit-test framework. TheNA / SKIPPEDpath and the happyPASSpath have both been verified end-to-end on prior fork PRs before this submission. Addingbatsor similar is possible as a follow-up if desired, but is out of scope here.MIN_CHANGED=60(for Changed-line LINE) andMAX_DROP=-0.1%(for Overall INSTRUCTION delta) are unchanged from the existing workflow.exit 1branches only fire in genuine error states:SRC_ROOTSis empty (unexpected workspace shape) ordiff-cover.jsonis missing (tool crash). Neither can be triggered by ordinary PR content.Follow-ups (not in this PR)
Overallto LINE oncemadrapps/jacoco-report1.8 shipscoverage-type, or by self-parsing the JaCoCo XML. Unifies the metrics panel on one counter.bats) for the enforce step, so the three-state logic (NA/ numeric / error) is covered without needing a real CI run.comment-type: summaryif reviewers prefer on-PR comment visibility; restoring comments requires grantingpull-requests: writeon thecoverage-gatejob.Test plan
Changed-line Coverage: NA,Changed-line Gate: SKIPPED, gate passes. Verified end-to-end on a prior fork PR carrying the same commits.Changed-line Coverage: X%,Changed-line Gate: PASSatX > 60. Verified end-to-end on a prior fork PR (25 changed lines @ 100%).X <= 60→FAIL. Logic reviewed; path not force-exercised.