Skip to content

Add test-coverage-detector workflow (detector/fixer pair for test-improver)#488

Merged
strawgate merged 8 commits intomainfrom
copilot/create-detector-version-test-improver
Feb 28, 2026
Merged

Add test-coverage-detector workflow (detector/fixer pair for test-improver)#488
strawgate merged 8 commits intomainfrom
copilot/create-detector-version-test-improver

Conversation

Copy link
Contributor

Copilot AI commented Feb 28, 2026

Adds a scheduled Test Coverage Detector workflow that reports high-confidence coverage gaps and serves as the detector counterpart to test-improver.

Detector workflow

  • Runs coverage tools when available; falls back to code analysis when coverage tooling is unavailable.
  • Reviews 28-day git history to find recent code changes without corresponding test updates.
  • Uses Pick Three, Keep One parallel investigation with sub-agents during gap discovery.
  • Applies a high bar for reporting and defaults to noop unless gaps are concrete, user-scenario-backed, and actionable.
  • Safe output: create-issue (max 1, expires in 7d).

New files

  • .github/workflows/gh-aw-test-coverage-detector.md — workflow source (scheduled-audit based)
  • .github/workflows/gh-aw-test-coverage-detector.lock.yml — compiled workflow lock file
  • gh-agent-workflows/test-coverage-detector/ — workflow README + trigger example
  • docs/workflows/gh-agent-workflows/test-coverage.md — detector/fixer pair documentation
  • .github/workflows/trigger-test-coverage-detector.yml — dogfood trigger workflow

Updated references

  • Added Test Coverage to the detector/fixer pair docs and MkDocs navigation.
  • Added test-coverage-detector to workflow catalogs (gh-agent-workflows/README.md) and setup tooling (scripts/quick-setup.sh).
  • Added detector triggering in CI helper scripts and smoke-test install matrix.
  • Added pairing cross-references in both workflow READMEs.

Quick install

# Detector only
mkdir -p .github/workflows && curl -sL \
  https://raw.githubusercontent.com/elastic/ai-github-actions/v0/gh-agent-workflows/test-coverage-detector/example.yml \
  -o .github/workflows/test-coverage-detector.yml

# Full loop (detector + fixer)
mkdir -p .github/workflows && \
curl -sL https://raw.githubusercontent.com/elastic/ai-github-actions/v0/gh-agent-workflows/test-coverage-detector/example.yml \
  -o .github/workflows/test-coverage-detector.yml && \
curl -sL https://raw.githubusercontent.com/elastic/ai-github-actions/v0/gh-agent-workflows/test-improver/example.yml \
  -o .github/workflows/test-improver.yml

Generated by Update PR Body for issue #488

Copilot AI and others added 2 commits February 28, 2026 20:50
Create a new detector workflow that identifies under-tested code paths and
files issue reports, forming a detector/fixer pair with the existing
test-improver workflow.

New files:
- .github/workflows/gh-aw-test-coverage-detector.md (workflow source)
- .github/workflows/gh-aw-test-coverage-detector.lock.yml (compiled)
- .github/workflows/trigger-test-coverage-detector.yml (dogfood trigger)
- gh-agent-workflows/test-coverage-detector/ (README + example.yml)
- docs/workflows/gh-agent-workflows/test-coverage.md (grouped docs)

Updated files:
- docs/workflows/gh-agent-workflows.md (detector/fixer pair table)
- mkdocs.yml (nav structure)
- gh-agent-workflows/README.md (detector/fixer lists)
- gh-agent-workflows/test-improver/README.md (pairing info)
- scripts/trigger-ci-workflows.sh (DETECTORS array)
- scripts/quick-setup.sh (continuous-improvement list)
- .github/workflows/smoke-test-install.yml (install matrix)

Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>
Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>
Copilot AI changed the title [WIP] Create detector version of test-improver Add test-coverage-detector workflow (detector/fixer pair for test-improver) Feb 28, 2026
@strawgate strawgate marked this pull request as ready for review February 28, 2026 21:11
@coderabbitai
Copy link

coderabbitai bot commented Feb 28, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 340573f and fbc6c72.

📒 Files selected for processing (2)
  • docs/workflows/gh-agent-workflows.md
  • scripts/quick-setup.sh

📝 Walkthrough

Walkthrough

This pull request adds a new Test Coverage Detector workflow (generated lockfile and human-readable workflow), a reusable trigger workflow, example and trigger files, and supporting documentation and README entries. It integrates the detector into installation and CI scripts (quick-setup, smoke-test-install, trigger scripts), updates navigation (mkdocs.yml), and includes docs and examples for pairing the detector with the Test Improver. The detector workflow is a multi-job Copilot-driven automation producing artifacts (prompts, agent outputs, safe-outputs, threat-detection logs) and implements activation, agent execution, safe outputs, threat detection, conclusion, and cleanup flows.

Possibly related PRs

  • elastic/ai-github-actions PR 312: Adds workflow/fragments and tooling changes that align with introducing the Test Coverage Detector workflow and related fragments.
  • elastic/ai-github-actions PR 173: Modifies quick-setup and related installer wiring that this change extends by adding the test-coverage-detector to defaults.
  • elastic/ai-github-actions PR 335: Adjusts generated workflow generation and docs for lock/trigger files, which overlap with this PR’s added generated lock and trigger workflows.
🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR successfully implements the detector version of test-improver (#483), including workflow definitions, documentation, and integration into existing tooling.
Out of Scope Changes check ✅ Passed All changes directly support the test-coverage-detector feature. Updates to documentation, scripts, and workflows are aligned with adding and integrating the new detector.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch copilot/create-detector-version-test-improver

Comment @coderabbitai help to get the list of available commands and usage tips.

| [Code Duplication Detector](gh-agent-workflows/code-duplication-detector.md) | [Code Duplication Fixer](gh-agent-workflows/code-duplication-fixer.md) | Duplicate / clustered code |
| [Docs Patrol](gh-agent-workflows/docs-patrol.md) | — | Stale internal documentation |
| [Newbie Contributor Patrol](gh-agent-workflows/newbie-contributor-patrol.md) | [Newbie Contributor Fixer](gh-agent-workflows/newbie-contributor-fixer.md) | Onboarding documentation gaps |
| [Test Coverage Detector](gh-agent-workflows/test-coverage-detector.md) | [Test Improver](gh-agent-workflows/test-improver.md) | Test coverage gaps |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Broken docs link for Test Coverage detector/fixer pair

This row links to gh-agent-workflows/test-coverage-detector.md, but that page does not exist in this PR (mkdocs.yml and docs add workflows/gh-agent-workflows/test-coverage.md). The current link will 404 in rendered docs.

Suggested change
| [Test Coverage Detector](gh-agent-workflows/test-coverage-detector.md) | [Test Improver](gh-agent-workflows/test-improver.md) | Test coverage gaps |
| [Test Coverage Detector](gh-agent-workflows/test-coverage.md) | [Test Improver](gh-agent-workflows/test-improver.md) | Test coverage gaps |

@strawgate
Copy link
Collaborator

/ai please fix the broken link

github-actions bot and others added 2 commits February 28, 2026 21:31
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@strawgate strawgate closed this Feb 28, 2026
@strawgate strawgate reopened this Feb 28, 2026
strawgate and others added 2 commits February 28, 2026 15:54
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@strawgate strawgate merged commit 392a595 into main Feb 28, 2026
11 of 13 checks passed
@strawgate strawgate deleted the copilot/create-detector-version-test-improver branch February 28, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create detector version of test-improver

2 participants