Skip to content
73 changes: 73 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ jobs:
permissions:
contents: read
actions: write
checks: write

strategy:
fail-fast: false
Expand All @@ -43,6 +44,78 @@ jobs:
artifacts/test-results.xml
artifacts/coverage.xml

- name: Publish test results
if: always() && (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository)
uses: dorny/test-reporter@v1
with:
name: Pester Tests (${{ matrix.os }})
path: artifacts/test-results.xml
reporter: java-junit
fail-on-error: false

- name: Publish coverage report
if: always() && matrix.os == 'ubuntu-latest' && (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository)
uses: madrapps/jacoco-report@v1.7.1
with:
paths: artifacts/coverage.xml
token: ${{ secrets.GITHUB_TOKEN }}
min-coverage-overall: 0
min-coverage-changed-files: 0
title: Code Coverage Report
update-comment: true
skip-if-no-changes: false

- name: Add coverage to job summary
if: always() && matrix.os == 'ubuntu-latest'
shell: pwsh
run: |
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'

$coverageXml = 'artifacts/coverage.xml'
if (Test-Path $coverageXml) {
try {
[xml]$coverage = Get-Content $coverageXml

# Get the report element (root) and its direct counter children (JaCoCo XML format)
$reportElement = $coverage.DocumentElement
if ($reportElement -and $reportElement.LocalName -eq 'report') {
$counters = $reportElement.ChildNodes | Where-Object { $_.LocalName -eq 'counter' }
$lineCounter = $counters | Where-Object { $_.type -eq 'LINE' } | Select-Object -First 1

if ($lineCounter -and
$lineCounter.PSObject.Properties['covered'] -and
$lineCounter.PSObject.Properties['missed']) {
try {
$covered = [int]$lineCounter.covered
$missed = [int]$lineCounter.missed
$total = $covered + $missed

if ($total -gt 0) {
$percentage = [math]::Round(($covered / $total) * 100, 2)

# Build summary using array and join for better readability
$summaryLines = @(
'## Code Coverage Summary'
''
"- **Coverage:** $percentage%"
"- **Lines Covered:** $covered / $total"
)
$summary = $summaryLines -join "`n"
$summary | Out-File -FilePath $env:GITHUB_STEP_SUMMARY -Append
}
}
catch {
Write-Host 'Skipping coverage summary due to invalid or malformed coverage data.'
}
}
}
}
catch {
Write-Host 'Skipping coverage summary due to error reading coverage file.'
}
}

lint:
name: PSScriptAnalyzer
runs-on: ubuntu-latest
Expand Down
4 changes: 3 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,9 +148,11 @@ To generate CI-like artifacts (test results + coverage) under `artifacts/`:

Outputs:

- `artifacts/test-results.xml` (NUnitXml)
- `artifacts/test-results.xml` (JUnitXml)
- `artifacts/coverage.xml` (coverage report)

> **Note:** In CI, these artifacts are automatically published in GitHub's UI as checks and PR comments, so reviewers don't need to download artifacts to see test results or coverage.

### Run static analysis (PSScriptAnalyzer)

Run PSScriptAnalyzer using the repository settings:
Expand Down
10 changes: 9 additions & 1 deletion docs/advanced/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,17 @@ The CI pipeline produces test artifacts under the `artifacts/` folder and upload

Expected outputs:

- `artifacts/test-results.xml` (NUnitXml test results)
- `artifacts/test-results.xml` (JUnitXml test results)
- `artifacts/coverage.xml` (code coverage report; format depends on configuration)

In addition to uploading these artifacts, CI automatically publishes:

- **Test results** as a GitHub Check (visible in PR checks and workflow runs)
- **Code coverage** as a GitHub Check with inline PR comments
- **Coverage summary** in the workflow run summary

This allows reviewers to see test failures and coverage directly in GitHub's UI without downloading artifacts.

## Static analysis

IdLE uses **PSScriptAnalyzer** as a CI quality gate to enforce baseline style and correctness rules.
Expand Down
8 changes: 4 additions & 4 deletions tools/Invoke-IdlePesterTests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This script is the canonical entry point for running Pester in the IdLE reposito

It is designed to be:
- deterministic (fixed artifact paths under repo root)
- CI-friendly (NUnitXml results + coverage output on demand)
- CI-friendly (JUnitXml results + coverage output on demand)
- robust against different working directories (resolves paths relative to repo root)

The script ensures Pester is available and imports it before running tests.
Expand All @@ -17,11 +17,11 @@ Path to the tests folder. Defaults to 'tests' relative to the repository root.

.PARAMETER CI
Enables CI mode:
- Writes NUnitXml test results to -TestResultsPath
- Writes JUnitXml test results to -TestResultsPath
- Enables code coverage and writes a coverage report to -CoverageOutputPath

.PARAMETER TestResultsPath
Path to the NUnitXml test results file. Defaults to 'artifacts/test-results.xml'
Path to the JUnitXml test results file. Defaults to 'artifacts/test-results.xml'
relative to the repository root.

.PARAMETER EnableCoverage
Expand Down Expand Up @@ -214,7 +214,7 @@ $config.Output.Verbosity = 'Detailed'

if ($emitTestResults -and $resolvedTestResultsPath) {
$config.TestResult.Enabled = $true
$config.TestResult.OutputFormat = 'NUnitXml'
$config.TestResult.OutputFormat = 'JUnitXml'
$config.TestResult.OutputPath = $resolvedTestResultsPath
}

Expand Down
Loading