Add mobile platform failure scanner agentic workflow by kotlarmilos · Pull Request #126952 · dotnet/runtime

kotlarmilos · 2026-04-15T17:03:07Z

Description

This adds a daily scheduled agentic workflow that scans the runtime-extra-platforms https://dev.azure.com/dnceng-public/public/_build?definitionId=154 for Apple mobile (iOS, tvOS, MacCatalyst) and Android failures on main.

What it does

Daily scan: queries the latest completed build, finds failed mobile jobs, reads failure logs
Triage: classifies failures as infrastructure (reports on tracking issues with machine details) or code (traces to recent commits)
Fix: opens draft PRs for code failures, caps at 2 PRs, 2 issues, 5 comments per run

Files

File	Purpose
`.github/skills/mobile-platforms/SKILL.md`	Domain knowledge: CI pipeline structure, code paths, failure triage, platform gotchas
`.github/workflows/mobile-scan.md`	Daily scan workflow (schedule trigger, write-access gated)
`.github/workflows/mobile-scan.lock.yml`	Compiled lock file

Safety & permissions

The workflow source (.md) declares read-only permissions (contents: read, issues: read, pull-requests: read). The agent job itself has no direct write access to the repository.

All write operations (creating draft PRs, issues, and comments) go through the safe-outputs boundary, which runs in a separate job with its own scoped permissions. Safe-outputs enforces strict caps:

Output	Cap	Constraints
Draft PRs	max 2	Title prefix `[mobile]` , labels `agentic-workflows`, protected files/paths fallback to issue
Issues	max 2	Labels `agentic-workflows`, `untriaged`
Comments	max 5	On existing issues/PRs only

Additional safeguards:

Only admins, maintainers, and contributors with write access can trigger via workflow_dispatch
External skill content (helix-investigation) is fetched pinned to a specific commit SHA, not a mutable branch
Log excerpts are sanitized before posting (no secrets, tokens, PII)
Concurrency group prevents overlapping runs
Network allowlist restricts outbound access to Azure DevOps, Helix, GitHub, and Helix blob storage only

Copilot

Pull request overview

Adds a new GitHub Agentic Workflow to automatically scan Azure DevOps “runtime-extra-platforms” (definition 154) for Apple mobile/Android failures, triage them, and produce bounded remediation outputs (issues/PRs/comments). Also introduces a new mobile triage skill document and the compiled workflow lockfile needed to run the workflow in production.

Changes:

Add a scheduled “Mobile Platform Failure Scanner” agentic workflow definition (mobile-scan.md) plus its compiled lock workflow (mobile-scan.lock.yml).
Add a new domain skill document for mobile-platform CI triage and fixes (.github/skills/mobile-platforms/SKILL.md).
Update the Agentic Workflows action lock entries and include an unrelated test change in System.Diagnostics.FileVersionInfo.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/libraries/System.Diagnostics.FileVersionInfo/tests/System.Diagnostics.FileVersionInfo.Tests/FileVersionInfoTest.cs`	Small platform check tweak (note: unrelated to the PR’s stated workflow purpose).
`.github/workflows/mobile-scan.md`	New agentic workflow source: schedule + prompt/instructions + safe-output caps.
`.github/workflows/mobile-scan.lock.yml`	Generated compiled workflow to execute in GitHub Actions.
`.github/skills/mobile-platforms/SKILL.md`	New skill doc capturing mobile CI pipeline structure and triage guidance.
`.github/aw/actions-lock.json`	Updates action lock entries used by agentic workflows.

Use OperatingSystem.IsWindows() instead of PlatformDetection.IsWindows for the OriginalTestAssemblyInternalName field initializer. On Apple mobile CI, EnableAggressiveTrimming=true causes PlatformDetection.IsWindows (which chains through RuntimeInformation.IsOSPlatform) to be incorrectly evaluated during cross-compilation trimming. OperatingSystem.IsWindows() is a compile-time constant in the BCL, immune to trimmer mis-evaluation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add a daily scheduled agentic workflow that scans the runtime-extra-platforms pipeline (AzDO definition 154) for Apple mobile and Android test/build failures. The agent triages failures as infrastructure or code, reports infrastructure issues on tracking issues with machine details, and opens draft PRs for code fixes. Files added: - .github/skills/mobile-platforms/SKILL.md: domain knowledge covering CI pipeline structure, code paths, failure triage, platform gotchas, and self-improvement guidance - .github/workflows/mobile-scan.md: daily scan workflow with safe-output caps (2 PRs, 2 issues, 5 comments) - .github/workflows/mobile-scan.lock.yml: compiled lock file Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Fix BUILD_ID capture and timeline API URL - Rename concurrency group to mobile-scan - Add network allowlist for dev.azure.com and helix.dot.net - Add known build error check before investigating - Reference ci-analysis skill for structured failure data - Align self-improvement guidance between workflow and skill - Skip PR creation if a fix PR already exists for the issue Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

- Move roles under on: so schedule trigger runs without role check - Delegate CI failure analysis to ci-analysis skill (Get-CIStatus.ps1) - Add pwsh to bash tool allowlist for ci-analysis script - Update self-improvement guidance: record learnings on issues instead of modifying SKILL.md in fix PRs (protected by safe-outputs) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

kotlarmilos · 2026-04-16T09:17:58Z

I don't have permissions to create/update .github/workflows. Could someone push it to the origin repo (instead of my fork) so I can run it manually to verify?

kotlarmilos · 2026-04-17T08:44:29Z

I don't have permissions to create/update .github/workflows. Could someone push it to the origin repo (instead of my fork) so I can run it manually to verify?

/cc: @danmoseley @vitek-karas @JanKrivanek Anyone with permissions to change workflows?

Port the net changes from #126952 onto a direct runtime branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

YAML parsed '!github.event.repository.fork' as a tag. Replace with 'github.event.repository.fork == false' to avoid the leading '!'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Port the net changes from #126952 onto a direct runtime branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…PR base) Squash of iterations from upstream/pr-126952-mobile-scan: - drill into Helix logs and auto-fix platform-unsupported tests - expand bash allowlist; persist state to files to avoid shell guard - allow blob storage; use script-file pattern for $(...) - branch from origin/main to avoid protected-file fallback on patch - require os-* and area-* labels on PRs/issues Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

kotlarmilos · 2026-04-17T13:43:33Z

Generated PRs: #127081 and #127082

steveisok · 2026-04-17T19:40:05Z

@kotlarmilos I think this should try to incorporate the "CI Investigator" agent and skills from https://github.com/dotnet/arcade-skills/tree/main/plugins/dotnet-dnceng. I also think we should not hold ourselves to just mobile.

- Load helix-investigation skill from dotnet/arcade-skills in Step 1 - Replace inline Helix log fetching (Step 5) with delegation to the skill - Add github ecosystem to network allowlist for raw.githubusercontent.com Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

- Expand SKILL.md description with concrete trigger phrases - Resolve known-build-error contradiction between skill and workflow - Reframe 'do not' rules as reasoning-led guidance - Soften 3-day git log heuristic to allow widening - Remove duplicate PR search between Steps 6 and 8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

kotlarmilos · 2026-04-20T15:07:26Z

@steveisok Added reference to the "CI Investigator". Regarding non-mobile platforms, I suggest doing that once we've confirmed it's useful and well constrained without adding noise.

Copilot AI review requested due to automatic review settings April 15, 2026 17:03

kotlarmilos requested review from a team and jeffhandley as code owners April 15, 2026 17:03

github-actions Bot added the area-skills Agent Skills label Apr 15, 2026

dotnet-policy-service Bot assigned kotlarmilos Apr 15, 2026

Copilot started reviewing on behalf of kotlarmilos April 15, 2026 17:04 View session

kotlarmilos requested review from JanKrivanek, PureWeen, danmoseley, matouskozak and vitek-karas April 15, 2026 17:04

Copilot AI reviewed Apr 15, 2026

View reviewed changes

kotlarmilos and others added 2 commits April 15, 2026 19:07

kotlarmilos force-pushed the apple-mobile-agentic-workflows branch from d13c49e to 3815708 Compare April 15, 2026 17:08

Revert

23cbc53

Copilot AI review requested due to automatic review settings April 15, 2026 17:11

Copilot started reviewing on behalf of kotlarmilos April 15, 2026 17:12 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/skills/mobile-platforms/SKILL.md

Comment thread .github/workflows/mobile-scan.md

JanKrivanek approved these changes Apr 15, 2026

View reviewed changes

vitek-karas reviewed Apr 16, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md Outdated

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md Outdated

kotlarmilos and others added 2 commits April 16, 2026 10:44

Add workflow_dispatch trigger for manual testing

56d7790

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 16, 2026 08:50

Copilot started reviewing on behalf of kotlarmilos April 16, 2026 08:52 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md Outdated

Comment thread .github/workflows/mobile-scan.lock.yml Outdated

vitek-karas added a commit that referenced this pull request Apr 17, 2026

Add mobile platform failure scanner workflow

9b59c17

Port the net changes from #126952 onto a direct runtime branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix invalid YAML tag in mobile-scan workflow if condition

6dee47a

YAML parsed '!github.event.repository.fork' as a tag. Replace with 'github.event.repository.fork == false' to avoid the leading '!'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 17, 2026 10:00

Copilot started reviewing on behalf of kotlarmilos April 17, 2026 10:01 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

Comment thread .github/skills/mobile-platforms/SKILL.md

Comment thread .github/skills/mobile-platforms/SKILL.md

Comment thread .github/workflows/mobile-scan.md Outdated

github-actions Bot added a commit that referenced this pull request Apr 17, 2026

Add mobile platform failure scanner workflow

f593315

Port the net changes from #126952 onto a direct runtime branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 20, 2026 06:48

Copilot started reviewing on behalf of kotlarmilos April 20, 2026 06:49 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md Outdated

Comment thread .github/workflows/mobile-scan.md

kotlarmilos and others added 3 commits April 20, 2026 09:05

Add PR search to triage step to reference existing fix PRs

a87d503

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Use tee instead of -o/> to avoid shell guard blocking output redirection

4732b4c

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Pin arcade-skills SKILL.md fetch to commit SHA for supply chain safety

143b845

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 20, 2026 08:25

Copilot started reviewing on behalf of kotlarmilos April 20, 2026 08:27 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md

Update .github/workflows/mobile-scan.md

c678f0e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 20, 2026 10:03

Copilot started reviewing on behalf of kotlarmilos April 20, 2026 10:04 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md

Comment thread .github/workflows/mobile-scan.md

kotlarmilos merged commit 6a1c436 into dotnet:main Apr 20, 2026
37 of 38 checks passed

dotnet-maestro Bot mentioned this pull request Apr 21, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#6162

Merged

Conversation

kotlarmilos commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What it does

Files

Safety & permissions

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kotlarmilos commented Apr 16, 2026

Uh oh!

kotlarmilos commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kotlarmilos commented Apr 17, 2026

Uh oh!

steveisok commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kotlarmilos commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

kotlarmilos commented Apr 15, 2026 •

edited

Loading