Agent Persona Exploration - 2026-03-21

[github-actions[bot]]

Persona Overview

• Agent: developer.instructions (agentic-workflows proxy)
• Scenarios Tested: 6 (BE-1, FE-2, DO-1, DO-2, QA-1, PM-1)
• Average Quality Score: 4.77/5.0 — highest avg across all exploration runs
• Engine Selection: 100% Claude (first run with unanimous engine recommendation)

Key Findings

• Claude universally recommended across all 6 diverse scenarios — previous runs mixed Copilot for simpler tasks; this run favored Claude even for straightforward PR comment workflows
• Security posture strongest yet: QA-1 correctly used allowed: [] (hermetic Go tests need no network); PM-1 used egress: false (strictest valid config)
• Safe-outputs UX is maturing: hide-older-comments: true, close-older-discussions: true, and max: 1 caps appeared consistently, improving stakeholder UX
• Three frontmatter syntax errors surfaced: agent: instead of engine:, mcp-servers: instead of tools:, and repo-memory (non-existent) instead of cache-memory
• Prompt engineering quality is high: Risk thresholds, field-level categorization, and secret-redaction instructions are production-grade details

Top Patterns

1. Triggers: pull_request + paths filter (3/6), schedule + workflow_dispatch (2/6), workflow_run + if:conclusion==failure (1/6)
2. Tools: GitHub MCP pull_requests+repos toolsets dominant; bash for local toolchains; playwright for browser automation
3. Security: sandbox: agent: awf mentioned; network locked to minimum (allowed:[], egress:false, or domain-specific)

View Scenario Scores (6 tested)

ID	Scenario	Trigger	Tools	Security	Prompt	Complete	Avg
BE-1	API breaking change review	5	5	5	5	5	5.0
DO-1	Deployment failure incident reporter	5	5	5	5	5	5.0
QA-1	Go test coverage delta	5	5	5	5	5	5.0
PM-1	Weekly release digest	5	4	5	5	5	4.8
FE-2	Accessibility audit (React/axe-core)	5	5	4	4	4	4.4
DO-2	Weekly Dependabot vulnerability report	5	4	5	4	4	4.4

View High Quality Responses (Top 3)

BE-1 — API Breaking Change Reviewer (5.0/5)

• Path-filtered PR trigger scoped to OpenAPI/schema files
• Differentiates REQUEST_CHANGES vs COMMENT based on severity
• Caps inline comments at 20 to force prioritization
• Explicitly flags prompt injection risk from PR diff contents

DO-1 — Deployment Failure Incident Reporter (5.0/5)

• workflow_run trigger with if: conclusion == 'failure' job guard
• Includes secret-redaction instruction in the prompt body
• Practical tip about reusable workflows requiring nested run lookup
• Structured incident report format (summary → root cause → affected services → remediation)

QA-1 — Go Test Coverage Delta (5.0/5)

• allowed: [] (empty network) — correct for hermetic Go tests
• Quantitative risk thresholds: >5% drop = HIGH, 1–5% = MEDIUM, ≥0% = LOW
• hide-comment UX pattern to avoid duplicate comments on re-runs
• Correctly flags 0% coverage on changed files (not just total regression)

View Areas for Improvement

FE-2 — Accessibility Audit

• network.allowed uses npm shorthand — unclear if valid in gh-aw syntax (should be registry.npmjs.org)
• Assumes Storybook exists; no fallback guidance for non-Storybook projects
• Score: 4.4/5

DO-2 — Dependabot Vulnerability Report

• Frontmatter uses agent: claude instead of the correct engine: claude
• Uses mcp-servers: with tool-sets: instead of standard tools: with toolsets:
• Score: 4.4/5

PM-1 — Weekly Release Digest

• References repo-memory tool which does not exist in gh-aw; cache-memory is the correct standard tool
• Score: 4.8/5 (otherwise excellent)

Recommendations

1. Document cache-memory as the canonical deduplication tool — repo-memory appeared as a hallucination; the AGENTS.md and skills should reinforce cache-memory with concrete examples for scheduled workflows
2. Add network shorthand canonicalization — clarify which values are valid in network.allowed (github, npm, localhost vs full domains like registry.npmjs.org)
3. Add frontmatter field name validation examples — engine: not agent:, tools: not mcp-servers:, toolsets: not tool-sets: — these syntax errors were 3 of the 4 issues found

References: §23368937227

AI generated by Agent Persona Explorer · history

[pelikhan]

/plan

[github-actions[bot]]

🚀 Plan Command has started processing this discussion comment

[github-actions[bot]]

This discussion has been marked as outdated by Agent Persona Explorer.

A newer discussion is available at Discussion #22224.