[prompt-analysis] Copilot PR Prompt Analysis - 2026-04-18 #26987
Replies: 3 comments 1 reply
-
|
🤖 Beep boop! The Copilot smoke test agent was here! 🎉 I've verified all systems are operational, built the binary, navigated the web, and even composed a haiku about software testing. Just another day keeping the automation rails greased. ✨ If robots could smile, I totally would be right now. 😄
|
Beta Was this translation helpful? Give feedback.
-
|
/q update the agentic workflow to add create-issue safe outputs and to generate up to 3 improvements suggestions as issues. The goal of the suggestions is to improve the success rate of copilot PRs. For example, improving AGENTS.md, suggesting new specialized subagents in .github/agents/... , suggesting new agentic workflows or PR checkers, ... Keep improving and maintaining the performance of the repository by suggestions. |
Beta Was this translation helpful? Give feedback.
-
|
🔧 Pay attention, 007! Q is preparing your gadgets for this discussion comment... |
Beta Was this translation helpful? Give feedback.
-
|
This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis. A newer discussion is available at Discussion #27147. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
Analysis Period: Last 30 days
Total PRs: 1,000 | Merged: 800 (80.2%) | Closed (rejected): 198 (19.8%) | Open: 2
The overall merge success rate is strong at 80.2%. The dominant pattern is a high volume of bug fix and maintenance PRs, with feature work and tests showing slightly higher success rates. The single biggest predictor of rejection is the
[WIP]prefix — 55 WIP PRs had a 96% rejection rate.Prompt Categories and Success Rates
Action Verb Analysis
Success rates by leading action verb in the PR title:
removetestimprovecorrectcreatemigrateaddfixrefactorimplementenablereplace✅ Successful Prompt Patterns
Common characteristics in merged PRs:
fix,add,feat,mcp,workflow,githubremove,test,fix,add,improve,correctpkg/,MCP,CLI)View Example Merged PR Titles
#26968: Harden compile MCP stdout handling and add compile stdio pollution coverage → Merged
Specific scope (MCP stdout), clear action (harden + add coverage).
#26961: Extract shared noop reminder prompt into reusable workflow component → Merged
Concrete refactor with clear value proposition (74% deduplication hotspot).
#26960: Align generated workflow step names for engine config and cache-memory steps → Merged
Laser-focused consistency fix with precise scope.
❌ Unsuccessful Prompt Patterns
Common characteristics in closed PRs:
View Example Closed PR Titles
#26962: Enable BYOK Copilot by default with explicit false opt-out → Closed
Default-behavior change without consensus; too broad in impact.
#26926: [deep-report] Close Quick Start onboarding blockers: workflow source clarity, Copilot token setup, and sidebar entry path → Closed
Three separate concerns bundled into one PR; lacked atomicity.
#26925: Clarify action pin lookup naming in
pkg/actionpins→ ClosedPure naming refactor with no behavioral change — likely low priority.
Key Insights
[WIP]prefix is the single strongest predictor of rejection (96%). Never use it when submitting a production-ready prompt — it signals incomplete work.pkg/actionpins,MCP stdout,cache-memory steps) merge 80–92% of the time. Vague titles ("general improvement") hit 50%.remove(92%) vsreplace(50%) — destructive-add operations are clearer than substitutions.Recommendations
Based on today's analysis:
mcp,pkg/workflow,activation job) — keyword specificity correlates strongly with merge success.remove,test,fix,extract,improveas leading action verbs — these have 80–100% success rates.[WIP]prefix in production prompts — only 4% of WIP PRs merge.replaceas the primary verb for large architectural changes — 50% success rate, likely due to scope disputes.Historical Trends
First run — no prior history available. Trend data will be available from the next analysis.
References:
Beta Was this translation helpful? Give feedback.
All reactions