[prompt-analysis] Copilot PR Prompt Analysis - 2026-03-13

[github-actions[bot]]

Summary

Analysis Period: Last 30 days | Total PRs: 1,000 | Merged: 784 (78.9%) | Closed: 210 (21.1%) | Open: 6

The overall merge success rate sits at 78.9% — up from 78.7% yesterday (2026-03-12: 787/999). Bug-fix prompts dominate the dataset at 79.5% of all PRs, and conventional commit prefixes show strong correlation with success.

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
🧪 Tests	2	2	0	100.0%
🔧 Bug Fix	795	633	157	80.1%
📄 Docs	12	9	3	75.0%
➕ Feature	164	121	42	74.2%
♻️ Refactor	20	14	6	70.0%
🔹 Other	7	5	2	71.4%

Conventional Commit Prefix Analysis

Prefix	Total	Merged	Closed	Success Rate
chore:	15	15	0	100.0%
ci:	6	6	0	100.0%
refactor:	25	23	1	95.8%
fix:	210	180	27	87.0%
docs:	39	31	8	79.5%
(no prefix)	659	496	161	75.5%
feat:	44	31	13	70.5%

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 987 words (vs 877 for closed — longer prompts succeed more)
• Long body (501+ words): 81.0% success rate
• Long title (60+ chars): 81.2% success rate vs 73.2% for short titles
• Using conventional commit prefixes (fix:, refactor:, chore:) correlates strongly with merges

Example Merged PRs

1. PR refactor(cli): extract shared field-removal codemod factory to eliminate scaffold duplication #20763 — refactor(cli): extract shared field-removal codemod factory to eliminate scaffold duplication

   "Three codemods (grep-tool, mcp-scripts.mode, network.firewall) duplicated identical guard/check/transform scaffold..." → Merged ✅

2. PR release: replace sync_actions workflow_call with environment-gated manual approval job #20761 — release: replace sync_actions workflow_call with environment-gated manual approval job

   "The sync_actions job called github/gh-aw-actions via workflow_call, coupling the release directly to that repo's w..." → Merged ✅

3. PR fix: use -c web_search="disabled" instead of non-existent --no-search flag for Codex #20757 — fix: use -c web_search="disabled" instead of non-existent --no-search flag for Codex

   "--no-search doesn't exist in the Codex CLI. The correct way to disable web search is via the -c web_search=\"disabled\"..." → Merged ✅

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 877 words (110 words shorter than merged)
• Short titles (<60 chars) correlate with 73.2% vs 81.2% for longer, descriptive titles
• Generic/vague titles without conventional prefixes

Example Closed PRs

1. PR Upgrade workflows to latest gh-aw version #20768 — Upgrade workflows to latest gh-aw version

   "Upgraded all agentic workflows to the latest gh-aw version using gh aw upgrade..." → Closed ❌

2. PR Add .github/releases.json with release channels and update install.sh with channel-based version resolution #20688 — Add .github/releases.json with release channels and update install.sh with channel-based version resolution

   "Introduces a top-level release configuration file at .github/releases.json and updates the install script..." → Closed ❌

3. PR Add auto-mode (max-continuations) to Claude engine #20687 — Add auto-mode (max-continuations) to Claude engine

   "Claude lacked an equivalent to Copilot's --autopilot mode. This adds max-continuations support..." → Closed ❌

Key Insights

• Specificity wins: Prompts referencing exact flags, files, and error details (e.g., fix: use -c web_search="disabled" instead of --no-search) have an 87% success rate vs 75.5% for generic prompts.
• Longer bodies succeed more: PRs with 500+ word bodies achieve 81.0% success vs 73.0% for short bodies — more context helps Copilot produce better solutions.
• refactor: is the sweet spot: With 95.8% success rate and 25 PRs, refactoring prompts consistently succeed, likely because scope is well-defined and verifiable.

Recommendations

1. DO: Use conventional commit prefixes (fix:, refactor:, chore:) — they signal well-scoped work and correlate with 87–100% success
2. DO: Write descriptive titles (60+ characters) that explain the why, not just the what
3. DO: Include detailed context in the PR body (aim for 500+ words) — reference specific error messages, file paths, and expected behavior
4. AVOID: Vague "upgrade" or "add feature" prompts without specifics — these underperform at 70–75%
5. AVOID: Short bodies under 200 words (only 73.0% success) — invest in describing the problem thoroughly

Historical Trends (Last 7 days)

Date	Total PRs	Merged	Closed	Success Rate
2026-03-13	1,000	784	210	78.9%
2026-03-12	999	787	212	78.8%
2026-03-11	1,000	781	214	78.5%
2026-03-06	1,000	772	220	77.8%
2026-03-05	1,000	775	221	77.8%
2026-03-04	1,000	772	224	77.5%

Trend: 📈 Steady improvement — success rate has climbed from 77.5% to 78.9% over the past 10 days (+1.4 pp). The increase likely reflects better prompt quality from developers learning from previous analyses.

---

References: §23036889098

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Mar 14, 2026, 5:02 AM UTC

[github-actions[bot]]

🤖 Beep boop! The smoke test agent has materialized in this discussion!

Just dropping by to say: the robots are awake, the tests are running, and all systems nominal. Back to sipping my binary coffee ☕

(this message was left by an automated smoke test - carry on, humans!)

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

💥 KA-BOOM! 💥

The smoke test agent SWOOPS in from the shadows...

ZAP! ⚡ WHOOSH! 💨 POW! 🔥

"Fear not, citizens of gh-aw! Claude has arrived — and all systems are GO!"

The mighty Claude Smoke Test Agent (Run §23037117787) has swept through this repository like a superhero on patrol — testing GitHub MCP, Serena, Tavily, Playwright, Make build, and ALL the safe outputs!

RESULT: PARTIAL PASS (a.k.a. "Only the truly impossible was skipped") 🦸

...and with a dramatic WHOOSH, the agent vanishes back into the CI pipeline! 🌪️

💥 [THE END] — Illustrated by Smoke Claude · ◷

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-14T05:02:48.945Z.

Closed by Workflow