[prompt-analysis] Copilot PR Prompt Analysis - 2026-03-21

[github-actions[bot]]

Summary

Analysis Period: Last 30 days | Data snapshot: 1,000 most recent Copilot PRs (copilot/* branches)
Total PRs: 1,000 | Merged: 821 (82.3%) | Closed: 177 (17.7%) | Open: 2

Overall merge rate is 82.3%, up from 81.9% yesterday and 78.4% one week ago — showing a clear improving trend.

---

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Documentation	217	187	86.2% ✅
Bug Fix	723	605	83.7% ✅
Testing	664	550	82.8%
Refactoring	463	382	82.5%
Feature Addition	898	735	81.8%
Other	12	10	83.3%

Note: PRs are multi-categorized (e.g., "fix a bug and add a test" counts in both bug_fix and testing). Totals exceed dataset size.

---

Title Prefix Success Rates

Conventional commit prefixes correlate strongly with merge success:

Prefix	Merged / Total	Success Rate
ci:	7/7	100% 🏆
perf:	4/4	100% 🏆
chore:	19/20	95.0% ✅
refactor:	37/42	88.1% ✅
fix:	351/401	87.5% ✅
docs:	42/49	85.7% ✅
update:	18/21	85.7% ✅
add:	93/111	83.8%
improve:	14/17	82.4%
feat:	51/67	76.1%
remove:	16/23	69.6% ⚠️
(no prefix)	165/232	71.1% ⚠️

---

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average PR body length: 904 words
• Most common action verbs: test, run, add, check, set, fix, update
• File references present in 85.6% of merged PRs
• WIP markers ([WIP], wip) in only 2.2% of merged PRs

Example Merged PR Titles

1. PR fix: add github.event_name to JS allowed expressions list #22084: fix: add github.event_name to JS allowed expressions list → Merged
2. PR chore: remove trivy #22065: chore: remove trivy → Merged
3. PR Remove session management from safe outputs MCP HTTP server #22056: Remove session management from safe outputs MCP HTTP server → Merged
4. PR fix: cache action inputs in actions-lock.json for deterministic smoke-codex compilation #22048: fix: cache action inputs in actions-lock.json for deterministic smoke-codex compilation → Merged
5. PR docs: add integrity.md reference documentation #22044: docs: add integrity.md reference documentation → Merged

---

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average PR body length: 937 words (slightly longer than merged — verbosity ≠ quality)
• WIP markers present in 20.9% of closed PRs (vs 2.2% for merged) — 9.5× higher rate
• PRs without conventional commit prefixes: 37.9% of closed (vs 20.1% of merged)

Example Closed PR Titles

1. PR Fix MCP gateway session expiry for long-running workflows via SSE keepalive #22042: Fix MCP gateway session expiry for long-running workflows via SSE keepalive → Closed
2. PR Add data-science.md prompt for chart/trend workflow generation #22032: Add data-science.md prompt for chart/trend workflow generation → Closed
3. PR Support .lock.yaml files as compiled workflow output #22030: Support .lock.yaml files as compiled workflow output → Closed
4. PR [WIP] Fix create-pull-request signed commits failure for new branches #22008: [WIP] Fix create-pull-request signed commits failure for new branches → Closed
5. PR Warn when GitHub App safe-outputs can self-cancel comment-triggered workflows via shared concurrency #21979: Warn when GitHub App safe-outputs can self-cancel comment-triggered workflows via shared concurrency → Closed

---

Key Insights

• WIP is the strongest signal of failure: PRs with [WIP] or wip in the title are closed at a 9.5× higher rate (20.9% of closed vs 2.2% of merged). Never submit a prompt asking Copilot to create a WIP PR.
• Conventional commit prefixes significantly improve success: PRs without a recognized prefix (fix/feat/chore/docs/refactor/etc.) have a 71.1% success rate vs 85%+ for prefixed PRs. The top performers are ci:, perf:, and chore:.
• Verbosity doesn't help: Closed PR bodies average 937 words vs 904 for merged. Clarity and specificity matter more than length.
• Documentation prompts outperform all others at 86.2% — likely because the scope is well-defined and changes are easier to validate.
• Feature prompts underperform at 81.8% — feature requests may be more complex, have dependencies, or require design discussion.

---

Recommendations

Based on today's analysis:

1. DO: Start prompts/titles with conventional commit prefixes (fix:, chore:, refactor:, docs:) — they yield 85–95% success rates.
2. DO: Keep prompts focused and specific. Reference exact file names and clear acceptance criteria rather than long prose descriptions.
3. DO: For infrastructure changes, use ci: prefix — 100% success rate with clear, testable scope.
4. AVOID: Marking PRs as [WIP] — if the work isn't ready, don't submit it. WIP PRs close at nearly 10× the rate of regular PRs.
5. AVOID: Vague prompts without a recognized prefix or clear action. Unstructured titles have the lowest success rate at 71.1%.
6. AVOID: Feature requests without clear acceptance criteria. At 76.1%, feat: PRs underperform compared to maintenance work.

---

Historical Trends (Last 7 Days)

Date	Total PRs	Merged	Closed	Success Rate
2026-03-15	1,000	797	201	79.8%
2026-03-16	1,000	803	196	80.4%
2026-03-17	1,000	812	185	81.4%
2026-03-18	1,000	818	179	81.9%
2026-03-19	1,000	816	180	81.9%
2026-03-21	1,000	821	177	82.3%

Trend: 📈 Strong upward trend — merge rate has improved by +2.5 percentage points over the past week. The decline in WIP PRs and increasing use of conventional commit prefixes likely contribute to this improvement.

---

References:

• §23372394025

AI generated by Copilot PR Prompt Pattern Analysis · history

• expires on Mar 22, 2026, 4:59 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-22T04:59:33.498Z.

Closed by Workflow