[prompt-analysis] Copilot PR Prompt Analysis - 2026-04-05

[github-actions[bot]]

Summary

Analysis Period: Last 30 days | Total PRs: 1,000 | Merged: 825 (83%) | Closed: 172 (17%) | Open: 3

Overall success rate is 82.8%, consistent with prior days (82.5–83.5% range). The dataset is large enough for statistically meaningful patterns.

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate	Trend
testing	276	233	85%	✅ Highest
docs	105	88	84%	✅ High
feature	51	42	82%	—
other	17	14	82%	—
bug_fix	537	436	81%	—
refactor	10	8	80%	—
update	4	4	100%	✅ Small sample

Conventional Commit Prefix Success Rates

Prefix	Count	Merged	Success Rate
ci:	12	12	100%
perf:	10	10	100%
docs:	51	46	90%
fix:	299	262	88%
refactor:	42	37	88%
feat:	122	104	85%
chore:	30	25	83%
(no prefix)	428	327	76%
test:	3	2	67%

Conventional commit style boosts success rate by ~12 percentage points (88% vs 76%).

✅ Successful Prompt Patterns

Characteristics of merged PRs:

• Average body length: ~8,849 characters
• 61% include code blocks (examples, diffs, context)
• 41% reference issue numbers
• Only 2% have checklists (merged PRs rarely use [ ] checklists)
• Common title keywords: feat, remove, docs, agent, step, refactor, files, failure

Example merged PRs

PR #24666: fix(otel): show real OTLP trace ID in observability job summary

"The observability job summary was displaying workflow_call_id (e.g. 12345678901-1) as the trace ID — a value that doesn't exist in any observabili..."

PR #24661: feat(otel-advisor): query live Sentry OTel data to ground analysis

"The Sentry MCP server was configured in daily-otel-instrumentation-advisor but never called — all analysis was purely static code reading, making re..."

PR #24659: feat(otel): enrich resource attributes with service.version, github.repository, github.run_id

"buildOTLPPayload only set service.name as a resource attribute, causing github.repository, github.run_id to be repeated on every span (wasting..."

Pattern: clear problem statement, specific code references, technical rationale.

❌ Unsuccessful Prompt Patterns

Characteristics of closed PRs:

• Average body length: ~8,388 characters (slightly shorter)
• 52% include code blocks (vs 61% for merged — less technical context)
• 17% have checklists — 8x higher than merged PRs (often signals WIP/incomplete work)
• Common title keywords: workflow, feat, failure, support, missing, mode, safeoutputs, replace

The biggest failure predictor: [WIP] prefix

• WIP PRs: 14% success rate (vs 86% for non-WIP)
• 51 WIP PRs in dataset, only 7 merged

Example closed PRs

PR #24654: feat: add gh-aw.agent.execute OpenTelemetry span around agentic engine execution

"Adds an OpenTelemetry gh-aw.agent.execute span around RunWorkflowOnGitHub..." — superseded by a more complete implementation.

PR #24639: feat(smoke-copilot): token optimization — reduce turns, drop heavy import, pre-fetch PR context

"Smoke Copilot is the #2 token consumer at 20M tokens/week. Per-request cost is stable at ~63K tokens..." — overly aggressive scope.

PR #24506: [WIP] Add tests for repo-root-relative import path resolution

"Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date..." — classic WIP, never completed.

Key Insights

• [WIP] is a near-guarantee of closure: 14% success rate confirms WIP PRs are rarely completed. Avoid using WIP labels unless you intend to close and reopen.
• Conventional commit prefixes matter: PRs with ci:, perf:, docs:, or fix: prefixes succeed at 88–100% vs 76% for no prefix — the prefix signals intent clearly and sets reviewers' expectations.
• Checklists correlate with closure: Merged PRs almost never use - [ ] checklists (2%), while 17% of closed PRs do — uncompleted checklists often signal an abandoned or partial effort.

Recommendations

1. DO: Use conventional commit prefixes (fix:, feat:, docs:, etc.) in PR titles — they improve success rates by ~12 points
2. DO: Include code blocks with context, diffs, or rationale in your PR body — 61% of merged PRs do this vs 52% for closed
3. DO: Prioritize testing-type prompts when possible — 85% success rate, highest of any category
4. AVOID: [WIP] prefix — only 14% of WIP PRs ever get merged; close them instead and reopen when complete
5. AVOID: Open-ended task descriptions with checklists that are incomplete — signals unfinished work to reviewers

Historical Trends

Date	PRs	Success Rate	Top Category
2026-04-05	1,000	82.8%	testing (85%)
2026-04-03	1,000	82.7%	refactor (93%)
2026-04-02	1,000	82.5%	docs (93%)
2026-04-01	1,000	83.5%	test (88%)

Trend: Overall success rate is remarkably stable at ~82–84% over the past 4 days. The conventional commit prefix advantage (+12 points) and WIP penalty (−72 points) are consistent across all runs.

References:

• §23994666386

Generated by Copilot PR Prompt Pattern Analysis · ● 942.8K · ◷

• expires on Apr 6, 2026, 5:06 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #24833.