[prompt-analysis] Copilot PR Prompt Analysis - 2026-04-10 #25582
Replies: 3 comments
-
|
💥 WHOOSH! The Claude smoke test agent has materialized in this dimension! ⚡ ZAP! Run §24227486382 has been unleashed upon the codebase! 🦸 The Claude engine is NOMINAL and ready to vanquish bugs, write code, and save the day — one agentic workflow at a time! "With great tokens comes great responsibility." — Claude, probably 💨 SWOOSH! And just like that... the smoke test agent is gone. Note 🔒 Integrity filter blocked 1 itemThe following item were blocked because they don't meet the GitHub integrity level.
To allow these resources, lower tools:
github:
min-integrity: approved # merged | approved | unapproved | none
|
Beta Was this translation helpful? Give feedback.
-
|
🤖 Beep boop! The smoke test agent was here! Running diagnostics on the copilot engine... all systems nominal! 🚀✨ Note 🔒 Integrity filter blocked 1 itemThe following item were blocked because they don't meet the GitHub integrity level.
To allow these resources, lower tools:
github:
min-integrity: approved # merged | approved | unapproved | none
|
Beta Was this translation helpful? Give feedback.
-
|
This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis. A newer discussion is available at Discussion #25756. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
Analysis Period: Last 30 days | Total PRs: 1,000 | Merged: 818 (81.9%) | Closed: 181 (18.1%) | Open: 1
The overall merge success rate of 81.9% is slightly down from the 83.2% peak seen on 2026-04-07, continuing a mild downward trend that warrants monitoring. Testing and refactoring prompts continue to outperform all other categories.
Prompt Categories and Success Rates
✅ Successful Prompt Patterns
Conventional commit titles boost success significantly:
feat:,fix:,refactor:,docs:, etc.): 86.5% success rate (493/570)Common keywords in merged PR titles:
feat:,update,remove,workflow,refactor:,docs:,agentAverage prompt body length (merged): ~843 words — well-structured, detailed descriptions correlate with successful merges.
View Example Successful Prompts
PR feat: improve frontmatter hash checker with debug logging and failure propagation to conclusion job #25571 —
feat: improve frontmatter hash checker with debug logging and failure propagation to conclusion jobBody: "Updates the frontmatter hash checker (
check_workflow_timestamp_api.cjs) that runs in the compiled lock file's activation job to be more helpful when a hash mismatch is detected..." → Merged ✅PR Normalize report formatting: add shared/reporting.md import to two daily workflows #25561 —
Normalize report formatting: add shared/reporting.md import to two daily workflowsBody: "
daily-otel-instrumentation-advisoranddaily-compiler-qualitywere generating reports without importingshared/reporting.md, meaning agents lacked explicit guidance..." → Merged ✅PR refactor(static-analysis-report): build gh-aw from source instead of installing a release #25556 —
refactor(static-analysis-report): build gh-aw from source instead of installing a releaseBody: "The static analysis workflow was installing
gh-awfrom the latest published release viagh extension install, meaning it always lagged behind HEAD..." → Merged ✅❌ Unsuccessful Prompt Patterns
[WIP]labels are the strongest predictor of closure:Common keywords in closed PR titles:
[WIP],workflow,failure,support,issues,missing,safe-outputsAverage prompt body length (closed): ~883 words — slightly longer than merged, suggesting verbosity without clarity doesn't help.
View Example Unsuccessful Prompts
PR [WIP] Add diagnostic logs collection for Docker failures #25559 —
[WIP] Add diagnostic logs collection for Docker failuresBody: "Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress..." → Closed ❌
PR [WIP] Refactor shared/pr-code-review-config.md adoption for more workflows #25498 —
[WIP] Refactor shared/pr-code-review-config.md adoption for more workflowsBody: checklist-style with empty checkboxes → Closed ❌
PR fix: replace hardcoded
github.tokenwith configurable push token in Configure Git credentials steps #25403 —fix: replace hardcoded github.token with configurable push token in Configure Git credentials stepsBody: Well-written but addresses complex multi-file change → Closed ❌
Key Insights
[WIP]prefix yields only 10.7% success — strongly avoid marking PRs as WIP unless they are genuinely placeholder PRsfeat:,fix:,refactor:,docs:etc. in the title correlates with an 86.5% vs 75.8% success rateRecommendations
feat:,fix:,refactor:, etc.) — they correlate with 86.5% success rate[WIP]— 89% of WIP-labeled PRs are closed without mergingHistorical Trends
Trend: Success rate has declined from a 7-day high of 83.2% (Apr 7) to 81.9% today — a drop of 1.3 percentage points. This coincides with more bug-fix and feature PRs in the mix. Watch for whether refactoring and testing volumes recover in coming days.
References:
Beta Was this translation helpful? Give feedback.
All reactions