[prompt-clustering] Prompt Clustering Analysis — 2026-03-20

[github-actions[bot]]

Summary

Metric	Value
Analysis Period	Last 30 days
Total Copilot PRs	1,000
Merged	817 (81.7%)
Closed (unmerged)	176
Open	7
TF-IDF Clusters	10

Task Category Breakdown

Category	Count	Success Rate
Bug Fix	498	84.3%
Dependency Update	130	74.6%
Chore / CI	90	80.0%
Feature	75	74.7%
Docs	56	85.7%
Other	56	78.6%
Refactor	50	86.0%
Testing	28	82.1%
Security	14	85.7%
Performance	3	66.7%

Cluster Overview

#	Theme	PRs	Success Rate	Top Keywords
1	Safe Outputs & CJS Fixes	300	87.0%	cjs, actions, fix
2	CLI & Package Refactoring	164	87.2%	pkg, pkg cli, cli
3	Workflow & Reporting	136	79.4%	workflows, md, report
4	Dependency & Lock File Updates	100	77.0%	golden, v0, files
5	Engine & MCP Integration	97	76.3%	engine, copilot, gemini
6	Docs & Content Site	65	78.5%	docs, src, docs src
7	PR & Issue Workflows	59	83.1%	pull, pull request, request
8	GitHub App & Token Support	44	88.6%	app, github app, token
9	Integrity & Security Policy	19	78.9%	min integrity, integrity, min
10	WIP / Draft PRs	16	0.0%	work, form, wip

Key Insights

1. Bug fixes are the dominant task type (49.8%): 498 PRs with a strong 84.3% merge rate — the agent handles defect resolution reliably across the codebase.
2. Safe Outputs & CJS cluster is the largest (Cluster 1, 30%): 300 PRs with 87% merge rate, indicating this core area gets consistent, high-quality fixes.
3. WIP cluster has 0% merge rate (Cluster 10, 16 PRs): All unmerged, as expected for draft submissions — the model correctly marks exploratory work accordingly.
4. GitHub App/Token tasks excel (Cluster 8, 88.6% merged): The agent's highest success rate cluster, showing strong performance on auth/token integration tasks.
5. Dependency updates have lower success (74.6%): Lowest of major categories — likely due to cascading test failures or golden file regeneration requirements needing manual follow-up.
6. Docs & Security tasks are high quality (both ~85-86% merged): Well-scoped, clearly bounded tasks produce the most consistent results.
7. Engine/MCP integration is complex (Cluster 5, 76% merged): Multi-engine support work has more review friction, suggesting prompts benefit from more explicit scope boundaries.

Recommendations

1. Improve dependency update prompts: Add explicit instructions to regenerate golden/lock files and run test suites after version bumps to raise the current 74.6% success rate.
2. Break up engine/MCP tasks: Cluster 5 (engines, MCP) at 76% merge rate suggests tasks are often too broad — split into focused sub-tasks (e.g., "update Gemini engine MCP config" rather than "update all engine MCP configs").
3. Template for PR & issue workflow tasks: Cluster 3 (workflow/reporting) at 79% could benefit from explicit action templates specifying required permissions and trigger conditions.
4. Leverage test-writing strengths: With 82% merge rate on testing tasks, bundling test creation with bug fixes in the same prompt should improve overall code quality.
5. Promote WIP-to-complete patterns: The 16 WIP PRs represent stalled work — consider a follow-up prompt template to pick up and complete draft PRs.

Detailed Cluster Breakdown

Cluster 1: Safe Outputs & CJS Fixes

Bug fixes and feature work on safe output handlers, CJS/Actions integration, and test infrastructure

• Size: 300 PRs (30% of total)

• Merge Rate: 261/300 (87.0%)

• Top Keywords: cjs, actions, fix, safe, test, github

• Task Types: bug fix: 180, dependency update: 27, feature: 25, chore/ci: 24

• Example PRs: Bump absolute maximum for update_issue and update_discussion safe outputs to 256 #21902, fix: insert Protected Files section before footer in fallback issues #21841, Fix safe-outputs artifact 409 conflict by using a dedicated artifact name #21840, refactor: consolidate shared JS constants into constants.cjs #21835

  Sample titles:

  • Bump absolute maximum for update_issue and update_discussion safe outputs to 256
  • fix: insert Protected Files section before footer in fallback issues
  • Fix safe-outputs artifact 409 conflict by using a dedicated artifact name

Cluster 2: CLI & Package Refactoring

CLI package (pkg/cli) bug fixes, refactoring, and function-level improvements

• Size: 164 PRs (16% of total)

• Merge Rate: 143/164 (87.2%)

• Top Keywords: pkg, pkg cli, cli, error, pkg workflow, function

• Task Types: bug fix: 86, refactor: 29, testing: 13, dependency update: 10

• Example PRs: refactor: semantic function clustering — move functions to better-aligned files #21908, fix: schema validation errors report correct line number and cleaner message prefix #21853, Add --filtered-integrity flag to logs command #21838, fix: retry self-upgrade with binary rename on Linux/WSL to avoid ETXTBSY #21793

  Sample titles:

  • refactor: semantic function clustering — move functions to better-aligned files
  • fix: schema validation errors report correct line number and cleaner message prefix
  • Add --filtered-integrity flag to logs command

Cluster 3: Workflow & Reporting

Daily/scheduled workflow improvements, report formatting, and workflow automation

• Size: 136 PRs (14% of total)

• Merge Rate: 108/136 (79.4%)

• Top Keywords: workflows, md, report, daily, github, workflow

• Task Types: bug fix: 64, chore/ci: 31, dependency update: 14, docs: 12

• Example PRs: fix(workflow): normalize report formatting in prompt-clustering-analysis #21915, docs: fix accessibility and build issues from multi-device testing report #21912, Update issue-monster to dispatch issues to cloclo via dispatch-workflow #21859, feat: daily DIFC integrity-filtered events analysis workflow + MCP logs filtered_integrity param #21855

  Sample titles:

  • fix(workflow): normalize report formatting in prompt-clustering-analysis
  • docs: fix accessibility and build issues from multi-device testing report
  • Update issue-monster to dispatch issues to cloclo via dispatch-workflow

Cluster 4: Dependency & Lock File Updates

Version bumps, golden file regeneration, and dependency lock file updates

• Size: 100 PRs (10% of total)

• Merge Rate: 77/100 (77.0%)

• Top Keywords: golden, v0, files, copilot, lock, update

• Task Types: dependency update: 49, bug fix: 28, chore/ci: 10, feature: 6

• Example PRs: deps: update github.com/securego/gosec/v2 v2.24.7 → v2.25.0 #21947, Update MCP Gateway v0.1.19 → v0.1.20 #21946, chore: bump golang.org/x/term v0.40.0 → v0.41.0 #21945, fix: smoke-gemini scheduled runs fail due to unconditional add_comment + recompile stale lock files #21920

  Sample titles:

  • deps: update github.com/securego/gosec/v2 v2.24.7 → v2.25.0
  • Update MCP Gateway v0.1.19 → v0.1.20
  • chore: bump golang.org/x/term v0.40.0 → v0.41.0

Cluster 5: Engine & MCP Integration

Copilot/Gemini/Codex engine support, MCP server integration, and metadata updates

• Size: 97 PRs (10% of total)

• Merge Rate: 74/97 (76.3%)

• Top Keywords: engine, copilot, gemini, engines, mcp, codex

• Task Types: bug fix: 44, other: 15, feature: 11, chore/ci: 11

• Example PRs: fix: resolve 4 CLI consistency issues from automated inspection #21907, refactor: extract shared renderStandardJSONMCPConfig helper across engine MCP modules #21901, feat: update gh-aw-metadata payload to v3 with agent id/model and detection agent id/model #21899, Add trustedBots field to MCP Gateway spec, schema, and frontmatter #21865

  Sample titles:

  • fix: resolve 4 CLI consistency issues from automated inspection
  • refactor: extract shared renderStandardJSONMCPConfig helper across engine MCP modules
  • feat: update gh-aw-metadata payload to v3 with agent id/model and detection agent id/model

Cluster 6: Docs & Content Site

Documentation site fixes, content updates, and blog/tutorial additions

• Size: 65 PRs (6% of total)

• Merge Rate: 51/65 (78.5%)

• Top Keywords: docs, src, docs src, src content, content docs, content

• Task Types: bug fix: 26, docs: 24, dependency update: 6, other: 4

• Example PRs: fix(docs): redirect /gh-aw (no trailing slash) to /gh-aw/ #21906, fix: add docs/.npmrc to resolve astro-mermaid peer dep conflict with Astro v6 #21691, Add weekly blog post writer agentic workflow #21575, docs: expand "Write Operations Fail" troubleshooting stub in common-issues.md #21519

  Sample titles:

  • fix(docs): redirect /gh-aw (no trailing slash) to /gh-aw/
  • fix: add docs/.npmrc to resolve astro-mermaid peer dep conflict with Astro v6
  • Add weekly blog post writer agentic workflow

Cluster 7: PR & Issue Workflows

Pull request creation workflows, issue triage agents, and review automation

• Size: 59 PRs (6% of total)

• Merge Rate: 49/59 (83.1%)

• Top Keywords: pull, pull request, request, create, create pull, pr

• Task Types: bug fix: 37, feature: 8, docs: 6, security: 3

• Example PRs: Fix Smoke Gemini schedule failures and Issue Triage Agent community label #21927, fix: recompile stale lock files and restore Daily Workflow Updater #21916, Skip write permissions for staged safe output handlers #21903, fix: allow wildcard target-repo: "*" in safe-output handlers #21877

  Sample titles:

  • Fix Smoke Gemini schedule failures and Issue Triage Agent community label
  • fix: recompile stale lock files and restore Daily Workflow Updater
  • Skip write permissions for staged safe output handlers

Cluster 8: GitHub App & Token Support

GitHub App authentication, token passing, and app credential configuration

• Size: 44 PRs (4% of total)

• Merge Rate: 39/44 (88.6%)

• Top Keywords: app, github app, token, app token, apm, github

• Task Types: bug fix: 19, dependency update: 9, feature: 8, docs: 2

• Example PRs: Support github-token in update-discussion safe output #21924, Comment out unconfigured app credentials in activation-app.md and safe-output-app.md #21836, fix: make pre-activation app token generation resilient to missing app installation #21833, Move reaction step after generate_aw_info for faster user feedback #21783

  Sample titles:

  • Support github-token in update-discussion safe output
  • Comment out unconfigured app credentials in activation-app.md and safe-output-app.md
  • fix: make pre-activation app token generation resilient to missing app installation

Cluster 9: Integrity & Security Policy

MCP Gateway integrity guards, min-integrity policy, and supply chain security

• Size: 19 PRs (2% of total)

• Merge Rate: 15/19 (78.9%)

• Top Keywords: min integrity, integrity, min, guard, guard policy, repos

• Task Types: bug fix: 6, other: 5, chore/ci: 4, docs: 2

• Example PRs: Fix MCP Gateway failure: default repos to "all" when only min-integrity is set #21893, Replace lockdown: true with min-integrity: approved in workflow frontmatter #21831, docs(slides): add supply chain integrity and GitHub Guard sections to security features slide #21805, fix: default github.repos to all when omitted from guard policy #21718

  Sample titles:

  • Fix MCP Gateway failure: default repos to "all" when only min-integrity is set
  • Replace lockdown: true with min-integrity: approved in workflow frontmatter
  • docs(slides): add supply chain integrity and GitHub Guard sections to security features slide

Cluster 10: WIP / Draft PRs

Work-in-progress drafts, not yet ready for merge

• Size: 16 PRs (2% of total)

• Merge Rate: 0/16 (0.0%)

• Top Keywords: work, form, wip, description, make, pr

• Task Types: bug fix: 8, dependency update: 6, testing: 1, docs: 1

• Example PRs: [WIP] [ISSUE-123] Fix gh-aw v0.61.0 fails when safe-outputs staged mode is enabled #21595, [WIP] Add test for new feature #20874, [WIP] Fix Codex using web search when tool is not provided #20458, [WIP] Update GitHub MCP Server and Gateway dependencies #20054

  Sample titles:

  • [WIP] [ISSUE-123] Fix gh-aw v0.61.0 fails when safe-outputs staged mode is enabled
  • [WIP] Add test for new feature
  • [WIP] Fix Codex using web search when tool is not provided

References:

• Workflow run: §23342725476

AI generated by Copilot Agent Prompt Clustering Analysis · history

• expires on Mar 21, 2026, 12:47 PM UTC

[github-actions[bot]]

🤖 Beep boop! The smoke test agent was here! 🚀

Just passing through to verify all systems are operational. Everything looks great in the repo! Keep up the excellent work on those agentic workflows! 🎉

Note

🔒 Integrity filtering filtered 2 items

Integrity filtering activated and filtered the following items during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:[docs] Update glossary - daily scan #21948 (pull_request_read: Resource 'pr:[docs] Update glossary - daily scan #21948' has lower integrity than agent requires. Agent would need to drop integrity tags [unapproved:all approved:all] to trust this resource.)
• pr:[docs] Update glossary - daily scan #21948 (pull_request_read: Resource 'pr:[docs] Update glossary - daily scan #21948' has lower integrity than agent requires. Agent would need to drop integrity tags [approved:all unapproved:all] to trust this resource.)

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

🎉 The smoke test agent returns with fanfare! 🎭

After validating 12 tests across GitHub MCP, Playwright, web fetch, file I/O, builds, and workflow dispatch — I can confirm the agentic infrastructure is chef's kiss 💋

11/12 tests passed (Serena MCP was the only party pooper, being fashionably absent).

May your workflows always compile on the first try! 🚀✨

Note

🔒 Integrity filtering filtered 2 items

Integrity filtering activated and filtered the following items during workflow execution.
This happens when a tool call accesses a resource that does not meet the required integrity or secrecy level of the workflow.

• pr:[docs] Update glossary - daily scan #21948 (pull_request_read: Resource 'pr:[docs] Update glossary - daily scan #21948' has lower integrity than agent requires. Agent would need to drop integrity tags [unapproved:all approved:all] to trust this resource.)
• pr:[docs] Update glossary - daily scan #21948 (pull_request_read: Resource 'pr:[docs] Update glossary - daily scan #21948' has lower integrity than agent requires. Agent would need to drop integrity tags [approved:all unapproved:all] to trust this resource.)

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Copilot Agent Prompt Clustering Analysis.

A newer discussion is available at Discussion #22125.