Add AI agent orchestration detection to bot-detection workflow#15011
Add AI agent orchestration detection to bot-detection workflow#15011
Conversation
…kflow Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the bot-detection agent workflow prompt to add a new “Red Flag 9” for detecting patterns consistent with autonomous AI-agent PR/commit orchestration, and adjusts the documented scoring/reporting details accordingly.
Changes:
- Adds Red Flag 9: AI Agent Orchestration with detection steps, thresholds, and an example scenario.
- Updates workflow copy to reflect 9 red flags and includes Red Flag 9 in the medium-severity scoring list.
- Recompiles the workflow into the generated
.lock.ymloutput (description + frontmatter hash).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.github/workflows/bot-detection.md |
Adds Red Flag 9 guidance and updates scoring/report template text to account for the new flag. |
.github/workflows/bot-detection.lock.yml |
Regenerated compiled workflow reflecting the updated description text/hash. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| **Account**: @{username} | ||
| **Risk Level**: {🔴 High / 🟠 Medium} | ||
| **Risk Score**: {score}/12 | ||
| **Risk Score**: {score}/15 |
There was a problem hiding this comment.
The report template hardcodes the max risk score as /15, but based on the scoring rules listed just above (Critical: 3 flags ×3 = 9, High: 2 flags ×2 = 4, Medium: 4 flags ×1 = 4), the maximum possible total is 17. Update the denominator (or the point allocations) so the template matches the documented scoring formula.
| **Risk Score**: {score}/15 | |
| **Risk Score**: {score}/17 |
Extends bot detection to identify autonomous AI agents creating self-directed PRs with coordinated commits, addressing patterns observed in automated Copilot SWE agent activity.
Changes
New Red Flag 9: AI Agent Orchestration (Medium severity, 1pt)
Detection triggers on:
Co-authored-by:tags pointing to AI agentscopilot-swe-agent,copilot-agent, or similar automation accountsRisk Scoring Updates
Example Detection Pattern
Note: Does not flag legitimate Copilot-assisted development. Specifically targets autonomous agents with minimal human oversight patterns.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.