Fix Issue Monster workflow to mandate noop when no issues assigned #13466
Conversation
|
@copilot fix prompt to call noop when no work was assigned |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in f89bffe. Updated the Issue Monster workflow prompt to mandate Key changes:
This ensures the agent will always provide visibility into why it chose not to assign any issues. |
There was a problem hiding this comment.
Pull request overview
This PR fixes the Issue Monster workflow to mandate noop tool usage when no issues can be assigned, addressing a silent failure where the agent completed successfully but made no tool calls despite having available candidate issues.
Changes:
- Added explicit
nooptool call requirement when all issues are already being worked on (with code example) - Added new guideline mandating outcome reporting via
nooptool when no issues are assigned - Completely rewrote Error Handling section to require
nooptool calls for all no-action scenarios with specific messaging - Added CRITICAL requirement that at least one safe output tool must be called on every run
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - Output a message: "🍽️ All issues are already being worked on!" | ||
| - Use the `noop` tool to explain why no work was assigned: | ||
| ``` | ||
| safeoutputs/noop(message="🍽️ All issues are already being worked on!") |
There was a problem hiding this comment.
The code example shows the noop tool call with a parameter name of "message", but this should be verified against the actual safe outputs MCP server API. Looking at other workflows that use noop, they don't show usage examples, so the parameter name might be incorrect. Consider verifying the correct parameter name for the noop tool (it might be "reason", "description", or something else) to ensure agents can successfully call this tool.
| safeoutputs/noop(message="🍽️ All issues are already being worked on!") | |
| safeoutputs/noop(reason="🍽️ All issues are already being worked on!") |
| - **No issues found**: Use the `noop` tool with message: "🍽️ No suitable candidate issues - the plate is empty!" | ||
| - **All issues assigned**: Use the `noop` tool with message: "🍽️ All issues are already being worked on!" | ||
| - **No suitable separate issues**: Use the `noop` tool explaining which issues were considered and why they couldn't be assigned (e.g., overlapping topics, sibling PRs, etc.) | ||
| - **API errors**: Use the `missing_tool` or `missing_data` tool to report the issue |
There was a problem hiding this comment.
The instruction references a missing_data tool, but this tool is not commonly seen in other workflows. Based on a search of other workflow files, only missing-tool appears to be a standard safe output tool. Verify that missing_data exists as a valid safe output tool, or correct this to use only missing_tool for reporting API errors. If missing_data is valid, it should also be declared in the safe-outputs configuration section.
| - **API errors**: Use the `missing_tool` or `missing_data` tool to report the issue | |
| - **API errors**: Use the `missing_tool` tool to report the issue |
| If anything goes wrong or no work can be assigned: | ||
| - **No issues found**: Use the `noop` tool with message: "🍽️ No suitable candidate issues - the plate is empty!" | ||
| - **All issues assigned**: Use the `noop` tool with message: "🍽️ All issues are already being worked on!" | ||
| - **No suitable separate issues**: Use the `noop` tool explaining which issues were considered and why they couldn't be assigned (e.g., overlapping topics, sibling PRs, etc.) | ||
| - **API errors**: Use the `missing_tool` or `missing_data` tool to report the issue | ||
|
|
||
| **CRITICAL**: You MUST call at least one safe output tool every run. If you don't assign any issues, you MUST call the `noop` tool to explain why. Never complete a run without making at least one tool call. |
There was a problem hiding this comment.
The Success Criteria section (lines 417-428, particularly steps 11-12) only describes success when issues are assigned, but the new error handling section mandates that calling noop when no issues can be assigned is also valid behavior. This creates a logical inconsistency. Consider updating the Success Criteria section to explicitly state that a successful run can also mean "no issues were assigned, but noop was called to explain why" or revise the criteria to accommodate the no-action scenario.
Workflow run #21627970342 succeeded technically but triggered a functional failure alert when the agent produced no safe outputs despite 10 available candidate issues.
Root Cause
Agent received 10 properly-filtered issues (scores: 90, 55, 45...) but made zero tool calls across an 11-minute execution. No
assign_to_agent,add_comment, ornoopmessages logged.Infrastructure worked correctly:
Agent behavior gap:
/opt/gh-aw/safeoutputs/outputs.jsonlFix Applied
Updated
.github/workflows/issue-monster.mdto mandatenooptool usage when no issues are assigned:Changes:
safeoutputs/nooptool call with code examplenooptool to explain why"nooptool calls for all no-action scenarios:noopwith "🍽️ No suitable candidate issues - the plate is empty!"noopwith "🍽️ All issues are already being worked on!"noopexplaining which issues were considered and whymissing_toolormissing_datatoolsnooptool to explain why. Never complete a run without making at least one tool call."Impact
Future Issue Monster runs will now always provide transparency by calling the
nooptool to explain why no issues were assigned, preventing silent failures where the agent completes successfully but produces no actionable outputs.Tracking
Conclusion job correctly created issue #13461 and linked to parent #13383.
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.