Skip to content

Add shared/gh.md safe-input workflow for gh CLI access#5157

Merged
pelikhan merged 9 commits intomainfrom
copilot/create-gh-shared-workflow
Dec 1, 2025
Merged

Add shared/gh.md safe-input workflow for gh CLI access#5157
pelikhan merged 9 commits intomainfrom
copilot/create-gh-shared-workflow

Conversation

Copy link
Contributor

Copilot AI commented Dec 1, 2025

  • Create a new shared agentic workflow shared/gh.md that exposes a "gh" safe-input tool to execute any gh CLI command
  • Modify dev.md to:
    • Use Codex engine
    • Import the new shared/gh.md
    • Remove the github tool and use the "gh" safe-input to summarize the last PR
    • Remove assign-to-agent safe-output
  • Compile both workflows and verify they work
  • Lint and test the changes
  • Address code review feedback:
    • Updated documentation to clarify GH_TOKEN usage
    • Fixed example command syntax in documentation
    • Added security notes about safe-inputs validation
  • Fix authentication: Changed ${{ secrets.GITHUB_TOKEN }} to ${{ github.token }} for proper gh CLI authentication in GitHub Actions
  • Fix safe-inputs MCP server startup: Pass safe-inputs secrets (like GH_TOKEN) to the agent execution step in Codex, Copilot, and Claude engines
  • Simplify gh.md description to prevent "safe-input" from being incorrectly passed to gh CLI

Note: The github MCP server is added automatically by the compiler for all workflows. The gh.md shared workflow only defines safe-inputs, not the github tool. Disabling the github tool via github: false is not currently supported by the schema.

Original prompt

Create a "gh" shared agentic workflow that exposes a single safe-input tool "gh" that allows to call any gh cli tool using the GITHUB_TOKEN.

Modify dev.md to use "gh" and NOT the github tool to summarize the last PR in the repository.

Use Claude.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits December 1, 2025 03:28
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add gh shared agentic workflow with safe-input tool Add shared/gh.md safe-input workflow for gh CLI access Dec 1, 2025
Copilot AI requested a review from pelikhan December 1, 2025 03:40
@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

⚠️ Dev Hawk Report - Failure Analysis

Workflow Run: #3021

  • Status: failure
  • Commit: e1926dd
  • Duration: ~2 minutes
  • Triggered by: @pelikhan via workflow_dispatch

Root Cause Analysis

The workflow failed because the gh CLI tool is not authenticated when used through the safe-input mechanism. The agent attempted to use the gh command but received an authentication error.

Error Details

The key error message extracted from the logs:

Exit code 4
gh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable. Example:
  env:
    GH_TOKEN: ${{ github.token }}

This occurred when the Claude engine tried to execute a gh command through the new shared/gh.md safe-input tool.

Problem Breakdown

  1. New Feature: The PR introduces shared/gh.md which wraps the gh CLI as a safe-input tool
  2. Missing Configuration: The safe-input tool execution environment doesn't have the GH_TOKEN environment variable set
  3. Authentication Failure: The gh CLI requires GH_TOKEN to authenticate with GitHub API, causing it to fail with exit code 4

Recommended Actions

  • Update shared/gh.md: Add GH_TOKEN environment variable to the safe-input configuration
    safe-inputs:
      gh:
        env:
          GH_TOKEN: ${{ github.token }}
  • Verify Authentication: Ensure the GITHUB_TOKEN is properly passed through to the safe-input execution context
  • Test Locally: Run the gh command in the workflow environment to confirm authentication works
  • Update Documentation: Add notes about required environment variables for gh CLI integration

Investigation Notes

  • The error occurred during agent execution in the main workflow job
  • The Claude engine successfully started but failed when invoking the gh tool
  • This is a configuration issue, not a code bug - the safe-input mechanism works, but lacks necessary authentication
  • Similar GitHub CLI integrations in other workflows successfully use GH_TOKEN environment variable

Related Context

The PR description mentions:

Safe-input tool wrapping gh CLI with GITHUB_TOKEN auth

However, the actual implementation appears to be missing the authentication configuration in the safe-input execution environment.

🦅 Observed from above by Dev Hawk

@pelikhan
Copy link
Contributor

pelikhan commented Dec 1, 2025

@copilot apply comment #5157 (comment)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 1, 2025

@copilot apply comment #5157 (comment)

Fixed in b059a64. Changed ${{ secrets.GITHUB_TOKEN }} to ${{ github.token }} for proper gh CLI authentication in GitHub Actions.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

⚠️ Dev Hawk Report - Failure Analysis

Workflow Run: #3022

  • Status: failure
  • Commit: b059a64
  • Duration: ~2 minutes
  • Failed Step: Parse agent logs for step summary (step 34)

Root Cause Analysis

The workflow failed due to an MCP server initialization failure. The safeinputs MCP server failed to launch during the agent execution phase.

Critical Error

##[error]MCP server(s) failed to launch: safeinputs

This error occurred immediately after the Claude Code CLI execution step completed successfully, indicating that:

  1. The Claude agent itself ran successfully (2 turns, 16s duration, 1 tool call succeeded)
  2. The agent was able to call github::list_pull_requests successfully
  3. However, the workflow expected to use the safeinputs MCP server but it failed to initialize

Context: What is safeinputs?

Based on the PR description, this workflow is testing a new shared/gh.md safe-input workflow that exposes a "gh" safe-input tool to execute gh CLI commands. The safeinputs MCP server is responsible for handling these safe-input operations.

Why Did It Fail?

The failure suggests one of these scenarios:

  1. Configuration mismatch: The safeinputs MCP server configuration in the compiled workflow may be incorrect
  2. Missing dependencies: The MCP server may require dependencies that aren't available in the execution environment
  3. Authentication issue: The server may need proper GH_TOKEN configuration to initialize
  4. Timing issue: The server may have attempted to start before all prerequisites were ready

Agent Execution Summary

The Claude agent did execute successfully before the failure:

  • Model: claude-sonnet-4-5-20250929
  • Turns: 2
  • Duration: 16s
  • Tool Calls: 1/1 succeeded (github::list_pull_requests)
  • Tokens: 9 in / 522 out
  • Cost: $0.1427

Recommended Actions

  • Review shared/gh.md compilation: Check if the safe-inputs MCP server configuration is correct in the compiled .lock.yml file
  • Verify MCP server setup: Ensure the safeinputs MCP server is properly configured in the workflow's MCP configuration
  • Check authentication: Verify that GH_TOKEN is properly configured - the recent fix changed from ${{ secrets.GITHUB_TOKEN }} to ${{ github.token }}
  • Review MCP logs: Check the MCP logs artifact for detailed server startup errors
  • Test locally: Try compiling and validating the workflow locally with gh aw compile dev.md

Investigation Notes

  • The agent execution itself was successful, suggesting the core workflow logic is sound
  • The failure is specifically in the MCP server infrastructure layer, not the AI agent's task execution
  • This appears to be related to the new safe-inputs feature being tested in this PR
  • The error occurred in the "Parse agent logs for step summary" step, which is a post-execution validation step

Related Files to Review

  • .github/workflows/shared/gh.md - The new shared workflow
  • .github/workflows/dev.md - The workflow importing shared/gh.md
  • .github/workflows/dev.lock.yml - The compiled workflow (check MCP configuration)

Dev Hawk is monitoring your copilot/ branches and will report on workflow completions automatically.*

🦅 Observed from above by Dev Hawk

@pelikhan
Copy link
Contributor

pelikhan commented Dec 1, 2025

@copilot change dev.md engine to codex

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 1, 2025

@copilot change dev.md engine to codex

Done in 8ae091c.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

✅ Dev Hawk Report - Success

Workflow Run: #3023

  • Status: ✅ completed
  • Conclusion: ✅ success
  • Commit: 8ae091c9815ded090bfa8d2a356a1df21679126f
  • Branch: copilot/create-gh-shared-workflow
  • Duration: ~3 minutes 37 seconds
  • Triggered by: @pelikhan via workflow_dispatch

Summary

The Dev workflow completed successfully! 🎉

Commit Message:

Change dev.md engine from claude to codex

Co-authored-by: pelikhan 4175913+pelikhan@users.noreply.github.com

This commit switched the engine in dev.md from Claude to Codex, which appears to be working correctly based on the successful workflow run.


🦅 Automated analysis by Dev Hawk - monitoring workflow completions on copilot/ branches*

🦅 Observed from above by Dev Hawk

@pelikhan
Copy link
Contributor

pelikhan commented Dec 1, 2025

@copilot review the safeinputs MCP server JavaScript code as it seems to the fail to load.

Add more test, check file paths, add more tests local.

Here is a log:
2025-12-01T04:13:42.766079Z DEBUG codex_exec: Received event: Event { id: "", msg: McpStartupUpdate(McpStartupUpdateEvent { server: "safeinputs", status: Failed { error: "MCP client for safeinputs timed out after 10 seconds. Add or adjust startup_timeout_sec in your config.toml:\n[mcp_servers.safeinputs]\nstartup_timeout_sec = XX" } }) }

The safeinputs MCP server was timing out because environment variables
(like GH_TOKEN) defined in safe-inputs were not being passed to the
agent execution step. This caused codex to start the MCP server without
access to the required environment variables.

Changes:
- Add safe-inputs secrets passthrough in Codex engine (codex_engine.go)
- Add safe-inputs secrets passthrough in Copilot engine (copilot_engine.go)
- Add safe-inputs secrets passthrough in Claude engine (claude_engine.go)
- Add unit test for Codex engine to verify safe-inputs secrets are included

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 1, 2025

@copilot review the safeinputs MCP server JavaScript code as it seems to the fail to load.

Add more test, check file paths, add more tests local....

Fixed in 36e0854. The safeinputs MCP server was timing out because environment variables (like GH_TOKEN) defined in safe-inputs were not being passed to the agent execution step. Added safe-inputs secrets passthrough to the Codex, Copilot, and Claude engine execution steps so the MCP server can access the required environment variables when started.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

✅ Dev Hawk Report - Success

Workflow Run: #3024

  • Status: completed
  • Conclusion: success
  • Commit: 36e0854be5476acb8c444bfa354b4b16f88d96ae
  • Duration: ~1 minute 47 seconds
  • Branch: copilot/create-gh-shared-workflow

Summary

The Dev workflow completed successfully! 🎉

What Was Fixed

This commit addresses the safe-inputs MCP server startup timeout issue by ensuring environment variables (like GH_TOKEN) are properly passed through to the agent execution step.

Changes Applied:

  • ✅ Added safe-inputs secrets passthrough in Codex engine
  • ✅ Added safe-inputs secrets passthrough in Copilot engine
  • ✅ Added safe-inputs secrets passthrough in Claude engine
  • ✅ Added unit test to verify safe-inputs secrets are included

Commit Details

Fix safe-inputs MCP server startup by passing secrets to execution step

The safeinputs MCP server was timing out because environment variables
(like GH_TOKEN) defined in safe-inputs were not being passed to the
agent execution step. This caused codex to start the MCP server without
access to the required environment variables.


This automated report is brought to you by Dev Hawk 🦅 - monitoring your workflow runs so you don't have to!

🦅 Observed from above by Dev Hawk

@pelikhan
Copy link
Contributor

pelikhan commented Dec 1, 2025

@copilot gh.md to remove the "safe-input argument from the gh cli call.

See error:

2025-12-01T04:54:46.885574Z DEBUG codex_exec: Received event: Event { id: "0", msg: ExecCommandEnd(ExecCommandEndEvent { call_id: "call_RHxPBfYCveAnLxKhuN4TL7oA", turn_id: "0", command: ["/bin/bash", "-lc", "gh safe-input pr list --limit 1 --json number,title,body,author,state,createdAt,mergedAt,url"], cwd: "/home/runner/work/gh-aw/gh-aw", parsed_cmd: [Unknown { cmd: "gh safe-input pr list --limit 1 --json 'number,title,body,author,state,createdAt,mergedAt,url'" }], source: Agent, interaction_input: None, stdout: "", stderr: "unknown command "safe-input" for "gh"\n\nUsage: gh [flags]\n\nAvailable commands:\n agent-task\n alias\n api\n attestation\n auth\n browse\n cache\n co\n codespace\n completion\n config\n extension\n gist\n gpg-key\n issue\n label\n org\n pr\n preview\n project\n release\n repo\n ruleset\n run\n search\n secret\n ssh-key\n status\n variable\n workflow\n\n", aggregated_output: "unknown command "safe-input" for "gh"\n\nUsage: gh [flags]\n\nAvailable commands:\n agent-task\n alias\n api\n attestation\n auth\n browse\n cache\n co\n codespace\n completion\n config\n extension\n gist\n gpg-key\n issue\n label\n org\n pr\n preview\n project\n release\n repo\n ruleset\n run\n search\n secret\n ssh-key\n status\n variable\n workflow\n\n", exit_code: 1, duration: 190.038197ms, formatted_output: "unknown command "safe-input" for "gh"\n\nUsage: gh [flags]\n\nAvailable commands:\n agent-task\n alias\n api\n attestation\n auth\n browse\n cache\n co\n codespace\n completion\n config\n extension\n gist\n gpg-key\n issue\n label\n org\n pr\n preview\n project\n release\n repo\n ruleset\n run\n search\n secret\n ssh-key\n status\n variable\n workflow\n\n" }) }

…ing passed to gh CLI

The agent was incorrectly adding "safe-input" as an argument to gh CLI calls.
Simplified the tool description and documentation to be clearer about the expected usage pattern.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 1, 2025

@copilot gh.md to remove the "safe-input argument from the gh cli call.

See error:...

Fixed in a98b6b4. Simplified the gh.md safe-input description to be clearer that "safe-input" is not part of the gh CLI command. The tool description now explicitly states: "The tool will run: gh "

@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

✅ Dev Hawk Report - Success

Workflow Run: #3025

  • Status: ✅ success
  • Commit: a98b6b4
  • Branch: copilot/create-gh-shared-workflow
  • Duration: ~9 minutes (05:14:51 - 05:23:42 UTC)
  • Triggered by: @pelikhan via workflow_dispatch

Summary

The Dev workflow completed successfully! 🎉

Commit message: Simplify gh.md safe-input description to prevent "safe-input" from being passed to gh CLI

The agent was fixing an issue where "safe-input" was incorrectly being added as an argument to gh CLI calls. The changes simplified the tool description and documentation to clarify the expected usage pattern.


Monitored by Dev Hawk 🦅

🦅 Observed from above by Dev Hawk

@pelikhan
Copy link
Contributor

pelikhan commented Dec 1, 2025

@copilot disable github tool in gh.md

Remove assign-to-agent

The gh.md shared workflow only defines safe-inputs, not the github tool.
The github MCP server is added automatically by the compiler for all workflows.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 1, 2025

@copilot disable github tool in gh.md

Remove assign-to-agent

Removed assign-to-agent from dev.md in 6affa03.

Regarding disabling the github tool: The gh.md shared workflow only defines safe-inputs, not the github tool. The github MCP server is added automatically by the compiler for all workflows. The schema doesn't currently support github: false to disable it. Would you like me to add support for disabling the github tool in the compiler?

@pelikhan pelikhan marked this pull request as ready for review December 1, 2025 05:51
@pelikhan pelikhan merged commit d230e65 into main Dec 1, 2025
4 checks passed
@pelikhan pelikhan deleted the copilot/create-gh-shared-workflow branch December 1, 2025 05:51
github-actions bot added a commit that referenced this pull request Dec 1, 2025
- Document the reusable shared/gh.md workflow in safe-inputs reference
- Add 'Shared gh CLI Tool' subsection with usage examples
- Show how to import and use the gh tool with various commands

Related to #5157
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants