feat: add AWF /reflect endpoint fetch to agent harness and step summary#29420
feat: add AWF /reflect endpoint fetch to agent harness and step summary#29420
Conversation
- Add fetchAWFReflect() helper to copilot_harness.cjs that calls http://api-proxy:10000/reflect from inside the AWF container and persists the response to /tmp/gh-aw/awf-reflect.json before exit - Create awf_reflect_summary.cjs GitHub Actions script that reads the saved reflect data and writes a provider/model table to core.summary (wrapped in a collapsible <details> block) - Create awf_reflect_summary.test.cjs with 18 unit tests covering readReflectData, formatModelList, buildReflectSummary, and main - Add generateAWFReflectSummary() step generator in compiler_yaml_ai_execution.go - Wire the new step in compiler_yaml_main_job.go alongside the existing firewall token-usage summary steps - Update wasm golden test fixtures to include the new step Agent-Logs-Url: https://github.com/github/gh-aw/sessions/0a7f35fc-382f-421a-969b-6329c1a31f8b Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot save reflect result to a folder that will be added to the agent artifact |
|
📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing... |
|
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
|
|
🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨ |
|
🚀 Smoke OpenCode MISSION COMPLETE! OpenCode delivered. 🔥 |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…nclusion Move AWF_REFLECT_OUTPUT_PATH from /tmp/gh-aw/awf-reflect.json to /tmp/gh-aw/sandbox/firewall/awf-reflect.json so the file is: - co-located with other firewall observability data - covered by the existing chmod -R a+r /tmp/gh-aw/sandbox/firewall step - explicitly added to the agent artifact alongside AWFConfigFilePath, AWFProxyLogsDir and AWFAuditDir Also add fs.mkdirSync(recursive) before writing in fetchAWFReflect() to ensure the parent directory exists (defensive, since AWF normally creates /tmp/gh-aw/sandbox/firewall/ before the harness runs). Add AWFReflectFilePath constant to pkg/constants/constants.go. Update tests to create /tmp/gh-aw/sandbox/firewall before writing. Agent-Logs-Url: https://github.com/github/gh-aw/sessions/27d4ced8-fe09-4875-bb07-b81a5fae232c Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in ce0f732. The reflect result is now saved to Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Smoke Test Results for PR #29420
Overall: ✅ PASS
|
|
🤖 BEEP BOOP — The smoke test agent was here! 🎉 I've successfully completed my mission: running through all the tests, building binaries, navigating the web, and generally causing harmless robotic mischief. Your workflows are in good hands (or rather, good circuits). 🦾 — Copilot Smoke Test Agent, on duty since 2026-05-01
|
Comment MemoryNote This comment is managed by comment memory.It stores persistent context for this thread in the code block at the top of this comment. Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "ab.chatgpt.com"
- "chatgpt.com"See Network Configuration for more information.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR consistently adds the AWF /reflect endpoint fetch step across all lock files. The implementation is clean — using continue-on-error: true ensures observability data collection doesn't block workflows. Good systematic approach to adding telemetry.
📰 BREAKING: Report filed by Smoke Copilot · ● 950.3K
| uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 | ||
| with: | ||
| script: | | ||
| const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); |
There was a problem hiding this comment.
The AWF reflect summary step uses continue-on-error: true which is good practice for observability steps that shouldn't block the workflow on failure.
| uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 | ||
| with: | ||
| script: | | ||
| const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); |
There was a problem hiding this comment.
Consistent pattern across all lock files: the AWF reflect summary step is inserted after token usage parsing. This systematic approach ensures all workflows capture reflect endpoint data.
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
@copilot merge main and recompile |
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/99d84c55-c7cd-425e-8249-529ccdf123d5 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
🚀 Smoke OpenCode MISSION COMPLETE! OpenCode delivered. 🔥 |
|
🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨ |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
|
|
📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing... |
Agent Container Tool Check
Result: 12/12 tools available ✅ Overall Status: PASS
|
|
|
|
PRs: #29429 Fix integration tests: update domain checks from CLI flags to JSON config keys; #29418 Convert 55.8% of agentic workflows to use tools.github mode: gh-proxy and tools.cli-proxy Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "ab.chatgpt.com"
- "chatgpt.com"See Network Configuration for more information.
|
|
Commit pushed:
|
|
🤖 Beep boop! Smoke test agent dropped by for a visit! All systems are go — Copilot is feeling especially sparkly today. May your builds be green and your deploys be boring! 🚀✨
|
|
Smoke Test 25201607915 — Copilot ✅ PASS
Overall: PASS 🎉
|
Comment MemoryNote This comment is managed by comment memory.It stores persistent context for this thread in the code block at the top of this comment.
|
There was a problem hiding this comment.
Smoke test review: PR systematically adds AWF /reflect observability to all lock files. Implementation is clean with proper error handling (continue-on-error: true) and artifact collection. ✅
📰 BREAKING: Report filed by Smoke Copilot · ● 1.2M
| uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 | ||
| with: | ||
| script: | | ||
| const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); |
There was a problem hiding this comment.
The new 'Print AWF reflect summary' step correctly uses continue-on-error: true, ensuring that observability data collection never blocks the main workflow. Good defensive design.
| @@ -808,6 +818,7 @@ jobs: | |||
| /tmp/gh-aw/awf-config.json | |||
There was a problem hiding this comment.
Including awf-reflect.json in the artifact upload paths is a great addition — it ensures the reflect payload is preserved for post-run debugging and analysis.
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
💥 Smoke Test 25201607850 — Claude Engine
Overall: PARTIAL (test #10 failed — AWF MCP tool unavailable)
|
There was a problem hiding this comment.
💥 Automated smoke test review - all systems nominal! This review validates PR review safe output tooling for Claude engine run 25201607850. The inline review comments above test review comment creation on the PR diff. The continue-on-error: true pattern for observability steps is correctly applied across all compiled lock files.
💥 [THE END] — Illustrated by Smoke Claude · ● 235.3K
| uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 | ||
| with: | ||
| script: | | ||
| const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); |
There was a problem hiding this comment.
🔍 Smoke test review comment #1 — The continue-on-error: true pattern here is good defensive programming for the AWF reflect summary step. This ensures observability data collection failures don't block the main workflow execution. ✅ Approved pattern.
| uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 | ||
| with: | ||
| script: | | ||
| const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); |
There was a problem hiding this comment.
🔍 Smoke test review comment #2 — The AWF reflect summary step placement after parse_token_usage.cjs ensures consistent ordering across all 205 compiled workflows. The systematic recompilation approach is the right call here — no manual inconsistencies possible. ✅
|
@copilot collect reflect info on all agent engine runs |
Summary
Adds a JavaScript helper to the agent harness that fetches the AWF API proxy
/reflectendpoint and exposes the available LLM providers and models in the GitHub Actions step summary. The reflect result is saved inside the AWF firewall sandbox folder so it is automatically included in the agent artifact.Background
The AWF api-proxy sidecar exposes a
/reflectmanagement endpoint on port 10000. This endpoint returns the list of configured LLM providers (openai, anthropic, copilot, gemini, opencode) along with their available model lists, and amodels_fetch_completeflag indicating whether model discovery has finished. This information is useful for debugging which providers were available during a workflow run.Changes
actions/setup/js/copilot_harness.cjsfetchAWFReflect()async helper that callshttp://api-proxy:10000/reflectfrom inside the AWF container (using Node.jshttpmodule, 5s timeout)/tmp/gh-aw/sandbox/firewall/awf-reflect.json— co-located with other firewall observability data so it is covered by the existingchmod -R a+rstep and included in the agent artifactmkdirSync({ recursive: true })as a defensive guardmain()just before the harness exits somodels_fetch_completeis likelytrueextractModelIds(json)helper that parses both OpenAI/Anthropic/Copilot format ({ data: [{ id }] }) and Gemini format ({ models: [{ name }] }) API responsesfetchModelsFromUrl(url, timeout, logger)helper that fetches models from amodels_urlendpoint on the api-proxy; the api-proxy injects the correct auth headers when forwarding requests upstreamenrichReflectModels(reflectData, timeout, logger)that runs fallback fetches for all configured endpoints where the api-proxy's startup model-fetch leftmodelsasnull; called insidefetchAWFReflect()before saving, so model names are populated in the artifact even when the api-proxy's startup fetch encountered a transient errorAWF_API_PROXY_REFLECT_URL,AWF_REFLECT_OUTPUT_PATH,AWF_REFLECT_TIMEOUT_MS,AWF_MODELS_URL_TIMEOUT_MS,GEMINI_MODEL_NAME_PREFIXactions/setup/js/awf_reflect_summary.cjs(new)coreglobal) that reads/tmp/gh-aw/sandbox/firewall/awf-reflect.json<details>/<summary>block so it doesn't dominate the step outputactions/setup/js/awf_reflect_summary.test.cjs(new)readReflectData,formatModelList,buildReflectSummary, andmainactions/setup/js/copilot_harness.test.cjsextractModelIds,fetchModelsFromUrl,enrichReflectModels, and the enrichedfetchAWFReflectbehavior (including mock HTTP servers with try-finally cleanup)pkg/constants/constants.goAWFReflectFilePath = "/tmp/gh-aw/sandbox/firewall/awf-reflect.json"constantpkg/workflow/compiler_yaml_ai_execution.gogenerateAWFReflectSummary()step generator following the same pattern asgenerateTokenUsageSummarypkg/workflow/compiler_yaml_main_job.goAWFReflectFilePathto the agent artifact paths alongsideAWFConfigFilePath,AWFProxyLogsDir, andAWFAuditDirCompiled workflows (lock files)
/tmp/gh-aw/sandbox/firewall/awf-reflect.jsonin the artifact uploadExample step summary output
Changeset
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
ab.chatgpt.comchatgpt.comSee Network Configuration for more information.
✨ PR Review Safe Output Test - Run 25201607850