feat: add Gemini engine smoke test workflow

[lpcox]

Summary

Adds a new smoke-gemini agentic workflow to validate the Gemini engine, mirroring the existing smoke-copilot pattern.

What it tests

• GitHub MCP connectivity — lists merged PRs via github-list_pull_requests
• github.com reachability — curl connectivity check
• File I/O — write and read-back verification
• Safe-outputs — post-step validates add_comment was invoked on PR triggers

Configuration

• Engine: gemini
• Secret required: GEMINI_API_KEY (Google Gemini API key)
• Timeout: 5 minutes
• Schedule: every 12h
• Label on success: smoke-gemini

Security review (safe-update)

• New secret: GEMINI_API_KEY — Google Gemini API key for inference. Used only within the AWF sandbox; not exposed to untrusted code.

[github-actions]

🔥 Smoke Test: Copilot BYOK (Offline) Mode

Test	Result
GitHub MCP (list merged PRs)	✅
GitHub.com connectivity (HTTP 200)	✅
File write/read	✅
BYOK inference (agent → api-proxy → api.githubcopilot.com)	✅

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

Overall: PASS — @lpcox (no assignees)

🔑 BYOK report filed by Smoke Copilot BYOK

[github-actions]

Smoke Test Results
✅ GitHub MCP: #2171, #2170
✅ Playwright: GitHub page verified
✅ File write: /tmp/gh-aw/agent/smoke-test-claude-24850357735.txt
✅ Bash tool: File creation verified
Status: PASS

💥 [THE END] — Illustrated by Smoke Claude

[github-actions]

Smoke test report:

• PR title: feat: add Gemini engine smoke test workflow ✅
• PR title: chore: upgrade gh-aw to v0.69.3 and recompile workflows ✅
• GitHub MCP review (last 2 merged PRs): ✅
• safeinputs-gh PR query: ❌ (tool unavailable; used gh fallback)
• Playwright github.com title check: ✅
• Tavily web search: ❌ (Tavily tool unavailable)
• File write + cat verification: ✅
• Discussion tool flow + oracle comment: ❌ (github-discussion-query tool unavailable; fallback posted)
• Build (npm ci && npm run build): ✅
  Overall status: FAIL

Warning

⚠️ Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

[github-actions]

🏗️ Build Test Suite Results

⚠️ ALL CLONES FAILED — The gh CLI proxy (localhost:18443) returned HTTP 403 "access denied: unrecognized endpoint" for every external test repository (Mossaka/gh-aw-firewall-test-*). Only the current repository (github/gh-aw-firewall) is accessible.

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	❌	N/A	❌ CLONE_FAILED
Bun	hono	❌	N/A	❌ CLONE_FAILED
C++	fmt	❌	N/A	❌ CLONE_FAILED
C++	json	❌	N/A	❌ CLONE_FAILED
Deno	oak	❌	N/A	❌ CLONE_FAILED
Deno	std	❌	N/A	❌ CLONE_FAILED
.NET	hello-world	❌	N/A	❌ CLONE_FAILED
.NET	json-parse	❌	N/A	❌ CLONE_FAILED
Go	color	❌	N/A	❌ CLONE_FAILED
Go	env	❌	N/A	❌ CLONE_FAILED
Go	uuid	❌	N/A	❌ CLONE_FAILED
Java	gson	❌	N/A	❌ CLONE_FAILED
Java	caffeine	❌	N/A	❌ CLONE_FAILED
Node.js	clsx	❌	N/A	❌ CLONE_FAILED
Node.js	execa	❌	N/A	❌ CLONE_FAILED
Node.js	p-limit	❌	N/A	❌ CLONE_FAILED
Rust	fd	❌	N/A	❌ CLONE_FAILED
Rust	zoxide	❌	N/A	❌ CLONE_FAILED

Overall: 0/8 ecosystems passed — ❌ FAIL

Error Details

All gh repo clone Mossaka/gh-aw-firewall-test-<ecosystem> calls failed with:

remote: access denied: unrecognized endpoint
fatal: unable to access '(localhost/redacted) The requested URL returned error: 403

The workflow's gh CLI proxy only permits access to github/gh-aw-firewall. To fix this, grant the workflow token read access to the Mossaka org repos, or pre-checkout test repositories as workflow artifacts or submodules.

Generated by Build Test Suite for issue #2171 · ● 274.4K · ◷

[Copilot]

Pull request overview

Adds a new agentic workflow to smoke-test the Gemini engine in this repo’s Agentic Workflows Firewall setup, following the existing smoke workflow patterns (pre-step data collection + agent execution + safe-outputs enforcement).

Changes:

• Introduces a new smoke-gemini workflow definition (.md) for scheduled/PR-triggered Gemini engine validation.
• Adds the compiled workflow lockfile (.lock.yml) generated from the markdown source.

Show a summary per file

File	Description
.github/workflows/smoke-gemini.md	New Gemini smoke-test workflow definition: pre-compute checks, prompt instructions, safe-outputs validation.
.github/workflows/smoke-gemini.lock.yml	Compiled lock workflow generated from the markdown source for execution in GitHub Actions.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 2/2 changed files
• Comments generated: 1

[Copilot]

This workflow doesn’t pin the MCP gateway version in frontmatter (unlike smoke-copilot), so the compiled lock ends up using gh-aw-mcpg v0.2.26. If the goal is to mirror smoke-copilot / stay on the newer gateway, add sandbox.mcp.version: v0.2.29 (or the repo’s current standard) and re-run gh aw compile to refresh the lock file.

Suggested change

	engine: gemini
	engine: gemini
	sandbox:
	mcp:
	version: v0.2.29

[github-actions]

Smoke Test Results: GitHub Actions Services Connectivity

Check	Result
Redis PING (host.docker.internal:6379)	❌ Failed — redis-cli not installed; TCP to 172.17.0.1:6379 timed out
PostgreSQL pg_isready (host.docker.internal:5432)	❌ Failed — no response (timeout)
PostgreSQL SELECT 1 (smoketest db)	❌ Failed — host unreachable (timed out before psql could connect)

All checks failed. host.docker.internal resolves to 172.17.0.1 but TCP connections to both ports 6379 and 5432 time out, suggesting the service containers are not running or not accessible from this sandbox.

🔌 Service connectivity validated by Smoke Services

[github-actions]

🔬 Copilot Smoke Test Results

Test	Status
GitHub MCP — PR #2171 "feat: add Gemini engine smoke test workflow"	✅
GitHub.com connectivity (HTTP 200)	✅
File write/read	❌ (pre-step template vars not resolved)

Overall: FAIL — pre-computed step outputs were not injected (${{ steps.smoke-data.outputs.* }} unresolved).

Author: @lpcox | Assignees: none

📰 BREAKING: Report filed by Smoke Copilot