Automated smoke test results for Claude engine validation.
Summary
| Status |
Count |
| ✅ Pass |
17 |
| ⚠️ Skip |
2 |
| ❌ Fail |
0 |
Overall Status: PARTIAL (2 tests skipped, all others passed)
PR Used for Review Testing: #26037 — Fix gh pr checkout failing with GH_HOST mismatch
Test Results
Core Tests (#1–12)
| # |
Test |
Status |
Notes |
| 1 |
GitHub MCP Testing |
✅ |
Last 2 merged: #26095, #26091 |
| 2 |
MCP Scripts GH CLI Testing |
✅ |
Got PRs #26060, #26037 |
| 3 |
Serena MCP Testing |
✅ |
Project activated; found 6 symbols (CompileWorkflow, validateWorkflowData, etc.) |
| 4 |
Make Build Testing |
✅ |
make build succeeded — binary compiled at ./gh-aw |
| 5 |
Playwright Testing |
✅ |
github.com loaded; title contains "GitHub" |
| 6 |
Tavily Web Search Testing |
✅ |
5 results returned for "GitHub Agentic Workflows" |
| 7 |
File Writing Testing |
✅ |
/tmp/gh-aw/agent/smoke-test-claude-24370050883.txt created |
| 8 |
Bash Tool Testing |
✅ |
File content verified: "Smoke test passed for Claude at Mon Apr 13 22:28:43 UTC 2026" |
| 9 |
Discussion Interaction Testing |
✅ |
Discussion #26096 found; comic-book comment added |
| 10 |
Agentic Workflows MCP Testing |
✅ |
smoke-claude: active, compiled, triggers on PR label/reaction/schedule |
| 11 |
Slack Script Safe Output Testing |
✅ |
Posted to #smoke-tests |
| 12 |
Code Scanning Alert Testing |
✅ |
Dummy warning alert created for README.md:1 |
PR Review Tests (#13–19)
| # |
Test |
Status |
Notes |
| 13 |
Update PR Testing |
✅ |
Appended test message to PR #26037 |
| 14 |
PR Review Comment Testing |
✅ |
2 review comments on git_helpers.cjs and checkout_pr_branch.cjs |
| 15 |
Submit PR Review Testing |
✅ |
COMMENT review submitted on PR #26037 |
| 16 |
Resolve Review Thread Testing |
⚠️ |
Skipped — PRRT thread node IDs not available from get_review_comments response |
| 17 |
Add Reviewer Testing |
✅ |
Added copilot as reviewer to PR #26037 |
| 18 |
Push to PR Branch Testing |
✅ |
smoke-test-files/smoke-claude-push-test.md committed and pushed to PR #26037 branch |
| 19 |
Close PR Testing |
⚠️ |
Skipped — no safe test/bot PR to close |
Details
Test #3 — Serena Symbols Found
File: pkg/workflow/compiler.go
normalizeHeredocDelimiters (Function)
MaxLockFileSize, MaxExpressionSize, MaxPromptChunkSize, MaxPromptChunks (Constants)
(*Compiler).CompileWorkflow
(*Compiler).validateWorkflowData
(*Compiler).generateAndValidateYAML
(*Compiler).writeWorkflowOutput
(*Compiler).readLockFileFromHEAD
(*Compiler).CompileWorkflowData
Test #10 — Agentic Workflows Status
Workflow: smoke-claude
Engine: claude
Compiled: Yes
Status: active
Triggers: pull_request (labeled "smoke"), reaction "heart", schedule every 12h, status-comment
Run: §24370050883
Timestamp: 2026-04-13T22:30 UTC
💥 [THE END] — Illustrated by Smoke Claude · ● 268.4K · ◷
Automated smoke test results for Claude engine validation.
Summary
Overall Status: PARTIAL (2 tests skipped, all others passed)
PR Used for Review Testing: #26037 — Fix
gh pr checkoutfailing with GH_HOST mismatchTest Results
Core Tests (#1–12)
make buildsucceeded — binary compiled at./gh-aw/tmp/gh-aw/agent/smoke-test-claude-24370050883.txtcreatedPR Review Tests (#13–19)
git_helpers.cjsandcheckout_pr_branch.cjsget_review_commentsresponsecopilotas reviewer to PR #26037smoke-test-files/smoke-claude-push-test.mdcommitted and pushed to PR #26037 branchDetails
Test #3 — Serena Symbols Found
File:
pkg/workflow/compiler.gonormalizeHeredocDelimiters(Function)MaxLockFileSize,MaxExpressionSize,MaxPromptChunkSize,MaxPromptChunks(Constants)(*Compiler).CompileWorkflow(*Compiler).validateWorkflowData(*Compiler).generateAndValidateYAML(*Compiler).writeWorkflowOutput(*Compiler).readLockFileFromHEAD(*Compiler).CompileWorkflowDataTest #10 — Agentic Workflows Status
Run: §24370050883
Timestamp: 2026-04-13T22:30 UTC