Skip to content

[cli-tools-test] Smoke test workflows fail strict-mode compilation due to sandbox.mcp.container #22604

@github-actions

Description

@github-actions

Found during daily exploratory testing of audit, logs, and compile tools (run §23474161949).

Problem Description

All four smoke test workflows fail to compile when strict mode is enabled because they use sandbox.mcp.container, which is now classified as an internal implementation detail and rejected by the strict-mode validator.

Affected Workflows

Workflow Field Error Line
smoke-copilot.md sandbox.mcp.container 19:1
smoke-claude.md sandbox.mcp.container (similar)
smoke-codex.md sandbox.mcp.container (similar)
smoke-copilot-arm.md sandbox.mcp.container (similar)

Compile Error Message

strict mode: 'sandbox.mcp.container' is not allowed because it is an internal
implementation detail. Remove 'sandbox.mcp.container' or set 'strict: false'
to disable strict mode.
See: https://github.github.com/gh-aw/reference/sandbox/

Current smoke-copilot.md Configuration

sandbox:
  mcp:
    container: "ghcr.io/github/gh-aw-mcpg"
```

### Steps to Reproduce

```
Use the agentic-workflows compile MCP tool (compiles all workflows)
```

Observe 4 smoke workflows reported as `valid: false`.

### Expected Behavior

Smoke test workflows should either:
1. Compile cleanly under strict mode (after removing `sandbox.mcp.container`), **or**
2. Have `strict: false` explicitly set since they intentionally test internal implementation details

### Actual Behavior

All 4 smoke workflows fail strict mode compilation — 10 out of 177 workflows total fail compilation (the other 6 have a separate `vulnerability-alerts` permission issue for the `dependabot` toolset).

### Additional Context: Other Compile Failures

6 additional workflows also fail to compile with a different error:

```
Missing required permissions for GitHub toolsets:
  - vulnerability-alerts: read (required by dependabot)

Affected: daily-firewall-report.md, deep-report.md, dependabot-go-checker.md, github-mcp-structural-analysis.md, github-mcp-tools-report.md, security-review.md

These use toolsets: [all] which includes dependabot, but are missing the vulnerability-alerts: read permission.

Environment

  • Repository: github/gh-aw
  • Testing Run ID: §23474161949
  • Date: 2026-03-24
  • Total workflows checked: 177

Impact

  • Severity: High
  • Frequency: Always (reproducible every time compile runs)
  • Workaround: Set strict: false in each smoke workflow's frontmatter, or remove sandbox.mcp.container and use the default MCP container

Recommendation

Add strict: false to smoke workflow frontmatter since they intentionally exercise internal implementation details, or update them to use the new non-internal API for specifying MCP containers.

Generated by Daily CLI Tools Exploratory Tester ·

  • expires on Mar 31, 2026, 5:20 AM UTC

Metadata

Metadata

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions