refactor: complete Engine abstraction — move install, invocation, and paths behind Engine enum

.github/workflows/update-awf-version.md

@@ -29,7 +29,7 @@ There are four items to check:
 | Item | Upstream Source | Local Path |
 |------|---------------|------------|
 | `AWF_VERSION` | [github/gh-aw-firewall](https://github.com/github/gh-aw-firewall) latest release | `src/compile/common.rs` |
-| `COPILOT_CLI_VERSION` | [github/copilot-cli](https://github.com/github/copilot-cli) latest release | `src/compile/common.rs` |
+| `COPILOT_CLI_VERSION` | [github/copilot-cli](https://github.com/github/copilot-cli) latest release | `src/engine.rs` |
 | `MCPG_VERSION` | [github/gh-aw-mcpg](https://github.com/github/gh-aw-mcpg) latest release | `src/compile/common.rs` |
 | `ecosystem_domains.json` | [github/gh-aw](https://github.com/github/gh-aw) `pkg/workflow/data/ecosystem_domains.json` on `main` | `src/data/ecosystem_domains.json` |
 
@@ -45,7 +45,7 @@ Fetch the latest release of the upstream repository. Record the tag name, stripp
 
 ### Step 2: Read the Current Version
 
-Read the file `src/compile/common.rs` in this repository and find the corresponding constant:
+Read the file `src/compile/common.rs` (for `AWF_VERSION`, `MCPG_VERSION`) or `src/engine.rs` (for `COPILOT_CLI_VERSION`) in this repository and find the corresponding constant:
 
 - `pub const AWF_VERSION: &str = "...";`
 - `pub const COPILOT_CLI_VERSION: &str = "...";`
@@ -89,7 +89,7 @@ If the latest version is newer than the current constant:
   ```markdown
   ## Dependency Update
 
-  Updates the pinned `COPILOT_CLI_VERSION` constant in `src/compile/common.rs` from `<old-version>` to `<latest-version>`.
+  Updates the pinned `COPILOT_CLI_VERSION` constant in `src/engine.rs` from `<old-version>` to `<latest-version>`.
 
   ### Release
 

.vscode/mcp.json

@@ -0,0 +1,11 @@
+{
+  "servers": {
+    "github-agentic-workflows": {
+      "command": "gh",
+      "args": [
+        "aw",
+        "mcp-server"
+      ]
+    }
+  }
+}

.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+    "github.copilot.enable": {
+        "markdown": true
+    }
+}

AGENTS.md

@@ -346,16 +346,13 @@ engine:
 | `id` | string | `copilot` | Engine identifier. Currently only `copilot` (GitHub Copilot CLI) is supported. |
 | `model` | string | `claude-opus-4.5` | AI model to use. Options include `claude-sonnet-4.5`, `gpt-5.2-codex`, `gemini-3-pro-preview`, etc. |
 | `timeout-minutes` | integer | *(none)* | Maximum time in minutes the agent job is allowed to run. Sets `timeoutInMinutes` on the `Agent` job in the generated pipeline. |
-| `version` | string | *(none)* | Engine CLI version to install (e.g., `"0.0.422"`, `"latest"`). **Not yet wired** — parsed but ignored with a warning. |
-| `agent` | string | *(none)* | Custom agent file identifier (Copilot only). **Not yet wired** — parsed but ignored with a warning. |
-| `api-target` | string | *(none)* | Custom API endpoint hostname for GHES/GHEC (e.g., `"api.acme.ghe.com"`). **Not yet wired** — parsed but ignored with a warning. |
-| `args` | list | `[]` | Custom CLI arguments injected before the prompt. **Not yet wired** — parsed but ignored with a warning. |
-| `env` | map | *(none)* | Engine-specific environment variables. **Not yet wired** — parsed but ignored with a warning. |
-| `command` | string | *(none)* | Custom engine executable path (skips default installation). **Not yet wired** — parsed but ignored with a warning. |
+| `version` | string | *(none)* | Engine CLI version to install (e.g., `"0.0.422"`, `"latest"`). Overrides the pinned `COPILOT_CLI_VERSION`. Set to `"latest"` to use the newest available version. |
+| `agent` | string | *(none)* | Custom agent file identifier (Copilot only). Adds `--agent <name>` to the CLI invocation, selecting a custom agent from `.github/agents/`. |
+| `api-target` | string | *(none)* | Custom API endpoint hostname for GHES/GHEC (e.g., `"api.acme.ghe.com"`). Adds `--api-target <hostname>` to the CLI invocation and adds the hostname to the AWF network allowlist. |
+| `args` | list | `[]` | Custom CLI arguments appended after compiler-generated args. Subject to shell-safety validation and blocked from overriding compiler-controlled flags (`--prompt`, `--allow-tool`, `--disable-builtin-mcps`, etc.). |
+| `env` | map | *(none)* | Engine-specific environment variables merged into the sandbox step's `env:` block. Keys must be valid env var names; values must not contain ADO expressions (`$(`, `${{`) or pipeline command injection (`##vso[`). Compiler-controlled keys (`GITHUB_TOKEN`, `PATH`, `BASH_ENV`, etc.) are blocked. |
+| `command` | string | *(none)* | Custom engine executable path (skips default NuGet installation). The path must be accessible inside the AWF container (e.g., `/tmp/...` or workspace-mounted paths). |
 
-> **Deprecated:** `max-turns` is still accepted in front matter for backwards compatibility but is ignored at compile time (a warning is emitted). It was specific to Claude Code and is not supported by Copilot CLI.
-
-> **Note:** Fields marked "not yet wired" are accepted in the schema for forward compatibility with gh-aw but produce a compile-time warning. Pipeline wiring for these fields is incremental.
 
 #### `timeout-minutes`
 
@@ -562,7 +559,7 @@ The compiler transforms the input into valid Azure DevOps pipeline YAML based on
 - **Standalone**: Uses `src/data/base.yml`
 - **1ES**: Uses `src/data/1es-base.yml`
 
-Explicit markings are embedded in these templates that the compiler is allowed to replace e.g. `{{ copilot_params }}` denotes parameters which are passed to the copilot command line tool. The compiler should not replace sections denoted by `${{ some content }}`. What follows is a mapping of markings to responsibilities (primarily for the standalone template).
+Explicit markings are embedded in these templates that the compiler is allowed to replace e.g. `{{ engine_run }}` denotes the full engine invocation command. The compiler should not replace sections denoted by `${{ some content }}`. What follows is a mapping of markings to responsibilities (primarily for the standalone template).
 
 ## {{ parameters }}
 
@@ -650,18 +647,57 @@ This distinction allows resources (like templates) to be available as pipeline r
 
 Should be replaced with the human-readable name from the front matter (e.g., "Daily Code Review"). This is used for display purposes like stage names.
 
-## {{ copilot_params }}
+## {{ engine_install_steps }}
+
+Should be replaced with engine-specific pipeline steps to install the engine binary. Generated by `Engine::install_steps()`. For Copilot, this produces:
+- `NuGetAuthenticate@1` task
+- `NuGetCommand@2` task to install `Microsoft.Copilot.CLI.linux-x64` (uses `engine.version` if set, otherwise `COPILOT_CLI_VERSION` constant)
+- Bash step to copy binary to `/tmp/awf-tools/copilot`
+- Bash step to verify installation
+
+Returns empty when `engine.command` is set (user provides own binary).
 
-Additional params provided to copilot CLI. The compiler generates:
+## {{ engine_run }}
+
+Should be replaced with the full AWF `--` command string for the Agent job. Generated by `Engine::invocation()`. For Copilot, this produces:
+```
+<command_path> --prompt "$(cat /tmp/awf-tools/agent-prompt.md)" --additional-mcp-config @/tmp/awf-tools/mcp-config.json <engine args> <user args>
+```
+
+The binary path defaults to `/tmp/awf-tools/copilot` but can be overridden via `engine.command`. The engine controls how the prompt is delivered (`--prompt "$(cat ...)"`), and how MCP config is referenced (`--additional-mcp-config @...`).
+
+Engine args include:
 - `--model <model>` - AI model from `engine` front matter field (default: claude-opus-4.5)
+- `--agent <name>` - Custom agent file from `engine.agent` (selects from `.github/agents/`)
+- `--api-target <hostname>` - Custom API endpoint from `engine.api-target` (GHES/GHEC)
 - `--no-ask-user` - Prevents interactive prompts
 - `--disable-builtin-mcps` - Disables all built-in Copilot CLI MCPs (single flag, no argument)
 - `--allow-all-tools` - When bash is omitted (default) or has a wildcard (`":*"` or `"*"`), allows all tools instead of individual `--allow-tool` flags
 - `--allow-tool <tool>` - When bash is NOT wildcard, explicitly allows configured tools (github, safeoutputs, write, and shell commands from the `bash:` field plus any runtime-required commands)
 - `--allow-all-paths` - When `edit` tool is enabled (default), allows the agent to write to any file path
+- Custom args from `engine.args` — appended after compiler-generated args (subject to shell-safety validation and blocked flag checks)
 
 MCP servers are handled entirely by the MCP Gateway (MCPG) and are not passed as copilot CLI params.
 
+## {{ engine_run_detection }}
+
+Same as `{{ engine_run }}` but for the Detection (threat analysis) job. Uses a different prompt path (`/tmp/awf-tools/threat-analysis-prompt.md`) and no MCP config.
+
+## {{ engine_env }}
+
+Generates engine-specific environment variable entries for the AWF sandbox step via `Engine::env()`. For the Copilot engine, this produces:
+
+- `GITHUB_TOKEN: $(GITHUB_TOKEN)` — GitHub authentication
+- `GITHUB_READ_ONLY: 1` — Restricts GitHub API to read-only access
+- `COPILOT_OTEL_ENABLED`, `COPILOT_OTEL_EXPORTER_TYPE`, `COPILOT_OTEL_FILE_EXPORTER_PATH` — OpenTelemetry file-based tracing for agent statistics
+- Custom env vars from `engine.env` — merged after compiler-controlled vars (YAML-quoted, validated for safety)
+
+ADO access tokens (`AZURE_DEVOPS_EXT_PAT`, `SYSTEM_ACCESSTOKEN`) are not part of this marker — they are injected separately by `{{ acquire_ado_token }}` and extension pipeline variable mappings when `permissions.read` is configured.
+
+## {{ engine_log_dir }}
+
+Should be replaced with the engine's log directory path, generated by `Engine::log_dir()`. For Copilot: `~/.copilot/logs`. Used by log collection steps to copy engine logs to pipeline artifacts.
+
 ## {{ pool }}
 
 Should be replaced with the agent pool name from the `pool` front matter field. Defaults to `AZS-1ES-L-MMS-ubuntu-22.04` if not specified.
@@ -845,8 +881,9 @@ This ensures the Copilot CLI config reflects MCPG's actual runtime state rather
 Should be replaced with the comma-separated domain list for AWF's `--allow-domains` flag. The list includes:
 1. Core Azure DevOps/GitHub endpoints (from `allowed_hosts.rs`)
 2. MCP-specific endpoints for each enabled MCP
-3. Ecosystem identifier expansions from `network.allowed:` (e.g., `python` → PyPI/pip domains)
-4. User-specified additional hosts from `network.allowed:` front matter
+3. Engine-required hosts (e.g., `engine.api-target` hostname for GHES/GHEC)
+4. Ecosystem identifier expansions from `network.allowed:` (e.g., `python` → PyPI/pip domains)
+5. User-specified additional hosts from `network.allowed:` front matter
 
 The output is formatted as a comma-separated string (e.g., `github.com,*.dev.azure.com,api.github.com`).
 
@@ -884,16 +921,6 @@ The step:
 
 If `permissions.read` is not configured, this marker is replaced with an empty string.
 
-## {{ engine_env }}
-
-Generates engine-specific environment variable entries for the AWF sandbox step via `Engine::env()`. For the Copilot engine, this produces:
-
-- `GITHUB_TOKEN: $(GITHUB_TOKEN)` — GitHub authentication
-- `GITHUB_READ_ONLY: 1` — Restricts GitHub API to read-only access
-- `COPILOT_OTEL_ENABLED`, `COPILOT_OTEL_EXPORTER_TYPE`, `COPILOT_OTEL_FILE_EXPORTER_PATH` — OpenTelemetry file-based tracing for agent statistics
-
-ADO access tokens (`AZURE_DEVOPS_EXT_PAT`, `SYSTEM_ACCESSTOKEN`) are not part of this marker — they are injected separately by `{{ acquire_ado_token }}` and extension pipeline variable mappings when `permissions.read` is configured.
-
 ## {{ acquire_write_token }}
 
 Generates an `AzureCLI@2` step that acquires a write-capable ADO-scoped access token from the ARM service connection specified in `permissions.write`. This token is used only by the executor in Stage 3 (`Execution` job) and is never exposed to the agent.
@@ -951,12 +978,7 @@ Should be replaced with the domain the AWF-sandboxed agent uses to reach MCPG on
 
 ## {{ copilot_version }}
 
-Should be replaced with the pinned version of the `Microsoft.Copilot.CLI.linux-x64` NuGet package (defined as `COPILOT_CLI_VERSION` constant in `src/compile/common.rs`). This version is used in the pipeline step that installs the Copilot CLI tool from Azure Artifacts.
-
-The generated pipelines install the package from:
-```
-https://pkgs.dev.azure.com/msazuresphere/_packaging/Guardian1ESPTUpstreamOrgFeed/nuget/v3/index.json
-```
+**Removed.** This marker has been absorbed into `{{ engine_install_steps }}`. The `COPILOT_CLI_VERSION` constant now lives in `src/engine.rs` and is used internally by `Engine::install_steps()`. The version can be overridden per-agent via `engine: { id: copilot, version: "..." }` in front matter.
 
 ### 1ES-Specific Template Markers