Skip to content

πŸ“ Documentation drift detected β€” prompts file has broken MCP examples and missing safe-output toolsΒ #227

@github-actions

Description

@github-actions

Documentation Freshness Audit

The weekly documentation audit found the following inconsistencies between code and documentation:

Findings

Area Issue File(s)
MCP server configuration Complete Example and Common Patterns use mcp-servers: ado: true in standalone context, but built-in MCPs without container: are silently skipped in standalone β€” agents will produce broken pipelines prompts/create-ado-agentic-workflow.md
Safe output tools Step 9 only documents 2 of 17 safe output tools; 15 tools are completely absent from the prompt prompts/create-ado-agentic-workflow.md
Architecture tree src/agent_stats.rs exists in the codebase but is missing from the architecture directory tree AGENTS.md
CLI commands mcp and mcp-http commands have an --enabled-tools flag that is not documented in the CLI Commands section AGENTS.md

Details

1. 🚨 Built-in MCP examples broken in standalone context

File: prompts/create-ado-agentic-workflow.md

Step 8 ("MCP Servers") lists "built-in" MCPs (ado, ado-ext, kusto, icm, bluebird, es-chat, msft-learn, stack, asa, calculator) and correctly includes a note:

Standalone target (the default): Built-in MCPs (entries without a container: or url: field) are silently skipped at compile time β€” they have no effect and will not be available to the agent.

However, the Complete Example at the bottom uses mcp-servers: ado: true without specifying target: 1es β€” so the agent in the example will have no ADO MCP available. Similarly, all three Common Patterns use mcp-servers: ado: true in standalone contexts.

For standalone target, the correct way to get Azure DevOps tooling is:

tools:
  azure-devops: true
```
This is a first-class tool (defined in `src/tools/cache_memory.rs`-pattern, configured in `src/compile/extensions.rs`) that generates the proper MCPG container config.

Any agent following the Complete Example will produce a pipeline where ADO tools are silently unavailable.

#### 2. πŸ“‹ Step 9 documents only 2 of 17 safe output tools

**File**: `prompts/create-ado-agentic-workflow.md`

Step 9 ("Safe Outputs") only covers `create-pull-request` and `create-work-item` in detail, with a single note: *"Other safe output tools (no configuration needed): `noop`, `missing-data`, `missing-tool`, `report-incomplete`."*

The following 13 configurable safe output tools are completely absent from the prompt:

| Tool | Purpose |
|------|---------|
| `comment-on-work-item` | Add a comment to an ADO work item |
| `update-work-item` | Update fields on an existing work item |
| `add-pr-comment` | Add a new comment thread to a PR |
| `reply-to-pr-comment` | Reply to an existing PR review thread |
| `resolve-pr-thread` | Resolve or update the status of a PR thread |
| `submit-pr-review` | Submit a review vote on a PR |
| `update-pr` | Update PR metadata (reviewers, labels, auto-complete, etc.) |
| `link-work-items` | Link two ADO work items |
| `queue-build` | Queue a pipeline build by definition ID |
| `create-git-tag` | Create a git tag on a repository ref |
| `add-build-tag` | Add a tag to an ADO build |
| `create-branch` | Create a new branch from an existing ref |
| `upload-attachment` | Upload a workspace file as a work item attachment |
| `create-wiki-page` | Create a new ADO wiki page |
| `update-wiki-page` | Update the content of an existing wiki page |

Agents generating workflows have no way to discover these tools and will default to the only two they know about.

#### 3. `src/agent_stats.rs` missing from architecture tree

**File**: `AGENTS.md` (Architecture section)

Running `find src -type f -name '*.rs' | sort` reveals `src/agent_stats.rs` exists but is not listed in the architecture directory tree. The file provides OTel-based agent statistics parsing (token usage, duration, turns) used during Stage 2 execution.

The current tree jumps from `sanitize.rs` directly to `safeoutputs/` with no mention of `agent_stats.rs`.

#### 4. `--enabled-tools` flag undocumented for `mcp` and `mcp-http`

**File**: `AGENTS.md` (CLI Commands section)

Both the `mcp` and `mcp-http` commands accept `--enabled-tools <name>` (repeatable) which restricts which safe output tools are exposed. This flag is documented for the `\{\{ enabled_tools_args }}` template marker but not in the CLI Commands section:

```
- `mcp <output_directory> <bounding_directory>` - Run SafeOutputs as a stdio MCP server
- `mcp-http <output_directory> <bounding_directory>` - Run SafeOutputs as an HTTP MCP server
  - `--port <port>` - Port to listen on (default: 8100)
  - `--api-key <key>` - API key for authentication (auto-generated if not provided)
  ← missing: --enabled-tools <name>  (can be repeated; if omitted, all tools are exposed)

Suggested Fixes

  • Critical: In prompts/create-ado-agentic-workflow.md Complete Example, replace mcp-servers: ado: true with tools: azure-devops: true (standalone-compatible). Update all three Common Patterns similarly β€” or annotate them as target: 1es examples.
  • Significant: In prompts/create-ado-agentic-workflow.md Step 9, add a table or brief descriptions for all 15 configurable safe output tools so agents know what's available.
  • Minor: In AGENTS.md architecture tree, add β”‚ β”œβ”€β”€ agent_stats.rs # Agent execution statistics (OTel JSONL parsing) after sanitize.rs.
  • Minor: In AGENTS.md CLI Commands section, add --enabled-tools <name> to both mcp and mcp-http command descriptions.

This issue was created by the automated documentation freshness check.

Generated by Documentation Freshness Check Β· ● 1.3M Β· β—·

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions