Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/test-mcp-e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: MCP E2E

on:
pull_request:
paths:
- "src/specleft/mcp/**"
- "src/specleft/commands/mcp.py"
- "tests/mcp/e2e_stdio.py"
- "pyproject.toml"
- ".github/workflows/test-mcp-e2e.yml"

jobs:
e2e:
name: "MCP E2E (Python ${{ matrix.python-version }})"
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

- name: Build and install wheel with MCP runtime deps
run: |
python -m pip install --upgrade pip
python -m pip install build
python -m build
WHEEL_PATH=$(ls dist/*.whl | head -n1)
python -m pip install "${WHEEL_PATH}[mcp]"

- name: Verify MCP server stdio behavior
run: python tests/mcp/e2e_stdio.py
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -233,3 +233,5 @@ prd.md
.licenses/policy.yml
bandit-report.json
PLAN.md
.mcpregistry_github_token
.mcpregistry_registry_token
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ SHELL := /bin/sh

BADGE_OUTPUT ?= .github/assets/spec-coverage-badge.svg

.PHONY: test pre-commit lint lint-fix badge
.PHONY: test pre-commit lint lint-fix badge test-mcp-e2e

test:
pytest tests/ -v -rs
Expand All @@ -21,3 +21,8 @@ lint-fix:

badge:
SPECLEFT_BADGE_OUTPUT="$(BADGE_OUTPUT)" python3 scripts/update_spec_coverage_badge.py

test-mcp-e2e: ## Run MCP stdio E2E against an installed wheel in a clean container
python -m build
docker build -f mcp/test-mcp.Dockerfile -t specleft-mcp-e2e .
docker run --rm specleft-mcp-e2e
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# SpecLeft: Planning-First Workflow for pytest

![Spec coverage](.github/assets/spec-coverage-badge.svg)
[![MCP Registry](https://img.shields.io/badge/MCP-Registry-blue)](https://registry.modelcontextprotocol.io/servers/io.github.specleft/specleft)

SpecLeft keeps feature intent and test coverage aligned by turning plans into version-controlled specs, then generating pytest test skeletons from those specs.

Expand Down Expand Up @@ -118,10 +119,14 @@ specleft skill verify --format json

## MCP Server Setup

SpecLeft includes an MCP server so agents can read specs, track status, and generate test scaffolding without leaving the conversation.
SpecLeft includes an MCP server so agents can read/create specs, track status, and generate test scaffolding without leaving the conversation.

See [GET_STARTED.md](https://github.com/SpecLeft/specleft/blob/main/GET_STARTED.md) for setup details.

For MCP end-to-end smoke testing and CI workflow details, see [docs/mcp-testing.md](https://github.com/SpecLeft/specleft/blob/main/docs/mcp-testing.md).

<!-- mcp-name: io.github.SpecLeft/specleft -->

## CI Enforcement Early Access

Want to enforce feature coverage and policy checks in CI with `specleft enforce`? Join Early Access to get setup guidance and rollout support.
Expand Down
59 changes: 59 additions & 0 deletions docs/mcp-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# MCP Testing

This document covers end-to-end testing for the SpecLeft MCP server as an installed package.

## Goal

Catch packaging/runtime issues that in-memory MCP tests do not catch:

- broken `python -m specleft.mcp` entrypoint
- missing runtime dependencies in built wheel
- import-time failures in installed package
- stdio protocol regressions

## Local E2E Smoke Test

Run:

```bash
make test-mcp-e2e
```

This target:

1. Builds wheel artifacts (`python -m build`)
2. Builds a clean container from `mcp/test-mcp.Dockerfile`
3. Installs the wheel with MCP extras (`[mcp]`)
4. Runs `tests/mcp/e2e_stdio.py`

## What `tests/mcp/e2e_stdio.py` Verifies

- MCP initialize handshake succeeds
- `resources/list` returns exactly:
- `specleft://contract`
- `specleft://guide`
- `specleft://status`
- `tools/list` returns exactly one tool: `specleft_init`
- `resources/read` for `specleft://contract` returns JSON with `guarantees`
- Exit code is `0` on success, `1` on any failure

## CI Workflow

Workflow file:

- `.github/workflows/test-mcp-e2e.yml`

Trigger:

- Pull requests that touch MCP server code, E2E script, or `pyproject.toml`

Matrix:

- Python 3.10, 3.11, 3.12

The workflow builds the wheel, installs it with `[mcp]`, and executes `python tests/mcp/e2e_stdio.py`.

## Notes

- The current MCP server transport behavior is newline-delimited JSON-RPC over stdio; the E2E script validates this behavior directly.
- Unit/integration MCP tests in `tests/mcp/test_server.py` and `tests/mcp/test_security.py` should still run alongside this smoke test.
13 changes: 13 additions & 0 deletions mcp/test-mcp.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM python:3.11-slim

WORKDIR /app

COPY dist/*.whl /app/
RUN WHEEL_PATH="$(ls /app/*.whl | head -n1)" && \
python -m pip install --no-cache-dir "${WHEEL_PATH}[mcp]"

RUN python -c "import specleft.mcp"

COPY tests/mcp/e2e_stdio.py /app/e2e_stdio.py

ENTRYPOINT ["python", "/app/e2e_stdio.py"]
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ specleft = ["py.typed", "templates/*.jinja2"]

[project]
name = "specleft"
version = "0.2.2"
version = "0.3.0"
description = "A planning-first CLI for AI coding agents to externalize intent before writing code, with optional CI enforcement for Python projects."
readme = "README.md"
requires-python = ">=3.10"
Expand Down
4 changes: 4 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Development dependencies (pinned for reproducible installs)
pytest-cov==4.0.0
pytest-subtests==0.15.0
pytest-asyncio==0.25.3
tiktoken==0.12.0
fastmcp<3
build
black==26.1.0
ruff==0.5.6
mypy==1.10.0
Expand Down
7 changes: 5 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Runtime dependencies (pinned for reproducible installs)
pytest==7.0.0
pydantic==2.0.0
pytest==8.3.5
pydantic==2.12.2
click==8.0.0
jinja2==3.1.6
python-frontmatter==1.0.1
python-slugify==8.0.4
pyyaml==6.0.2
cryptography==45.0.7
31 changes: 31 additions & 0 deletions server.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
"name": "io.github.SpecLeft/specleft",
"title": "SpecLeft",
"description": "Track and enforce Python feature coverage with verifiable safety guarantees. Generates pytest test scaffolding from markdown specifications, monitors implementation progress, and blocks PRs that violate coverage policies. Agent-optimised: 3 resources, 1 tool, offline-only, no telemetry, no API keys required.",
"version": "0.3.0",
"websiteUrl": "https://specleft.dev",
"repository": {
"url": "https://github.com/SpecLeft/specleft",
"source": "github"
},
"packages": [
{
"registryType": "pypi",
"registryBaseUrl": "https://pypi.org",
"identifier": "specleft",
"version": "0.3.0",
"runtimeHint": "uvx",
"transport": {
"type": "stdio"
},
"packageArguments": [
{
"type": "positional",
"value": "mcp"
}
],
"environmentVariables": []
}
]
}
4 changes: 3 additions & 1 deletion src/specleft/commands/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@

CLI_VERSION = SPECLEFT_VERSION
CONTRACT_VERSION = "1.1"
CONTRACT_DOC_PATH = "docs/agent-contract.md"
CONTRACT_DOC_PATH = (
"https://github.com/SpecLeft/specleft/blob/main/docs/agent-contract.md"
)
7 changes: 6 additions & 1 deletion src/specleft/commands/enforce.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,12 @@ def _augment_violations_with_fix_commands(


@click.command("enforce")
@click.argument("policy_file", type=click.Path(exists=False), default=None)
@click.argument(
"policy_file",
type=click.Path(exists=False),
required=False,
default=None,
)
@click.option(
"--format",
"fmt",
Expand Down
Loading
Loading