Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/publish-modules.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Publish module tarball and checksum when a release tag is pushed.
# Tag format: {module-name}-v{version} (e.g. module-registry-v0.1.3, backlog-v0.29.0)
#
# Optional signing: set repository secrets SPECFACT_MODULE_PRIVATE_SIGN_KEY (PEM string)
# and SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE to sign the module manifest before packaging.
name: Publish Modules

on:
workflow_dispatch:
inputs:
module_path:
description: "Path to module directory (e.g. src/specfact_cli/modules/module_registry)"
required: true
push:
tags:
- "*-v*"

jobs:
publish:
name: Validate and package module
runs-on: ubuntu-latest
permissions:
contents: read
env:
SPECFACT_MODULE_PRIVATE_SIGN_KEY: ${{ secrets.SPECFACT_MODULE_PRIVATE_SIGN_KEY }}
SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE: ${{ secrets.SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE }}
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pyyaml beartype icontract cryptography cffi

- name: Resolve module path from tag
id: resolve
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
run: |
TAG="${GITHUB_REF#refs/tags/}"
NAME="${TAG%-v*}"
VERSION="${TAG#*-v}"
echo "module_name=${NAME}" >> "$GITHUB_OUTPUT"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
NAME_NORM=$(echo "$NAME" | tr '-' '_')
if [ -d "src/specfact_cli/modules/${NAME_NORM}" ]; then
echo "module_path=src/specfact_cli/modules/${NAME_NORM}" >> "$GITHUB_OUTPUT"
elif [ -d "modules/${NAME}" ]; then
echo "module_path=modules/${NAME}" >> "$GITHUB_OUTPUT"
else
echo "module_path=src/specfact_cli/modules/${NAME_NORM}" >> "$GITHUB_OUTPUT"
fi

- name: Resolve module path (manual)
id: resolve_manual
if: github.event_name == 'workflow_dispatch'
run: |
echo "module_path=${{ github.event.inputs.module_path }}" >> "$GITHUB_OUTPUT"

- name: Sign module manifest (optional)
if: secrets.SPECFACT_MODULE_PRIVATE_SIGN_KEY != ""
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
MODULE_PATH="${{ github.event.inputs.module_path }}"
else
MODULE_PATH="${{ steps.resolve.outputs.module_path }}"
fi
MANIFEST="${MODULE_PATH}/module-package.yaml"
if [ -f "$MANIFEST" ]; then
python scripts/sign-modules.py "$MANIFEST"
fi

- name: Publish module
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
MODULE_PATH="${{ github.event.inputs.module_path }}"
else
MODULE_PATH="${{ steps.resolve.outputs.module_path }}"
fi
mkdir -p dist
python scripts/publish-module.py "$MODULE_PATH" -o dist

- name: Upload module artifacts
uses: actions/upload-artifact@v4
with:
name: module-package
path: |
dist/*.tar.gz
dist/*.sha256
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,5 @@ Language.ml
Language.mli

.artifacts
registry.bak/
.pr-body.md
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ All notable changes to this project will be documented in this file.
**Important:** Changes need to be documented below this block as this is the header section. Each section should be separated by a horizontal rule. Newer changelog entries need to be added on top of prior ones to keep the history chronological with most recent changes first.


---

## [0.38.0] - 2026-02-27

### Added

- **Module dependency resolution**: Install resolves `pip_dependencies` and `module_dependencies` before installing marketplace modules; conflict detection with clear errors. Use `--skip-deps` to bypass resolution or `--force` to override conflicts.
- **Command aliases**: `specfact module alias create/list/remove` to map custom command names to module commands. Aliases stored in `~/.specfact/registry/aliases.json`. Aliases do not create top-level CLI commands (CLI surface unchanged).
- **Custom registries**: `specfact module add-registry`, `list-registries`, `remove-registry` to configure additional module registries with priority and trust levels (`always` / `prompt` / `never`). Config in `~/.specfact/config/registries.yaml`. Search queries all configured registries and shows a **Registry** column when multiple exist.
- **Namespace enforcement**: Marketplace modules must use `namespace/name` format; invalid format or name collisions are rejected with guidance (alias or uninstall).
- **Module publishing**: `scripts/publish-module.py` to validate, package (tarball + SHA-256), optionally sign, and write registry index fragments. `.github/workflows/publish-modules.yml` runs on tags `*-v*` and workflow_dispatch, with optional signing via `SPECFACT_MODULE_PRIVATE_SIGN_KEY` and `SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE` secrets.
- **Documentation**: New guides publishing-modules.md, custom-registries.md, reference dependency-resolution.md. Updated installing-modules.md, module-marketplace.md, module-signing-and-key-rotation.md, and commands reference.

---

## [0.37.5] - 2026-02-25
Expand Down
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ For implementation details, see:
- [Module Contracts](reference/module-contracts.md)
- [Installing Modules](guides/installing-modules.md)
- [Module Marketplace](guides/module-marketplace.md)
- [Custom registries](guides/custom-registries.md)
- [Publishing modules](guides/publishing-modules.md)
- [Module Signing and Key Rotation](guides/module-signing-and-key-rotation.md)

---
Expand Down
2 changes: 2 additions & 0 deletions docs/guides/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ Practical guides for using SpecFact CLI effectively.
- **[Troubleshooting](troubleshooting.md)** - Common issues and solutions
- **[Installing Modules](installing-modules.md)** - Install, list, show, search, enable/disable, uninstall, and upgrade modules
- **[Module Marketplace](module-marketplace.md)** - Discovery priority, trust vs origin semantics, and security model
- **[Custom registries](custom-registries.md)** - Add, list, remove registries; trust levels and priority
- **[Publishing modules](publishing-modules.md)** - Package, sign, and publish modules to a registry
- **[Module Signing and Key Rotation](module-signing-and-key-rotation.md)** - Public key placement, signing workflow, CI verification, rotation, and revocation runbook
- **[Competitive Analysis](competitive-analysis.md)** - How SpecFact compares to other tools
- **[Operational Modes](../reference/modes.md)** - CI/CD vs CoPilot modes (reference)
Expand Down
78 changes: 78 additions & 0 deletions docs/guides/custom-registries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
layout: default
title: Custom registries
permalink: /guides/custom-registries/
description: Add, list, and manage custom module registries with trust levels and priority.
---

# Custom registries

SpecFact can use multiple module registries: the official registry plus private or third-party registries. You control which registries are used, their priority, and how much to trust them.

## Adding registries

```bash
# Add by URL (id derived from URL if not given)
specfact module add-registry https://company.example.com/specfact/registry/index.json

# With explicit id, priority, and trust
specfact module add-registry https://company.example.com/specfact/registry/index.json \
--id company-registry \
--priority 10 \
--trust always
```

- **URL**: Must point to a JSON index that follows the same schema as the official registry (e.g. `modules` array with `id`, `latest_version`, `description`, etc.).
- **--id**: Optional. Default is derived from the URL or `custom`. Use a short, stable id for `remove-registry` and for the **Registry** column in search results.
- **--priority**: Optional. Lower number = higher priority. Default is next available (after existing priorities). Official registry is always first.
- **--trust**: `always` (use without prompting), `prompt` (ask once per registry), or `never` (do not use). Default is `prompt`.

Config is stored in `~/.specfact/config/registries.yaml`.

## Listing and removing

```bash
# List all configured registries (official + custom)
specfact module list-registries

# Remove a custom registry by id
specfact module remove-registry company-registry
```

The official registry cannot be removed; only custom entries are modified.

## Trust levels

| Trust | Behavior |
|----------|----------|
| `always` | Use this registry without prompting. Prefer for internal/private registries. |
| `prompt` | Ask the user once whether to trust this registry (e.g. first install/search from it). |
| `never` | Do not use this registry. Use to disable without removing the config. |

Choose `always` for fully controlled internal registries; use `prompt` for unknown or third-party registries.

## Priority

When multiple registries are configured, they are queried in order: official first, then custom registries by ascending priority number. Search and install use this order; the first matching module id wins. Use priority to prefer an internal registry over the official one for overlapping names (e.g. `specfact/backlog` from your mirror).

## Search across registries

`specfact module search <query>` queries all configured registries and local modules. Results include a **Registry** column when more than one registry is configured, so you can see which registry each module came from.

## Enterprise use

- **Private index**: Host a JSON index (and tarballs) on an internal server or artifact store. Add it with `add-registry` and `--trust always`.
- **Air-gapped / proxy**: Serve a mirror of the official index (and artifacts) behind your proxy; point `add-registry` at the mirror URL.
- **Multiple teams**: Use different registry ids and priorities so team-specific registries are tried in the right order.

## Security considerations

- Only add registries from trusted sources; index and tarballs can be tampered with if the server is compromised.
- Use HTTPS for registry URLs.
- Integrity checks (checksum/signature) still apply to downloaded modules; custom registries do not bypass verification.

## See also

- [Module marketplace](module-marketplace.md) – Discovery and security model.
- [Installing modules](installing-modules.md) – Install, list, search, and upgrade.
- [Publishing modules](publishing-modules.md) – Package and publish modules to a registry.
41 changes: 40 additions & 1 deletion docs/guides/installing-modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,43 @@ Notes:
- If a module is already available locally (`built-in` or `custom`), install is skipped with a clear message.
- Invalid ids show an explicit error (`name` or `namespace/name` only).

## Dependency resolution

Before installing a marketplace module, SpecFact resolves its dependencies (other modules and optional pip packages) from manifest `pip_dependencies` and `module_dependencies`. If conflicts are detected (e.g. incompatible versions), install fails unless you override.

```bash
# Install with dependency resolution (default)
specfact module install specfact/backlog

# Skip dependency resolution (install only the requested module)
specfact module install specfact/backlog --skip-deps

# Force install despite dependency conflicts (use with care)
specfact module install specfact/backlog --force
```

- Use `--skip-deps` when you want to install a single module without pulling its dependencies or when you manage dependencies yourself.
- Use `--force` to proceed when resolution reports conflicts (e.g. for local overrides or known-compatible versions). Enable/disable and dependency-aware cascades still respect `--force` where applicable.

See [Dependency resolution](../reference/dependency-resolution.md) for how resolution works and conflict detection.

## Command aliases

You can alias a command name to a module-provided command so that a shorter or custom name invokes the same logic.

```bash
# Create an alias (e.g. "bp" for backlog’s "plan" command)
specfact module alias create bp backlog plan

# List all aliases
specfact module alias list

# Remove an alias
specfact module alias remove bp
```

Aliases are stored under `~/.specfact/registry/aliases.json`. **Aliases do not create or resolve top-level CLI commands**—the CLI surface stays the same; aliases are for reference and organization only. When you run a command, the registry resolves aliases first; if an alias would shadow a built-in command, a warning is shown. Use `--force` on create to override the shadow warning.

## Security and Trust Controls

- Denylist file: `~/.specfact/module-denylist.txt`
Expand Down Expand Up @@ -95,9 +132,11 @@ This prints detailed metadata:
specfact module search bundle-mapper
```

Search queries **all configured registries** (official first, then custom in priority order) plus locally discovered modules. Results show a **Registry** column when multiple registries are configured.

Search includes both:

- Marketplace registry entries (`scope=marketplace`)
- Marketplace registry entries (`scope=marketplace`) from every registry
- Locally discovered modules (`scope=installed`)

Results are sorted alphabetically by module id.
Expand Down
24 changes: 19 additions & 5 deletions docs/guides/module-marketplace.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,18 @@ SpecFact supports centralized marketplace distribution with local multi-source d

## Registry Overview

- Registry repository: <https://github.com/nold-ai/specfact-cli-modules>
- Index document: `registry/index.json`
- Marketplace module id format: `namespace/name` (for example `specfact/backlog`)
- **Official registry**: <https://github.com/nold-ai/specfact-cli-modules> (index: `registry/index.json`)
- **Marketplace module id format**: `namespace/name` (e.g. `specfact/backlog`). Marketplace modules must use this format; flat names are allowed only for custom/local modules with a warning.
- **Custom registries**: You can add private or third-party registries. See [Custom registries](custom-registries.md) for adding, listing, removing, trust levels, and priority.

## Custom registries and search

- **Add a registry**: `specfact module add-registry <index-url> [--id <id>] [--priority <n>] [--trust always|prompt|never]`
- **List registries**: `specfact module list-registries` (official is always first; custom registries follow by priority)
- **Remove a registry**: `specfact module remove-registry <registry-id>`
- **Search**: `specfact module search <query>` queries all configured registries; results show which registry each module came from.

Trust levels for custom registries: `always` (trust without prompt), `prompt` (ask once), `never` (do not use). Config is stored in `~/.specfact/config/registries.yaml`.

## Discovery and Priority

Expand Down Expand Up @@ -51,6 +60,11 @@ Install workflow enforces integrity and compatibility checks:

Checksum mismatch blocks installation.

**Namespace enforcement**:

- Modules installed from the marketplace must use the `namespace/name` format (e.g. `specfact/backlog`). Invalid format is rejected.
- If a module with the same logical name is already installed from a different source or namespace, install reports a collision and suggests using an alias or uninstalling the existing module.

Additional local hardening:

- Denylist enforcement via `~/.specfact/module-denylist.txt` (or `SPECFACT_MODULE_DENYLIST_FILE`)
Expand All @@ -67,7 +81,7 @@ Release signing automation:
- Wrapper alternative: `bash scripts/sign-module.sh --key-file "$KEY_FILE" <manifest>`
- Without key material, the script fails by default and recommends `--key-file`; checksum-only mode is explicit via `--allow-unsigned` (local testing only)
- Encrypted keys are supported with passphrase via `--passphrase`, `--passphrase-stdin`, or `SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE`
- CI workflows inject private key material via `SPECFACT_MODULE_PRIVATE_SIGN_KEY` and passphrase via `SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE`
- CI workflows inject private key material via `SPECFACT_MODULE_PRIVATE_SIGN_KEY` (inline PEM string) or `SPECFACT_MODULE_PRIVATE_SIGN_KEY_FILE` (path), and passphrase via `SPECFACT_MODULE_PRIVATE_SIGN_KEY_PASSPHRASE`
- Private signing keys must stay in CI secrets and never in repository history

Public key for runtime verification:
Expand All @@ -79,7 +93,7 @@ Public key for runtime verification:
Scope boundary:

- This change set hardens local and bundled module safety.
- The online multi-registry ecosystem and production marketplace rollout remain tracked in `marketplace-02`.
- For publishing your own modules to a registry, see [Publishing modules](publishing-modules.md).

## Marketplace vs Local Modules

Expand Down
3 changes: 3 additions & 0 deletions docs/guides/module-signing-and-key-rotation.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ openssl pkey -in module-signing-private.pem -pubout -out module-signing-public.p

Preferred (strict, with private key):

- **Key file**: `--key-file <path>` or set `SPECFACT_MODULE_PRIVATE_SIGN_KEY_FILE` (or legacy `SPECFACT_MODULE_SIGNING_PRIVATE_KEY_FILE`).
- **Inline PEM**: Set `SPECFACT_MODULE_PRIVATE_SIGN_KEY` (or legacy `SPECFACT_MODULE_SIGNING_PRIVATE_KEY_PEM`) to the PEM string; no file needed. Useful in CI where the key is in a secret.

```bash
KEY_FILE="${SPECFACT_MODULE_PRIVATE_SIGN_KEY_FILE:-.specfact/sign-keys/module-signing-private.pem}"
python scripts/sign-modules.py --key-file "$KEY_FILE" src/specfact_cli/modules/*/module-package.yaml
Expand Down
Loading