Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- **`apm marketplace add` accepts full HTTPS URLs and nested HOST/group/sub/.../REPO shorthands.** You can now paste a repository URL straight from the browser (e.g., `apm marketplace add https://github.com/acme/plugin-marketplace`) and register marketplaces hosted under nested sub-paths on GitHub Enterprise (`ghes.corp.example.com/org/team/repo`). Path-traversal sequences in the parsed segments are rejected via `validate_path_segments`. Non-GitHub hosts (GitLab, Bitbucket, etc.) are explicitly rejected at registration time with an actionable error -- this avoids forwarding GitHub credentials to unintended hosts and the silent fetch-time 404 that previously resulted; native non-GitHub support is tracked separately. (#1034, closes #1027)
- Regression tests for `apm compile` placement of narrow `applyTo` patterns: instructions whose matches all live deep inside one subtree are now pinned to the deepest covering directory instead of being hoisted to the project root, across both selective and single-point placement strategies. Also covers the file-walk cache that skips repeated filesystem scans for the same glob. (#871)
- **`apm pack` marketplace builder hardening.** Local source paths are now emitted relative to `metadata.pluginRoot` (fixes double-prefix bug). New pass-through fields: `author`, `license`, `repository`, `keywords` (alias for `tags`). Curator-wins override semantics for `description`/`version` on remote entries. Security guards reject path traversal and absolute paths post-subtraction. (#1061)
- **Plugin manifest schema-conformance tests.** `tests/unit/test_plugin_exporter_schema.py` validates every shape of `plugin.json` produced by `apm pack` (synthesized, authored, and authored-with-stale-keys) against the vendored official schema. Companion marketplace conformance lives in `tests/unit/marketplace/test_schema_conformance.py`. (#1061)
Expand Down
14 changes: 13 additions & 1 deletion docs/src/content/docs/reference/cli-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -1162,29 +1162,41 @@ Register a GitHub repository as a plugin marketplace.
```bash
apm marketplace add OWNER/REPO [OPTIONS]
apm marketplace add HOST/OWNER/REPO [OPTIONS]
apm marketplace add HOST/group/sub/.../REPO [OPTIONS]
apm marketplace add https://HOST/owner/.../repo[.git] [OPTIONS]
```

**Arguments:**
- `OWNER/REPO` - GitHub repository containing `marketplace.json`
- `HOST/OWNER/REPO` - Repository on a non-github.com host (e.g., GitHub Enterprise)
- `HOST/group/sub/.../REPO` - Repository nested under sub-paths (e.g., GHES org/team/repo)
- `https://HOST/owner/.../repo[.git]` - Full HTTPS URL pasted from the browser. The `.git` suffix is stripped.

**Options:**
- `-n, --name TEXT` - Custom display name for the marketplace
- `-b, --branch TEXT` - Branch to track (default: main)
- `--host TEXT` - Git host FQDN (default: github.com or `GITHUB_HOST` env var)
- `-v, --verbose` - Show detailed output

> **Supported hosts.** `apm marketplace add` currently fetches `marketplace.json` via the GitHub Contents API, so only `github.com`, GitHub Enterprise Cloud (`*.ghe.com`), and the host configured via `GITHUB_HOST` are accepted. GitLab, Bitbucket, and other generic Git hosts are rejected at registration time with an actionable error -- this prevents silent fetch failures and avoids forwarding GitHub credentials to unintended hosts. Native non-GitHub support is tracked separately.

**Examples:**
```bash
# Register a marketplace
apm marketplace add acme/plugin-marketplace

# Register from a full HTTPS URL pasted from the browser
apm marketplace add https://github.com/acme/plugin-marketplace

# Register with a custom name and branch
apm marketplace add acme/plugin-marketplace --name acme-plugins --branch release

# Register from a GitHub Enterprise host
# Register from a GitHub Enterprise host (Cloud or Server)
apm marketplace add acme/plugin-marketplace --host ghes.corp.example.com
apm marketplace add ghes.corp.example.com/acme/plugin-marketplace

# Register a repo nested under sub-paths on a GHES instance
apm marketplace add ghes.corp.example.com/org/team/plugin-marketplace
```

#### `apm marketplace list` - List registered marketplaces
Expand Down
2 changes: 1 addition & 1 deletion packages/apm-guide/.apm/skills/apm-usage/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@

| Command | Purpose | Key flags |
|---------|---------|-----------|
| `apm marketplace add OWNER/REPO` | Register a marketplace | `-n NAME`, `-b BRANCH`, `--host HOST` |
| `apm marketplace add OWNER/REPO` | Register a marketplace (also accepts `HOST/OWNER/REPO`, nested `HOST/group/sub/.../REPO`, or full HTTPS URL) | `-n NAME`, `-b BRANCH`, `--host HOST` |
| `apm marketplace list` | List registered marketplaces | -- |
| `apm marketplace browse NAME` | Browse marketplace plugins | -- |
| `apm marketplace update [NAME]` | Update marketplace index | -- |
Expand Down
191 changes: 163 additions & 28 deletions src/apm_cli/commands/marketplace/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

import builtins
import json
import os # noqa: F401
import re
import sys
import traceback
Expand Down Expand Up @@ -238,67 +237,201 @@ def _check_gitignore_for_marketplace_json(logger):
return


def _parse_marketplace_repo(repo: str, host_flag: str | None) -> tuple[str, str, str | None]:
"""Parse a marketplace repo argument into ``(owner, repo_name, embedded_host)``.

Accepted forms:
* ``OWNER/REPO`` (2 segments)
* ``HOST/OWNER/REPO`` (3 segments, first is FQDN)
* ``HOST/group/sub/.../REPO`` (N>=4 segments, first is FQDN -- GHES nested paths)
* ``OWNER/group/sub/.../REPO`` (N>=3 segments, first is NOT a FQDN)
* ``https://HOST/owner/.../repo[.git]`` (full HTTPS URL)
* ``http://HOST/owner/.../repo[.git]`` (full HTTP URL -- rejected with explicit error)

Returns ``(owner, repo_name, embedded_host)`` where ``embedded_host`` is the
host carried by the input itself (``HOST/...`` shorthand or HTTPS URL host)
or ``None`` for bare ``OWNER/REPO`` shorthand.

Raises ``ValueError`` on malformed input. The caller is responsible for
enforcing the trusted-host allowlist on the returned ``embedded_host``.

The returned segments are validated through ``validate_path_segments`` to
reject path-traversal sequences (``..``, ``.``, ``~``).
"""
from urllib.parse import urlparse

from ...utils.github_host import is_valid_fqdn

raw = (repo or "").strip()
if not raw:
raise ValueError("Empty repository argument")

# Reject control characters and percent-encoded traversal. urlparse normalizes
# the path but does not unescape; we unescape eagerly so the security guards
# below see the real bytes the user typed.
import urllib.parse as _up

if any(ord(c) < 32 for c in raw):
raise ValueError("Repository argument contains invalid control characters")

embedded_host: str | None = None
lowered = raw.lower()

if lowered.startswith("http://"):
# Reject HTTP at parse time. APM does not ship an --allow-insecure
# escape hatch for marketplace add: a MITM adversary on an HTTP fetch
# of marketplace.json could inject attacker-controlled plugin source
# URLs, with no audit trail.
raise ValueError(
f"Insecure HTTP URL rejected: '{raw}'. Use HTTPS for marketplace registration."
)

if lowered.startswith("https://"):
parsed = urlparse(raw)
embedded_host = (parsed.hostname or "").strip().lower()
if not embedded_host:
raise ValueError(f"HTTPS URL is missing a host: '{raw}'")
# urlparse leaves the path percent-encoded; decode for segment splitting
# so traversal markers like '%2E%2E' are caught by validate_path_segments.
path = _up.unquote(parsed.path or "")
if path.endswith(".git"):
path = path[:-4]
segments = [seg for seg in path.split("/") if seg]
else:
# Mirror the HTTPS branch: decode percent-encoded sequences before splitting
# so '%2E%2E' becomes '..' and is caught by validate_path_segments below.
raw_decoded = _up.unquote(raw)
segments = [seg for seg in raw_decoded.split("/") if seg]

if len(segments) < 2:
raise ValueError(
f"Invalid format: '{raw}'. "
f"Expected 'OWNER/REPO', 'HOST/OWNER/REPO', or a full HTTPS URL."
)

if embedded_host is None and is_valid_fqdn(segments[0]):
# Shorthand carries an explicit host (e.g. 'gitlab.com/org/repo').
if len(segments) < 3:
raise ValueError(
f"Invalid format: '{raw}'. When the first segment is a host FQDN, "
f"at least 'HOST/OWNER/REPO' is required."
)
embedded_host = segments[0].lower()
segments = segments[1:]

repo_name = segments[-1]
owner_segments = segments[:-1]
if not owner_segments or not repo_name:
raise ValueError(f"Invalid format: '{raw}'. Expected 'OWNER/REPO'.")

# Reject conflicting --host BEFORE security validation so the user gets the
# clearest possible error.
if embedded_host and host_flag and host_flag.strip().lower() != embedded_host:
# shlex.quote prevents shell-metacharacter injection in the
# copy-paste suggestion (round-4 supply-chain nit).
import shlex as _shlex

raise ValueError(
f"Conflicting host: --host '{host_flag}' does not match "
f"'{embedded_host}' in '{raw}'.\n"
f"To fix: drop --host and run: apm marketplace add {_shlex.quote(raw)}"
)

# validate_path_segments rejects '.', '..', '~' and cross-platform backslash
# variants in any single segment. Validate the joined owner path and the
# repo name independently so the error messages are precise.
owner_path = "/".join(owner_segments)
validate_path_segments(owner_path, context="marketplace owner path", reject_empty=True)
validate_path_segments(repo_name, context="marketplace repo name", reject_empty=True)

return owner_path, repo_name, embedded_host


# Host-trust classification is owned by AuthResolver.classify_host (see
# core/auth.py). The marketplace command layer routes through it so that the
# credential-leakage guard at registration time uses the same single source of
# truth as the fetch-time guard in marketplace/client.py. Adding a second
# implementation here would create silent drift on a security-critical path.
_TRUSTED_MARKETPLACE_HOST_KINDS = ("github", "ghe_cloud", "ghes")


@marketplace.command(help="Register a marketplace")
@click.argument("repo", required=True)
@click.option("--name", "-n", default=None, help="Display name (defaults to repo name)")
@click.option("--branch", "-b", default="main", show_default=True, help="Branch to use")
@click.option("--host", default=None, help="Git host FQDN (default: github.com)")
@click.option("--verbose", "-v", is_flag=True, help="Show detailed output")
def add(repo, name, branch, host, verbose):
"""Register a marketplace from OWNER/REPO or HOST/OWNER/REPO."""
"""Register a marketplace from OWNER/REPO, HOST/OWNER/.../REPO, or an HTTPS URL."""
logger = CommandLogger("marketplace-add", verbose=verbose)
try:
from ...marketplace.client import _auto_detect_path, fetch_marketplace
from ...marketplace.models import MarketplaceSource
from ...marketplace.registry import add_marketplace
from ...utils.github_host import default_host, is_valid_fqdn

# Parse OWNER/REPO or HOST/OWNER/REPO
if "/" not in repo:
try:
owner, repo_name, embedded_host = _parse_marketplace_repo(repo, host)
except PathTraversalError:
logger.error(
f"Invalid format: '{repo}'. Use 'OWNER/REPO' (e.g., 'acme-org/plugin-marketplace')"
f"Invalid repo path '{repo}': contains a path-traversal sequence. "
f"Remove '..', '.', or '~' from each path segment."
)
sys.exit(1)

from ...utils.github_host import default_host, is_valid_fqdn

parts = repo.split("/")
if len(parts) == 3 and parts[0] and parts[1] and parts[2]:
if not is_valid_fqdn(parts[0]):
logger.error(
f"Invalid host: '{parts[0]}'. Use 'OWNER/REPO' or 'HOST/OWNER/REPO' format."
)
sys.exit(1)
if host and host != parts[0]:
logger.error(f"Conflicting host: --host '{host}' vs '{parts[0]}' in argument.")
sys.exit(1)
host = parts[0]
owner, repo_name = parts[1], parts[2]
elif len(parts) == 2 and parts[0] and parts[1]:
owner, repo_name = parts[0], parts[1]
else:
logger.error(f"Invalid format: '{repo}'. Expected 'OWNER/REPO'")
except ValueError as exc:
logger.error(str(exc))
sys.exit(1)

# Resolve the effective host: explicit --host wins, then host embedded
# in the argument (HOST/... shorthand or HTTPS URL), then GITHUB_HOST.
if host is not None:
normalized_host = host.strip().lower()
if not is_valid_fqdn(normalized_host):
logger.error(
f"Invalid host: '{host}'. Expected a valid host FQDN "
f"(for example, 'github.com')."
f"(for example, 'github.com').",
symbol="error",
)
sys.exit(1)
resolved_host = normalized_host
elif embedded_host is not None:
resolved_host = embedded_host
else:
resolved_host = default_host()

# Trusted-host gate. Routes through AuthResolver.classify_host so the
# registration-time guard and the fetch-time guard in client.py share a
# single classification implementation.
from ...core.auth import AuthResolver

if AuthResolver.classify_host(resolved_host).kind not in _TRUSTED_MARKETPLACE_HOST_KINDS:
# Build a one-copy-paste recovery: tell the GHES user the exact
# export and the exact re-run command, with the resolved host
# and original repo string interpolated and shell-quoted.
import shlex as _shlex

quoted_repo = _shlex.quote(repo)
quoted_host = _shlex.quote(resolved_host)
logger.error(
f"Host '{resolved_host}' is not supported.\n"
f"Supported hosts: github.com, *.ghe.com, "
f"or the host set via GITHUB_HOST.\n"
f"To use this host:\n"
f" export GITHUB_HOST={quoted_host}\n"
f"Then re-run:\n"
f" apm marketplace add {quoted_repo}"
)
sys.exit(1)

# Hard-fail if the user-supplied --name flag is malformed; the
# manifest's name is validated softly below (publisher mistakes
# shouldn't break a successful add).
if name is not None and not _is_valid_alias(name):
logger.error(
f"Invalid marketplace name: '{name}'. "
f"Names must only contain letters, digits, '.', '_', and '-' "
f"(required for 'apm install plugin@marketplace' syntax)."
f"(required for 'apm install plugin@marketplace' syntax).",
symbol="error",
)
sys.exit(1)

Expand All @@ -317,7 +450,8 @@ def add(repo, name, branch, host, verbose):
logger.error(
f"No marketplace.json found in '{owner}/{repo_name}'. "
f"Checked: marketplace.json, .github/plugin/marketplace.json, "
f".claude-plugin/marketplace.json"
f".claude-plugin/marketplace.json",
symbol="error",
)
sys.exit(1)

Expand Down Expand Up @@ -350,7 +484,8 @@ def add(repo, name, branch, host, verbose):
logger.warning(
f"Manifest declares name '{manifest_name}' which is not a "
f"valid alias (must match [a-zA-Z0-9._-]+). "
f"Falling back to repo name."
f"Falling back to repo name.",
symbol="warning",
)
alias_source = f"repo name (manifest.name '{manifest_name}' invalid)"
else:
Expand Down
24 changes: 23 additions & 1 deletion src/apm_cli/marketplace/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,24 @@ def _fetch_file(
)
return None

# Defense-in-depth host-kind guard. Marketplace registration already
# rejects non-trusted hosts, but if a generic / non-GitHub host slips
# through (legacy registry entries, manual registry edits, future
# callers) we MUST NOT issue a GitHub Contents API request: doing so
# would attach Authorization: token <github_pat> headers to a request
# aimed at an unrelated host, leaking GitHub credentials. Fail closed.
from ..core.auth import AuthResolver as _AuthResolver

host_info = _AuthResolver.classify_host(source.host)
if host_info.kind not in ("github", "ghe_cloud", "ghes"):
raise MarketplaceFetchError(
source.name,
f"Host {source.host!r} is not a supported marketplace source. "
f"Only GitHub, GitHub Enterprise Cloud (*.ghe.com), and GHES "
f"(GITHUB_HOST) are supported. Refusing to fetch to avoid "
f"forwarding GitHub credentials to a non-GitHub host.",
)

# Fallback: GitHub Contents API
url = _github_contents_url(source, file_path)

Expand All @@ -234,7 +252,11 @@ def _do_fetch(token, _git_env):
"Accept": "application/vnd.github.v3.raw",
"User-Agent": "apm-cli",
}
if token:
# Only attach GitHub-namespaced credentials when the resolver-derived
# host kind is a GitHub variant. The outer guard already enforces
# this, but keep the conditional explicit so the credential-attach
# site is locally auditable.
if token and host_info.kind in ("github", "ghe_cloud", "ghes"):
headers["Authorization"] = f"token {token}"
resp = requests.get(url, headers=headers, timeout=30)
if resp.status_code == 404:
Expand Down
10 changes: 8 additions & 2 deletions src/apm_cli/marketplace/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,17 @@ class MarketplaceError(Exception):
class MarketplaceNotFoundError(MarketplaceError):
"""Raised when a registered marketplace cannot be found."""

def __init__(self, name: str):
def __init__(self, name: str, host: str = "github.com"):
self.name = name
self.host = host
# Interpolate the active host so GHES users get a copy-paste-ready
# URL that works for them. Callers should pass the current host
# (e.g. default_host()); fall back to github.com to preserve the
# public-cloud default.
super().__init__(
f"Marketplace '{name}' is not registered. "
f"Run 'apm marketplace add OWNER/REPO' to register it, "
f"Run 'apm marketplace add https://{host}/OWNER/REPO' "
f"or 'apm marketplace add OWNER/REPO' to register it, "
f"or 'apm marketplace list' to see registered marketplaces."
)

Expand Down
8 changes: 6 additions & 2 deletions src/apm_cli/marketplace/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,9 @@ def get_marketplace_by_name(name: str) -> MarketplaceSource:
for src in _load():
if src.name.lower() == lower:
return src
raise MarketplaceNotFoundError(name)
from ..utils.github_host import default_host

raise MarketplaceNotFoundError(name, host=default_host())


def add_marketplace(source: MarketplaceSource) -> None:
Expand All @@ -117,7 +119,9 @@ def remove_marketplace(name: str) -> None:
before = _load()
after = [s for s in before if s.name.lower() != name.lower()]
if len(after) == len(before):
raise MarketplaceNotFoundError(name)
from ..utils.github_host import default_host

raise MarketplaceNotFoundError(name, host=default_host())
_save(after)
logger.debug("Removed marketplace '%s'", name)

Expand Down
Loading