feat: Marketplace integration -- read marketplace.json for plugin discovery + governance

[danielmeppiel]

Problem

APM uses Git URLs directly for plugin installation (apm install owner/repo). Meanwhile, the plugin ecosystem converges on marketplaces -- Copilot CLI and Claude Code both use marketplace.json as a discovery catalog. Today, plugins are discovered in a marketplace, then separately installed and governed via APM. This two-tool workflow makes governance optional and easily skipped.

APM should read existing marketplace.json files for discovery, resolve plugins to Git URLs, then apply its full governance layer (lockfile, SHA pinning, audit trail, dev/prod separation) on top -- collapsing a two-tool workflow into one.

Supporting documentation

• Copilot CLI Plugin Reference -- plugin.json spec
• Copilot CLI Plugin Reference: marketplace.json -- marketplace.json spec
• Copilot CLI: Creating a plugin marketplace -- marketplace authoring guide
• Copilot CLI: Finding and installing plugins -- marketplace consumer workflow
• Claude Code: Plugins overview -- Claude plugin spec
• Claude Code: Discover plugins -- Claude marketplace discovery
• Claude Code: Plugin marketplaces -- Claude marketplace.json format and source types

---

Design Decisions

Informed by a 5-expert analysis panel (UX, npm ecosystem, product strategy, Python architecture, Anthropic spec compatibility).

1. Marketplace = discovery layer, Git = source of truth

APM reads marketplace.json as-is (both .github/plugin/marketplace.json and .claude-plugin/marketplace.json). The marketplace resolves to a Git coordinate. APM never depends on a marketplace being online for installs after initial resolution. This is the Go modules model (where pkg.go.dev is discovery but go get always resolves to VCS), not the npm model (where the registry IS the source).

2. Git ref in apm.yml, marketplace provenance in lockfile

When apm install security-checks@acme-tools runs:

1. Query marketplace -> resolve to owner/plugin-repo#v1.3.0
2. Store owner/plugin-repo#v1.3.0 in apm.yml (self-contained, portable)
3. Store discovered_via: acme-tools + marketplace_plugin_name: security-checks in apm.lock.yaml

If the marketplace disappears, existing installs still work. Future apm deps update --marketplace will use lockfile provenance to re-query.

Why not store NAME@MARKETPLACE in apm.yml? Storing the resolved Git ref makes apm.yml fully self-contained -- it does not require the recipient to have the same marketplace configured, and it survives marketplace deletion or migration.

3. No DependencyReference modifications (critical)

DependencyReference represents a resolved Git coordinate. A marketplace name is an indirect reference that gets resolved INTO a DependencyReference, not stored AS one. Use a pre-parse intercept in install.py before DependencyReference.parse() is called.

4. @ disambiguation: slash-less detection

Rule: If input matches ^[a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+$ (no /, no :, no git@), it is a marketplace ref. Everything else goes to existing parse.

Why this is safe: owner/repo@alias has / -> rejected. git@host:... has : -> rejected. Only word@word matches. These inputs previously raised ValueError ("Use 'user/repo' format"), so this is a backward-compatible grammar extension -- no existing valid specifiers change behavior.

5. Bare name install rejected in v1

apm install security-checks (without @MARKETPLACE) is rejected with a helpful error suggesting @MARKETPLACE syntax and apm search. This prevents dependency confusion attacks and non-deterministic resolution when multiple marketplaces are configured.

6. No default marketplaces in v1

A governance tool should not auto-trust third-party sources. apm marketplace add is frictionless. Future phase: suggest popular marketplaces during apm init.

7. User-scoped marketplace config at ~/.apm/

Registered marketplaces in ~/.apm/marketplaces.json. Marketplaces are a personal discovery preference, not a project dependency. Follows existing ~/.apm/config.json pattern.

8. Cache with TTL + stale-while-revalidate

~/.apm/cache/marketplace/ with 1h TTL. Network failure serves stale cache. marketplace add and marketplace browse force refresh.

9. Reuse AuthResolver unchanged

AuthResolver.try_with_fallback(unauth_first=True) handles public-first with fallback for private marketplace repos. No new auth code needed.

10. Support 4 of 5 Anthropic source types

Support: relative path, github, url, git-subdir. Skip: npm (out of APM's Git-native model).

---

Architecture

New module: src/apm_cli/marketplace/

src/apm_cli/marketplace/
  __init__.py          # Public exports
  models.py            # Frozen dataclasses: MarketplaceSource, MarketplacePlugin, MarketplaceManifest
  client.py            # MarketplaceClient: fetch, parse, cache marketplace.json via GitHub API
  registry.py          # Manage registered marketplaces in ~/.apm/marketplaces.json
  resolver.py          # parse_marketplace_ref() + resolve NAME -> canonical owner/repo string
  errors.py            # MarketplaceError, MarketplaceNotFoundError, PluginNotFoundError
  commands.py          # Click command group: marketplace add/list/browse/update/remove + search

Resolution flow

apm install security-checks@acme-tools
  |
  v
Pre-parse intercept in install.py _validate_and_add_packages_to_apm_yml() L130
  -> parse_marketplace_ref("security-checks@acme-tools") returns ("security-checks", "acme-tools")
  |
  v
resolve_marketplace_plugin("security-checks", "acme-tools")
  -> registry.get_marketplace_by_name("acme-tools") -> MarketplaceSource
  -> client.fetch_or_cache(source) -> MarketplaceManifest
  -> Find plugin: { name: "security-checks", source: { source: "github", repo: "owner/repo", ref: "v1.3.0" } }
  |
  v
Resolve source to canonical string: "owner/repo#v1.3.0"
  |
  v
Replace package variable -> existing install flow continues unchanged
  -> DependencyReference.parse("owner/repo#v1.3.0") -> validate -> apm.yml -> lockfile

CLI commands

apm marketplace add OWNER/REPO          # Register a marketplace
apm marketplace list                    # Rich table of registered marketplaces
apm marketplace browse NAME             # Rich table of plugins in a marketplace
apm marketplace update [NAME]           # Refresh cache (one or all)
apm marketplace remove NAME             # Unregister with confirmation

apm search QUERY                        # Search across all registered marketplaces
apm install NAME@MARKETPLACE            # Resolve via marketplace, full governance

Storage layout

~/.apm/
  config.json                           # Existing
  marketplaces.json                     # NEW: registered marketplace sources
  cache/
    marketplace/
      acme-tools.json                   # Cached marketplace.json content
      acme-tools.meta.json              # { fetched_at, ttl_seconds }

Data models

@dataclass(frozen=True)
class MarketplaceSource:
    name: str               # Display name (e.g., "acme-tools")
    owner: str              # GitHub owner
    repo: str               # GitHub repo
    host: str = "github.com"
    branch: str = "main"
    path: str = "marketplace.json"  # Auto-detected on add

@dataclass(frozen=True)
class MarketplacePlugin:
    name: str               # Plugin name (unique within marketplace)
    source: Any             # String (relative) or dict (github/url/git-subdir)
    description: str = ""
    version: str = ""
    tags: tuple = ()
    source_marketplace: str = ""

@dataclass(frozen=True)
class MarketplaceManifest:
    name: str
    plugins: tuple = ()     # Tuple of MarketplacePlugin
    owner_name: str = ""
    description: str = ""

Lockfile provenance (new optional fields on LockedDependency)

- repo_url: owner/plugin-repo
  resolved_commit: abc123def456
  resolved_ref: v1.3.0
  content_hash: sha256:2800e9f...
  discovered_via: acme-tools                # NEW
  marketplace_plugin_name: security-checks  # NEW

@ disambiguation regex

_MARKETPLACE_RE = re.compile(r'^([a-zA-Z0-9._-]+)@([a-zA-Z0-9._-]+)$')

def parse_marketplace_ref(specifier):
    if '/' in specifier or ':' in specifier:
        return None
    match = _MARKETPLACE_RE.match(specifier.strip())
    return (match.group(1), match.group(2)) if match else None

Modified existing files

• src/apm_cli/commands/install.py L130: pre-parse intercept (~10 lines)
• src/apm_cli/deps/lockfile.py: 2 optional fields on LockedDependency + serialization
• src/apm_cli/cli.py L71: register marketplace command group (2 lines)

---

Implementation Plan

Wave 1 -- Foundation (parallelizable, no dependencies)

marketplace-models

Create src/apm_cli/marketplace/models.py and errors.py:

• Frozen dataclasses: MarketplaceSource, MarketplacePlugin, MarketplaceManifest
• parse_marketplace_json(data, source_name) parser for both Copilot CLI and Claude Code marketplace.json formats
• Error classes: MarketplaceError, MarketplaceNotFoundError, PluginNotFoundError, MarketplaceFetchError
• Also create __init__.py with exports

lockfile-provenance

Edit src/apm_cli/deps/lockfile.py:

• Add discovered_via and marketplace_plugin_name optional fields to LockedDependency
• Update to_dict() and from_dict() (backward compatible -- None by default)

Wave 2 -- Registry

marketplace-registry

Create src/apm_cli/marketplace/registry.py:

• CRUD for ~/.apm/marketplaces.json: get_registered_marketplaces(), add_marketplace(), remove_marketplace(), get_marketplace_by_name()
• Process-lifetime cache, atomic writes, uses config.ensure_config_exists()
• Depends on: marketplace-models

Wave 3 -- Client

marketplace-client

Create src/apm_cli/marketplace/client.py:

• MarketplaceClient with AuthResolver integration
• fetch_marketplace() via GitHub Contents API with unauth_first=True
• fetch_or_cache() with 1h TTL, stale-while-revalidate on network errors
• search_plugins() across all registered marketplaces
• Auto-detect marketplace.json location (.claude-plugin/ vs .github/plugin/ vs repo root)
• Cache at ~/.apm/cache/marketplace/
• Depends on: marketplace-models, marketplace-registry

Wave 4 -- Resolver

marketplace-resolver

Create src/apm_cli/marketplace/resolver.py:

• parse_marketplace_ref() regex for NAME@MARKETPLACE detection
• resolve_marketplace_plugin() -> canonical owner/repo#ref string
• Handles 4 source types: relative path, github, url, git-subdir
• Rejects npm source with clear error message
• Actionable error messages for missing marketplace/plugin
• Depends on: marketplace-models, marketplace-client, marketplace-registry

Wave 5 -- CLI + Install Hook (parallelizable)

marketplace-cli

Create src/apm_cli/commands/marketplace.py:

• Click group with: add, list, browse, update, remove subcommands
• Top-level search command
• Rich tables with colorama fallbacks, STATUS_SYMBOLS
• Follow mcp.py pattern exactly
• Depends on: marketplace-client, marketplace-resolver, marketplace-registry

install-marketplace-hook

Edit src/apm_cli/commands/install.py _validate_and_add_packages_to_apm_yml() at L130:

• Pre-parse intercept: parse_marketplace_ref() before the "/" not in package check
• Resolve to canonical string, replace package variable
• Pass marketplace provenance to lockfile writer
• Lazy imports to avoid overhead when marketplace is unused
• Depends on: marketplace-resolver, lockfile-provenance

Wave 6 -- Wiring

register-and-wire

• Register marketplace command group in cli.py (2 lines)
• Finalize __init__.py exports
• Depends on: marketplace-cli

Wave 7 -- Tests + Docs (parallelizable)

tests-marketplace

~90 unit tests across 7 test files:

• test_marketplace_models.py (~15): dataclass behavior, JSON parsing, source types
• test_marketplace_resolver.py (~20): regex positive/negative cases, resolution for all 4 source types
• test_marketplace_client.py (~15): HTTP mock, caching, TTL, auth, auto-detection
• test_marketplace_registry.py (~10): CRUD with tmp_path isolation
• test_marketplace_commands.py (~15): CliRunner for each command, empty states, errors
• test_marketplace_install_integration.py (~10): install flow with mocked marketplace
• test_lockfile_provenance.py (~5): serialization round-trip, backward compat
• Depends on: all above

docs-marketplace

• New: docs/src/content/docs/guides/marketplaces.md (user guide)
• Update: CLI reference with marketplace commands + NAME@MARKETPLACE syntax
• Update: plugins guide with marketplace integration section
• Depends on: all above

---

Todo list

• marketplace-models -- Frozen dataclasses + JSON parser + error hierarchy (Wave 1)
• lockfile-provenance -- 2 optional fields on LockedDependency (Wave 1)
• marketplace-registry -- ~/.apm/marketplaces.json CRUD (Wave 2)
• marketplace-client -- GitHub API fetch, 1h TTL cache, AuthResolver reuse (Wave 3)
• marketplace-resolver -- parse_marketplace_ref() regex + source type resolution (Wave 4)
• marketplace-cli -- Click group: add/list/browse/update/remove + search (Wave 5)
• install-marketplace-hook -- ~10 lines in install.py L130 (Wave 5)
• register-and-wire -- 2 lines in cli.py (Wave 6)
• tests-marketplace -- ~90 tests across 7 files (Wave 7)
• docs-marketplace -- New guide + CLI reference updates (Wave 7)

Dependencies between todos

marketplace-models            -- no deps (Wave 1)
lockfile-provenance           -- no deps (Wave 1)
marketplace-registry          -- depends on: marketplace-models (Wave 2)
marketplace-client            -- depends on: marketplace-models, marketplace-registry (Wave 3)
marketplace-resolver          -- depends on: marketplace-models, marketplace-client, marketplace-registry (Wave 4)
marketplace-cli               -- depends on: marketplace-client, marketplace-resolver, marketplace-registry (Wave 5)
install-marketplace-hook      -- depends on: marketplace-resolver, lockfile-provenance (Wave 5)
register-and-wire             -- depends on: marketplace-cli (Wave 6)
tests-marketplace             -- depends on: all above (Wave 7)
docs-marketplace              -- depends on: all above (Wave 7)

---

Phase 2 (future, not in this plan)

• Claude settings.json integration (enabledPlugins + extraKnownMarketplaces)
• apm publish for self-hosted marketplaces
• NPM source type support
• Bare name install with disambiguation
• apm deps update --marketplace using lockfile provenance
• apm deps outdated with marketplace update info
• Default marketplace suggestions in apm init
• CI seed directory generation (CLAUDE_CODE_PLUGIN_SEED_DIR)