feat(marketplace): add experimental authoring CLI commands (#722)#790
feat(marketplace): add experimental authoring CLI commands (#722)#790sergio-sisternes-epam merged 23 commits intomainfrom
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Wrap the state-file path in rich.text.Text(..., no_wrap=True) so it renders on a single line and does not break the publish-state.json substring that CI tests rely on when the terminal width is 80 cols. (microsoft#790) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sergio-sisternes-epam
force-pushed
the
feat/722-marketplace-maintainer-ux
branch
3 times, most recently
from
72d1b7b to
56c20fe
Compare
APM Expert Review Panel -- PR #790Reviewers: Python Architect, CLI Logging Expert, DevX UX Expert, Supply Chain Security Expert, APM CEO, OSS Growth Hacker Panel Verdict: Approve with conditionsThis is APM's most significant feature since Two conditions must be met before merge. Seven high-priority findings should be tracked for fast-follow. Conditions for Merge
Findings by SeverityCritical (4 findings)
High / Important (14 findings)
Notes (11 findings)
What's Done WellThe panel unanimously highlighted these strengths:
Strategic Assessment (CEO)Positioning: This is APM's moat-defining feature. No competitor offers self-hosted marketplace authoring with semver resolution and cross-repo PR-driven publish. This is the Naming: Dependency: Release: This merits a dedicated Priority Fix Order
Panel composition and routingSix specialist agents reviewed independently, with findings consolidated by the orchestrator:
Per panel protocol, specialists raised findings independently. The CEO arbitrated strategic calls (naming ratification, dependency approval, release framing). The Growth Hacker annotated discoverability gaps and escalated the sidebar registration as critical. |
APM Expert Review Panel -- Round 2 (Fix Commit Validation)Scope: 3 fix commits ( Panel Verdict: APPROVE (6/6 specialists)
Merge Conditions -- Verified
Security Fixes -- Validated
Architecture Fixes -- Validated
Logging + UX Fixes -- Validated
New Findings from Round 2 (Non-Blocking)
Post-Merge Recommendations
Test Coverage4825 tests passing (+31 new tests from fix commits), 0 failures. Every security fix has corresponding test coverage:
|
sergio-sisternes-epam
added
the
panel-review
There was a problem hiding this comment.
Pull request overview
Introduces first-pass maintainer-side apm marketplace authoring tooling, centered around marketplace.yml as the source-of-truth and deterministic compilation to Anthropic-compliant marketplace.json, with supporting CLI commands, ref resolution, publishing orchestration, and expanded test/docs coverage.
Changes:
- Add marketplace authoring/editing primitives (round-trip YAML editor, semver + tag pattern helpers, git stderr translation, ref resolver cache).
- Extend CLI with marketplace authoring flows (
init,build,check,outdated,doctor,publish) and amarketplace plugin {add,set,remove}subgroup. - Add comprehensive unit/integration/live-e2e test scaffolding plus docs and changelog updates; add
ruamel.yamldependency.
Reviewed changes
Copilot reviewed 55 out of 57 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Adds ruamel.yaml dependency for round-trip YAML editing. |
uv.lock |
Locks ruamel-yaml artifact to support new YAML editor functionality. |
src/apm_cli/marketplace/_io.py |
Adds atomic write helper used by marketplace editing/build flows. |
src/apm_cli/marketplace/_git_utils.py |
Adds token redaction helper for git/gh stderr strings. |
src/apm_cli/marketplace/git_stderr.py |
Adds stderr classification into actionable, ASCII-only summaries/hints. |
src/apm_cli/marketplace/ref_resolver.py |
Adds cached git ls-remote ref discovery and SHA resolution. |
src/apm_cli/marketplace/semver.py |
Adds semver parsing and range evaluation used for tag selection. |
src/apm_cli/marketplace/tag_pattern.py |
Adds tag rendering + regex compilation for extracting versions from tags. |
src/apm_cli/marketplace/yml_editor.py |
Adds round-trip YAML editor for manipulating marketplace.yml entries. |
src/apm_cli/marketplace/pr_integration.py |
Adds gh-based PR open/update integration for publish workflow. |
src/apm_cli/marketplace/init_template.py |
Adds scaffold template used by apm marketplace init. |
src/apm_cli/marketplace/errors.py |
Adds marketplace build/YML/ref-resolution error types. |
src/apm_cli/marketplace/__init__.py |
Re-exports new marketplace authoring/public API symbols. |
src/apm_cli/commands/_helpers.py |
Adds shared TTY detection helper used by marketplace publish confirmation. |
src/apm_cli/commands/marketplace_plugin.py |
Adds CLI subgroup to add/set/remove marketplace.yml entries. |
tests/unit/marketplace/* |
Adds unit tests for new marketplace primitives and helpers. |
tests/unit/commands/test_marketplace_* |
Adds unit tests for new marketplace CLI commands. |
tests/integration/marketplace/* |
Adds integration + env-var-gated live e2e tests and shared fixtures/docs. |
tests/fixtures/marketplace/golden.json |
Adds golden fixture for Anthropic schema compatibility checks. |
docs/src/content/docs/reference/cli-commands.md |
Documents new marketplace authoring CLI commands. |
docs/src/content/docs/guides/marketplaces.md |
Adds authoring-related guidance (monorepo/subdir + ref auto-resolution). |
docs/astro.config.mjs |
Adds Marketplace Authoring guide to docs nav. |
packages/apm-guide/.apm/skills/apm-usage/commands.md |
Updates skills command reference for marketplace authoring tooling. |
packages/apm-guide/.apm/skills/apm-usage/package-authoring.md |
Adds marketplace authoring section for skill consumers. |
CHANGELOG.md |
Adds Unreleased entries for marketplace authoring tooling. |
sergio-sisternes-epam
force-pushed
the
feat/722-marketplace-maintainer-ux
branch
from
7e05174 to
8ac5007
Compare
sergio-sisternes-epam
changed the title
danielmeppiel
added
panel-review
APM Review Panel VerdictDisposition: REQUEST_CHANGES (one P0 merge-conflict blocker; one P1 auth pattern; two minor clean-ups) Per-persona findingsPython Architect: This is a major architectural addition: a new classDiagram
direction LR
class MarketplaceBuilder {
<<Builder>>
+build() BuildReport
-_resolve_entry(entry) ResolvedPackage
-_prefetch_metadata(resolved) dict
-_resolve_github_token() str
}
class MarketplacePublisher {
<<Orchestrator>>
+plan(targets) PublishPlan
+execute(plan) list
-_process_single_target(target, plan) TargetResult
}
class RefResolver {
<<Cache>>
+list_remote_refs(owner_repo) list
+resolve_ref_sha(owner_repo, ref) str
-_cache RefCache
-_remote_locks dict
}
class MarketplaceYml {
<<ValueObject>>
+name str
+version str
+packages list
}
class ResolvedPackage {
<<ValueObject, frozen>>
+name str
+ref str
+sha str
}
class BuildReport {
<<ValueObject, frozen>>
+resolved tuple
+errors tuple
+unchanged_count int
}
class PublishPlan {
<<ValueObject, frozen>>
+branch_name str
+commit_message str
+new_ref str
}
class PublishState {
<<TransactionalState>>
+begin_run(plan)
+record_result(result)
+finalise(dt)
}
class PrIntegrator {
<<Adapter>>
+open_or_update(plan, target) PrResult
}
MarketplaceBuilder *-- RefResolver : delegates git queries
MarketplaceBuilder ..> MarketplaceYml : reads
MarketplaceBuilder ..> ResolvedPackage : produces
MarketplaceBuilder ..> BuildReport : returns
MarketplacePublisher *-- PublishState : writes atomically
MarketplacePublisher ..> PublishPlan : executes
MarketplacePublisher ..> TargetResult : produces
PrIntegrator ..> PublishPlan : reads
note for RefResolver "Per-remote threading.Lock\nfor parallel build safety"
note for PublishState "Atomic write: tmp+fsync+os.replace"
class MarketplaceBuilder:::touched
class MarketplacePublisher:::touched
class RefResolver:::touched
class PublishState:::touched
class PrIntegrator:::touched
classDef touched fill:#fff3b0,stroke:#d47600
flowchart TD
A([apm marketplace build]) --> B[_load_yml_or_exit\nmarketplace.py]
B --> C{marketplace.yml valid?}
C -- No --> D([sys.exit 2])
C -- Yes --> E[MarketplaceBuilder.build\nbuilder.py]
E --> F[NET: RefResolver.list_remote_refs\nper-remote lock + 5-min TTL cache]
F --> G[EXEC: git ls-remote https://github.com/owner/repo.git\nGIT_TERMINAL_PROMPT=0]
G --> H{returncode 0?}
H -- No --> I[GitLsRemoteError\ntranslate_git_stderr]
H -- Yes --> J[resolve version range / explicit ref]
J --> K[NET: _fetch_remote_metadata\nraw.githubusercontent.com with AuthResolver token]
K --> L[FS: atomic_write marketplace.json\ntmp + fsync + os.replace]
L --> M[BuildReport rendered\n_render_build_report marketplace.py]
M --> N([sys.exit 0 or 1])
A2([apm marketplace publish]) --> B2[_load_targets_file\nvalidate_path_segments + _SAFE_REPO_RE]
B2 --> C2[MarketplacePublisher.plan\nvalidate repo/branch/path]
C2 --> D2[MarketplacePublisher.execute\nThreadPoolExecutor parallel=4]
D2 --> E2[NET/FS: git clone --depth=1 https://github.com/repo.git\nGIT_ASKPASS=echo]
E2 --> F2[FS: ensure_path_within apm_yml_path clone_dir]
F2 --> G2[FS: yml_editor update version ref]
G2 --> H2[EXEC: git commit + push branch]
H2 --> I2[PrIntegrator.open_or_update\ngh pr create/edit]
I2 --> J2[PublishState.finalise\natomic write .apm/publish-state.json]
J2 --> K2([sys.exit 0 or 1])
Design patterns
Two minor concerns: (1) CLI Logging Expert: All seven new commands wire One pattern deviation: DevX UX Expert: The One concern: Supply Chain Security Expert: The implementation defends well against the relevant threat categories.
One defensive gap (low severity): Auth Expert: ACTIVATED -- Finding 1 (P1): # builder.py _prefetch_metadata -- safer pattern
github_token = self._resolve_github_token()
...
pool.submit(self._fetch_remote_metadata, pkg, github_token)...and update Finding 2 (P2): The publisher clones consumer repos via plain Finding 3 (P2): OSS Growth Hacker: This is the "homebrew tap" moment for APM -- the feature that enables the ecosystem to self-organize around custom marketplaces rather than depending on a central registry. The The experimental gate is the right call here: it creates a natural cohort of power users who can provide structured feedback before GA. The CHANGELOG entry correctly marks all commands as Side-channel to CEO: This feature directly enables the "private marketplace" use case Lorenzo Storelli's AI Controls prototype was built around (github/agents/discussions/637). Ship fast and mention it in the release announcement as "Private marketplaces, first-class tooling." Update CEO arbitrationSpecialists agree on the overall direction: this is a well-structured, security-conscious addition that fills a real capability gap. The one genuine blocker is the unresolved merge conflict in Required actions before merge
Optional follow-ups
|
danielmeppiel
left a comment
danielmeppiel
left a comment
There was a problem hiding this comment.
Harden against all the panel suggestions, not just the summary below. This is important, because the 2 asks it makes are not the only ones that are key. Specially authentication has to be hardened. Let's do a full hardening based on the panelists independent reviews.
sergio-sisternes-epam
force-pushed
the
feat/722-marketplace-maintainer-ux
branch
from
e528dab to
d1b080d
Compare
|
@danielmeppiel All panel findings have been hardened. Here's the full breakdown: Security
Architecture
Logging
UX
Already fixed in prior commits (verified)S1 (path traversal), S2 (token redaction), S3 (GIT_TERMINAL_PROMPT), A1 (atomic_write), A2 (redact_token), A4 (_SOURCE_RE), A7 (locals().get), UX3 (--marketplace-yml docs), UX4 (version/ref exclusivity), UX7 (publish prompt), C2 (sidebar + cross-links) Deferred to follow-up issues (not blocking merge)
Commit: |
danielmeppiel
added
panel-review
APM Review Panel VerdictDisposition: REQUEST_CHANGES (two small pre-merge fixes; core feature is sound) Per-persona findingsPython Architect: This is a major new subsystem: 8 authoring commands ( 1. OO / class diagramclassDiagram
direction TB
class MarketplaceGroup {
<<CustomClickGroup>>
+format_commands(ctx, formatter)
+_authoring_visible() bool
}
class MarketplaceBuilder {
<<Builder>>
+yml_path Path
+options BuildOptions
+build() BuildReport
}
class BuildOptions {
<<ValueObject>>
+dry_run bool
+offline bool
+include_prerelease bool
}
class BuildReport {
<<ValueObject>>
+resolved list
+warnings list
}
class ResolvedPackage {
<<ValueObject>>
+name str
+ref str
+sha str
}
class MarketplacePublisher {
<<Facade>>
+publish() list
}
class ConsumerTarget {
<<ValueObject>>
+repo str
+branch str
+path_in_repo str
}
class RefResolver {
<<Adapter>>
+offline bool
+list_remote_refs(source) list
+resolve_ref_sha(source, ref) str
}
class MarketplaceYml {
<<ValueObject>>
+name str
+packages list
+build BuildConfig
}
class AuthResolver {
<<Strategy>>
+resolve(host) AuthContext
}
class MarketplaceGroup:::touched
class MarketplaceBuilder:::touched
class MarketplacePublisher:::touched
class ConsumerTarget:::touched
class RefResolver:::touched
MarketplaceBuilder *-- BuildOptions : configured by
MarketplaceBuilder ..> BuildReport : produces
BuildReport *-- ResolvedPackage
MarketplacePublisher *-- ConsumerTarget
MarketplaceBuilder ..> RefResolver : delegates to
MarketplaceBuilder ..> MarketplaceYml : reads
MarketplacePublisher ..> MarketplaceYml : reads
MarketplacePublisher ..> AuthResolver : uses gh CLI (not AuthResolver)
note for MarketplacePublisher "PR creation delegates to gh CLI subprocess\nAuth path is separate from AuthResolver"
note for RefResolver "Adapter over git ls-remote;\noffline mode uses cached refs"
classDef touched fill:#fff3b0,stroke:#d47600
2. Execution flow diagram (
|
Address UX review findings for the plugin subgroup: - Show subcommand names in plugin group help text - Guard `plugin set` against zero-field invocations - Standardise `plugin remove` confirmation via click.confirm - Extract shared _is_interactive() helper to _helpers.py - Remove dead --no-verify flag from `plugin set` - Document plugin commands in CLI reference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Security:
- Add path traversal guard on marketplace.yml output field (S1)
- Suppress git credential prompts with GIT_TERMINAL_PROMPT=0 (S3)
- Validate ConsumerTarget repo/branch against injection (S4)
Architecture:
- Replace locals().get("pr") with explicit variable (A7)
- Make SOURCE_RE public in yml_schema (A4)
Logging:
- Add Rich fallback for publish state-file display (L1)
UX:
- Enforce --version/--ref mutual exclusivity in plugin add (UX4)
- Remove phantom --marketplace-yml from CLI reference docs (UX3)
Docs:
- Wire marketplace authoring guide into docs sidebar (C2)
- Add consumer-to-author cross-link in marketplace guide (G1)
12 new tests covering all security and UX fixes.
Resolves panel findings S1, S3, S4, A7, A4, L1, UX3, UX4, C2, G1.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Architecture: - Extract shared atomic_write() to marketplace/_io.py (A1) - Extract shared redact_token() to marketplace/_git_utils.py (A2) - Fix token redaction regex to cover http:// and ?token= (S2) Logging: - Add verbose traceback output to 5 exception handlers (L3) UX: - Add summary line to outdated command output (UX5) - Exit code 1 when packages are outdated, matching npm/pip (UX5) 17 new tests covering DRY utilities, verbose tracebacks, and outdated summary/exit behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…xclusivity - Add GIT_TERMINAL_PROMPT=0 and GIT_ASKPASS=echo to publisher._run_git() chokepoint (8 subprocess calls including clone/push) — completes S3 fix - Add --version/--ref mutual exclusivity to plugin set (NEW-1 from panel) - Update stale _TOKEN_RE docstring references in publisher and pr_integration (N2) - Tests: TestRunGitEnv (publisher), plugin set conflict test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…/set When no --ref is provided, plugin add now resolves HEAD to a concrete 40-char SHA via git ls-remote before storing it in marketplace.yml. When --ref HEAD or a branch name is given, a warning is emitted and the ref is auto-resolved to its current SHA for supply-chain safety. Explicit SHAs and tags are stored as-is. Adds resolve_ref_sha() to RefResolver for single-ref lookups. 26 new tests covering all resolution paths. Updates CLI reference, marketplace guide, and CHANGELOG. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nit flags - Rename 'plugin' subgroup to 'package' for npm/pip/cargo familiarity - Group marketplace --help into Consumer and Authoring sections - Add --name/--owner flags to marketplace init - Hide unimplemented --check-refs flag - Fix includePrerelease typo in init template - Add short flags (-d, -s) to package add command - Update search metavar to QUERY@MARKETPLACE - Update CHANGELOG, docs, and apm-usage skill Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Builder now detects and warns when multiple packages share the same name in marketplace.yml. check and doctor commands also flag duplicates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uild When a marketplace.yml entry has no description, the builder now fetches the package's apm.yml from its resolved git source to extract the description. This is best-effort -- network failures are silently ignored and the build never fails because of a missing description. Explicit descriptions in marketplace.yml always take precedence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…apm.yml When a marketplace.yml entry lacks description or version, the builder now fetches the package's apm.yml from its resolved git source to extract these fields. Authentication is handled via APM's AuthResolver so private repos are supported. - Removed --description CLI flags from package add/set (metadata is always sourced from the package's own apm.yml) - Removed description from PackageEntry, ResolvedPackage, and yml_editor - Added concurrent metadata pre-fetching with ThreadPoolExecutor - Best-effort enrichment: network failures are silently ignored - 20 new tests covering metadata fetch, auth, and enrichment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ring-fence all marketplace commands behind the `marketplace_commands`
experimental feature flag (default: disabled). Users opt-in via:
apm experimental enable marketplace-commands
Commands are hidden from --help when disabled. Defence-in-depth guard
in the marketplace group callback provides a clear enablement message
if somehow reached directly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rename the experimental feature flag from marketplace-commands to
marketplace-authoring for clarity. The old name suggested the flag
gated CLI commands generically; the new name communicates that it
gates the marketplace authoring and discovery surface.
Users who previously enabled the old flag will see a stale-config
warning and can re-enable with:
apm experimental enable marketplace-authoring
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The marketplace_authoring flag was gating ALL marketplace commands including pre-existing consumer commands (add, list, browse, update, remove, validate, search) that already ship on main. This broke the consumer surface for all users. Now only authoring commands are gated: init, build, check, outdated, doctor, publish, and the package subgroup (add/set/remove). Consumer commands are always available without any flag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Authoring commands (init, build, check, outdated, doctor, publish, package) are now hidden from `marketplace --help` when the marketplace-authoring flag is disabled. Consumer commands remain always visible. The defence-in-depth guard (_require_authoring_flag) stays in each authoring command body so that direct invocation still shows the enablement message rather than a generic Click error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use _rich_warning/_rich_info in authoring flag guard with docs link - Fix "centre" → "center" typo in check table (3 occurrences) - Switch publish confirmation to click.confirm (fixes double-prompt) - Standardise --yes help text across all confirmation commands - Route validate summary through CommandLogger - Route package remove cancel through CommandLogger Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The regex in redact_token() was dropping ? and & separators when redacting query-string tokens (?token=VALUE, &token=VALUE), producing malformed URLs like '...archivetoken=***'. Capture the separator in a group and re-emit it in the replacement. Update test assertions to verify separators are preserved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Security: - ConsumerTarget.__post_init__ validates repo format, branch chars, path traversal - Doctor uses AuthResolver instead of raw env-var lookup - version_pins warns when expected pin file disappears Architecture: - Builder.resolve() returns ResolveResult dataclass (no state smuggling) - 15 bare click.echo -> CommandLogger methods - 3 Rich markup literals -> logger.progress UX: - Doctor checks gh CLI presence (informational) - Confirmation prompts fail loudly in non-interactive mode - Outdated summary simplified + exit code consistency Logging: - 25 exception handlers get verbose tracebacks (debug level) - 3 handlers narrowed from bare Exception to specific types Closes panel findings: S4, S5, L3, L4, L5, L7, A3, UX2, UX5, UX6, UX8 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Move all [Experimental] CHANGELOG entries from released sections (0.9.2, 0.9.3, 0.9.4) to [Unreleased] - Add *.json to gitignore pattern check in _check_gitignore_for_marketplace_json - Fix symbol="cross" -> symbol="error" (2 sites) - Add warning log when _resolve_ref falls back to unresolved ref Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sergio-sisternes-epam
force-pushed
the
feat/722-marketplace-maintainer-ux
branch
from
0555500 to
374f448
Compare
Promotes [Unreleased] to [0.10.0] - 2026-04-27. Each PR since v0.9.4 gets one 'so what' line: - #926 Microsoft 365 Cowork target ships impl - #790 marketplace authoring CLI (init, package add/set, build, check, outdated, doctor, publish) -- collapsed from 20+ bullets to one - #722 marketplace plugin -> package rename + --help sectioning -- collapsed - #980 README 'Coming from npx skills add' conversion block - #981 docs auto-deploy on tag push (real fix for the #953 attempt) - #985 pr-description-skill evals suite - #984 pr-description-skill mermaid hardening - #989 cowork sys.platform mock for Windows CI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Description
First pass of the maintainer-side marketplace tooling tracked in #722. Builds on the foundations landing in #677 (semver engine,
MarketplaceValidatorskeleton, security advisories, publish skeleton).Design context and the full UX proposal are captured in the discussion thread on the tracking issue: #722 (comment).
Hard rule driving this work
marketplace.jsonis Anthropic's standard, unaltered. APM emits the artifact byte-for-byte against Anthropic's schema; APM-only build inputs live inmarketplace.ymland are stripped at compile time. A golden-file test enforces round-trip compatibility with Claude Code.High-level scope
apm marketplace init— scaffoldmarketplace.ymlapm marketplace build— compilemarketplace.yml-> Anthropic-compliantmarketplace.json(concurrent ref resolution, diff output, dry-run)apm marketplace check— validation + freshness for CI (--strict, per-rule toggles)apm marketplace outdated— maintainer-side discovery of stale rangesapm marketplace publish— transactional add/update of a package entry (PR-first,--resume/--abortrecovery)apm marketplace doctor— preflight probe for git / gh / tokens /ls-remoteDetailed plan, UX previews, and the 45-item UX-risk absorption map live in the session plan and will be distilled into a design doc / changelog entry during delivery.
Fixes #722
Type of change
Testing
Draft — work in progress. Please hold review until marked ready.