Skip to content

feat: support allow-insecure HTTP dependencies#700

Merged
danielmeppiel merged 25 commits intomicrosoft:mainfrom
arika0093:feat/allow-http-request
Apr 21, 2026
Merged

feat: support allow-insecure HTTP dependencies#700
danielmeppiel merged 25 commits intomicrosoft:mainfrom
arika0093:feat/allow-http-request

Conversation

@arika0093
Copy link
Copy Markdown
Contributor

@arika0093 arika0093 commented Apr 14, 2026

Description

This PR adds explicit support for HTTP (insecure) APM dependencies behind an opt-in flow, and makes lockfile replays preserve that HTTP source information.

When adding a package, you can explicitly allow HTTP dependencies by providing the --allow-insecure flag.

$ apm install --allow-insecure http://my-server.example.com/owner/repo

# if not provided, it will error like this:
# [*] Validating 1 package...
# [x] http://my-server.example.com/owner/repo -- 'http://my-server.example.com/owner/repo' uses HTTP (insecure). Pass '--allow-insecure' or run 'apm config set allow-insecure true' to allow HTTP dependencies.
# All packages failed validation. Nothing to install.

When restoring packages, you also need to provide the --allow-insecure flag unless the global config is enabled. If there are HTTP URLs in the dependency chain, it will error without --allow-insecure or apm config set allow-insecure true.

$ apm install --allow-insecure

# if not provided, it will error like this:
# [>] Installing dependencies from apm.yml...
# [x] Dependency 'my-server.example.com/owner/repo' uses HTTP (insecure). Pass '--allow-insecure' to apm install, or run 'apm config set allow-insecure true' to allow HTTP dependencies globally.

The apm.yml file will record it in the following format:

dependencies:
  apm:
    - git: http://my-server.example.com/owner/repo
      allow_insecure: true

HTTP dependencies require two explicit signals:

  • allow_insecure: true on the dependency entry in apm.yml
  • either apm install --allow-insecure or apm config set allow-insecure true

The global configuration only removes the need to pass --allow-insecure on the CLI. It does not remove the allow_insecure: true requirement in apm.yml.

apm config set allow-insecure true

Fixes #636

TODO

  • Support for --allow-insecure flag when adding dependencies
  • Support for --allow-insecure flag when installing dependencies
  • Support for allow_insecure: true field in apm.yml and lockfile
  • Support for allow-insecure global config

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

@arika0093 arika0093 marked this pull request as ready for review April 14, 2026 08:43
Copilot AI review requested due to automatic review settings April 14, 2026 08:43
@arika0093
Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR introduces an explicit opt-in flow for allowing HTTP (insecure) APM dependencies, including persisting that decision in apm.yml and preserving HTTP metadata during lockfile replays.

Changes:

  • Add --allow-insecure flag + global allow-insecure config to gate installation of http:// dependencies.
  • Persist and replay HTTP dependency metadata (is_insecure, allow_insecure) via manifest and lockfile.
  • Update docs and add unit tests covering HTTP parsing, lockfile round-trip, CLI behaviors, and cloning behavior.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/unit/test_install_update.py Adds lockfile replay/round-trip tests for insecure HTTP metadata.
tests/unit/test_install_command.py Adds CLI tests for --allow-insecure and install-time security checks.
tests/unit/test_config_command.py Adds config command + config function tests for allow-insecure.
tests/unit/test_canonicalization.py Adds parsing/serialization tests for HTTP dependency handling.
tests/unit/test_auth_scoping.py Ensures HTTP deps don’t fall back to SSH during clone attempts.
src/apm_cli/models/dependency/reference.py Introduces is_insecure/allow_insecure, HTTP canonicalization, manifest serialization, and scheme-aware URLs.
src/apm_cli/drift.py Preserves insecure scheme info during lockfile replay.
src/apm_cli/deps/lockfile.py Persists is_insecure/allow_insecure in lockfile read/write and dependency refs.
src/apm_cli/deps/github_downloader.py Adds HTTP-only clone path and URL building for insecure deps.
src/apm_cli/config.py Adds get_allow_insecure / set_allow_insecure.
src/apm_cli/commands/install.py Adds --allow-insecure, manifest writing behavior, and install-time HTTP enforcement.
src/apm_cli/commands/config.py Generalizes config get/set to support allow-insecure.
src/apm_cli/commands/_helpers.py Propagates HTTP/artifactory/local fields when computing install paths.
src/apm_cli/bundle/plugin_exporter.py Propagates HTTP/artifactory/local fields for exported install paths.
packages/apm-guide/.apm/skills/apm-usage/commands.md Documents --allow-insecure in the command reference table.
docs/src/content/docs/reference/cli-commands.md Documents --allow-insecure and new config key.
docs/src/content/docs/guides/dependencies.md Documents HTTP dependency manifest format + caution guidance.

Comment thread src/apm_cli/models/dependency/reference.py Outdated
Comment thread src/apm_cli/models/dependency/reference.py Outdated
Comment thread src/apm_cli/models/dependency/reference.py Outdated
Comment thread src/apm_cli/commands/config.py
Comment thread src/apm_cli/deps/github_downloader.py Outdated
Copy link
Copy Markdown
Collaborator

@sergio-sisternes-epam sergio-sisternes-epam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid feature -- good security model with dual opt-in (dep-level + CLI/config), thorough test coverage (30+ tests), clean lockfile round-trip, and well-written docs. Nice work!

One blocking issue: to_apm_yml_entry() hardcodes allow_insecure = True, which could silently escalate security posture. See inline comment.

Also: this PR silently fixes a bug where artifactory_prefix was missing from DependencyReference construction in _helpers.py, plugin_exporter.py, and lockfile.py -- directly related to #614. Consider referencing that issue in the PR description.

entry["ref"] = self.reference
if self.alias:
entry["alias"] = self.alias
entry["allow_insecure"] = True
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: This hardcodes allow_insecure = True for all HTTP deps, ignoring self.allow_insecure. If this method is reused in a code path that hasn't applied the opt-in check, it silently grants insecure access.

Suggested change
entry["allow_insecure"] = True
entry["allow_insecure"] = self.allow_insecure

Copy link
Copy Markdown
Contributor Author

@arika0093 arika0093 Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in commit cd8bed0.

ado_project=ado_project,
ado_repo=ado_repo,
artifactory_prefix=artifactory_prefix,
is_insecure=dependency_str.startswith("http://"),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: startswith("http://") is case-sensitive, but URL schemes are case-insensitive per RFC 3986. HTTP://host/repo would bypass insecure detection.

Suggested change
is_insecure=dependency_str.startswith("http://"),
is_insecure=urllib.parse.urlparse(dependency_str).scheme.lower() == "http",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in commit 3c0122c.

Comment thread src/apm_cli/deps/github_downloader.py Outdated
return build_ssh_url(host, repo_ref)
elif is_github and github_token:
# Only send GitHub tokens to GitHub hosts
# # Only send GitHub tokens to GitHub hosts
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Double # — typo from the refactor commit.

Suggested change
# # Only send GitHub tokens to GitHub hosts
# Only send GitHub tokens to GitHub hosts

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in fe7179f.

sub_path = entry.get("path")
ref_override = entry.get("ref")
alias_override = entry.get("alias")
allow_insecure = bool(entry.get("allow_insecure", False))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: bool(entry.get("allow_insecure", False)) treats any non-empty string (e.g., "false") as True. Since this is a security-relevant field, consider validating the type explicitly:

Suggested change
allow_insecure = bool(entry.get("allow_insecure", False))
allow_insecure = entry.get("allow_insecure", False)
if not isinstance(allow_insecure, bool):
raise ValueError("'allow_insecure' field must be a boolean")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in commit 47b87ab.

@arika0093
Copy link
Copy Markdown
Contributor Author

@sergio-sisternes-epam
Thanks for the review. I addressed the points you raised, so could you please take another look?

On #614: this PR now explicitly passes artifactory_prefix=dep.registry_prefix when reconstructing DependencyReference, which fixes a related missing-prefix propagation path. But I do not think it resolves the core #614 issue around _parse_artifactory_base_url() ignoring PROXY_REGISTRY_URL; that still appears to be unresolved.

Copy link
Copy Markdown
Collaborator

@sergio-sisternes-epam sergio-sisternes-epam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All findings addressed — nice work, @arika0093! Each fix is clean and well-tested.

Thanks for the clarification on #614 — you're right that the artifactory_prefix propagation fix here is a partial improvement but the core _parse_artifactory_base_url() issue remains separate.

LGTM!

@danielmeppiel danielmeppiel added CI/CD Deprecated: use area/ci-cd. Kept for issue history; will be removed in milestone 0.10.0. and removed CI/CD Deprecated: use area/ci-cd. Kept for issue history; will be removed in milestone 0.10.0. labels Apr 19, 2026
Copy link
Copy Markdown
Collaborator

@danielmeppiel danielmeppiel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First — thanks for tackling this, @arika0093. The use case is real (internal mirrors without HTTPS exist in the wild), the dual opt-in design (per-dep allow_insecure: true + CLI flag) is the right shape, and the test coverage is thorough. Sergio's APPROVE isn't wrong on the implementation quality.

I'm requesting changes from a security/UX lens because APM is a supply-chain tool, and a few aspects of the current design create footguns that I don't think we can ship to the broader community. None of these are deal-breakers — they're scope tightening. If we land them, this becomes a feature I'd be proud to ship.

Blockers

1. Drop apm config set allow-insecure true (the persistent global config)

This is the biggest single risk in the PR and, IMO, the only true blocker.

Per-invocation --allow-insecure is the right friction ceiling. A persistent global setting:

  • Means a developer enables it once for one internal project and then forever silently allows HTTP for every subsequent apm install on that machine, including in unrelated repos.
  • Turns shared CI runners into a network-MITM surface for any repo that lands allow_insecure: true in apm.yml — the runner doesn't need to opt in per-job, the maintainer who set the config months ago already did.
  • Is invisible at install time. There's no per-install reminder. Users forget they set it.
  • Converts a per-action security decision into ambient permission, which is the textbook supply-chain footgun.

The per-invocation flag is the correct model. It's friction, but the friction is the security feature: every HTTP install requires an active, conscious choice. Please drop the apm config set allow-insecure path entirely (and the corresponding get_allow_insecure / set_allow_insecure config functions).

2. Be loud at install time about which URLs are insecure

Today, the user only sees output about HTTP when --allow-insecure is missing (the error path). On the success path there's no equivalent loud diagnostic listing exactly which URLs are about to be fetched over an unauthenticated channel.

Compare to pip's behavior on insecure indexes: a clear warning per fetch. APM should do the same.

Concretely:

  • Print one [!] line per HTTP dependency at install-prepare time, with the full URL: [!] Fetching insecurely (no transport auth): http://internal.example.com/team/foo
  • Make this print whether or not --allow-insecure was passed (i.e. the warning is informational; the flag is the gate).
  • This is what makes Alice notice when Bob's apm.yml grew an unexpected HTTP entry between git pulls.

3. Transitive HTTP dependencies need a separate gate

This is the most subtle one and the one I'd most appreciate your thinking on. Scenario:

  • Bob adds a top-level dep http://internal.bob.example/foo/bar with allow_insecure: true to the project's apm.yml. Alice can see this in the diff and consents by passing --allow-insecure.
  • But Bob's package itself has an apm.yml that pulls a transitive dep http://attacker-controlled.example/lib/x over HTTP, also with allow_insecure: true.
  • When Alice runs apm install --allow-insecure, that transitive dep gets fetched too. Alice never saw it in the project's own apm.yml. Her consent at the root extends silently to a graph she didn't review.

The lockfile makes this auditable after it's pinned, but the first install (and any update) doesn't surface this distinction.

Suggested approach (open to alternatives):

  • Track per-dep whether the HTTP+allow_insecure choice originated in the root apm.yml or in a transitive dep.
  • For transitive HTTP, require an additional explicit acknowledgment: either a separate --allow-insecure-transitive flag, or an interactive prompt listing the transitive HTTP URLs and their introducing parent, with --yes to skip in CI.
  • At minimum, the per-URL warning from #2 should annotate (transitive, introduced by <parent-dep>) so users can see it.

Without this, the dual-opt-in only protects the immediate dep — the transitive surface is a single root flag away from being silently expanded.

Non-blocking but please consider

  • Threat-model docs: the new HTTP section in dependencies.md documents the mechanism but not the why. A short paragraph explaining that HTTP has no transport auth (so an MITM can substitute the package contents — the SHA in the lockfile is itself MITM-able when fetched over HTTP) would help users make informed choices instead of copy-pasting the example.
  • Audit affordance: there's no way to ask "list every insecure dep in my dependency graph" before running install. With the lockfile already preserving is_insecure, an apm deps list --insecure (or apm install --dry-run listing them) would be cheap and high-value. Could be a follow-up.
  • CHANGELOG entry: this is a security-sensitive feature. The CHANGELOG entry should be explicit that this introduces an opt-in HTTP path with the threat-model implications, not just "feat: support http deps."
  • Hostname allowlist (future, not now): a future iteration could let apm.yml declare an insecure_hosts: [...] allowlist so that HTTP is only permitted from explicitly-named hosts. Not for this PR.

What I'd merge

If you land #1 (drop global config) and #2 (loud per-URL output), I'd merge. #3 (transitive gating) is the one I feel strongest about from a supply-chain perspective; if you'd prefer to defer it, I'd accept that as a follow-up issue only if the per-URL warning from #2 explicitly annotates transitive HTTP deps so the information is at least visible.

If you'd rather not drop the global config, I'd respectfully decline this PR and recommend users set up HTTPS on their internal mirrors. APM is a supply-chain tool and persistent ambient "allow insecure" is incompatible with that role.

Thanks again for the thoughtful design and the thorough tests — this is salvageable and worth landing well.

@arika0093
Copy link
Copy Markdown
Contributor Author

arika0093 commented Apr 20, 2026

Thanks for the thoughtful review. I think your concerns are reasonable, especially given that APM is a supply-chain tool.

I agree on the first two blockers:

  • I will drop the persistent global allow-insecure config.
  • I will make insecure fetches loud on the success path as well, with one warning per HTTP dependency showing the full URL.

On transitive HTTP dependencies: I agree this is the trickiest part. A root-level opt-in should not silently bless an unreviewed transitive graph.

After thinking more about it, I do not think a simple --allow-insecure-transitive flag is the right long-term shape, because it broadens trust based on graph position rather than which host is actually being trusted.

For this PR, my plan is:

  • a root HTTP dependency allowed via --allow-insecure will also allow transitive HTTP dependencies from the same host
  • transitive HTTP dependencies from a different host will still be blocked by default
  • for non-interactive environments, I will add an explicit host-level opt-in for additional hosts (for example, a repeatable --allow-insecure-host <host> flag)
  • install output will clearly warn for every HTTP dependency, including whether it is transitive and which parent introduced it

I think this is a better narrow fix for the security concern raised here, while also fitting the common internal-repository use case more naturally than a boolean transitive flag.

I will also update the docs and changelog accordingly. The audit affordance (apm deps list --insecure) also makes sense to me.

Thanks again for the careful review.

danielmeppiel added a commit that referenced this pull request Apr 20, 2026
- docs(dependencies): pin BREAKING-change caution panel to APM 0.8.13
  (per maintainer review comment)
- transport_selection: collapse explicit `http://` URLs into the plain-HTTPS
  branch so TransportAttempt.scheme stays in {ssh, https} and the downloader
  contract is consistent until #700 lands a first-class HTTP transport
- github_downloader: gate the [!] "Protocol fallback" warning on actual
  scheme change (ssh<->https) rather than label change, so an auth downgrade
  inside a single protocol is not misreported as a protocol switch
- pipeline / request / context / commands: switch
  `allow_protocol_fallback` to `Optional[bool] = None` end-to-end so
  programmatic callers (non-CLI) keep the documented "None => read
  APM_ALLOW_PROTOCOL_FALLBACK env" behavior
- test_auth_scoping: ASCII-only docstring (replace one stray Unicode arrow)

All 4061 unit tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel added a commit that referenced this pull request Apr 20, 2026
… (#779)

* feat(transport): TransportSelector + strict-by-default transport (#778)

Implements the core decision engine for issue #778 'transport selection v1'.
Strict-by-default semantics replace today's silent cross-protocol fallback:
explicit ssh:// and https:// dependencies no longer downgrade to a different
protocol, and shorthand (owner/repo) consults git insteadOf rewrites before
defaulting to HTTPS.

This commit ships Waves 1+2 of the transport-selection plan (per session plan):
- new module src/apm_cli/deps/transport_selection.py with ProtocolPreference,
  TransportAttempt/TransportPlan dataclasses, GitConfigInsteadOfResolver,
  and TransportSelector that returns a typed, strict-by-default plan
- DependencyReference grows explicit_scheme so the selector can distinguish
  user-stated transport from shorthand
- _clone_with_fallback in github_downloader.py now iterates the selector
  plan; per-attempt URL building stays in the orchestrator
- InstallContext / InstallRequest / pipeline / Service threaded with
  protocol_pref + allow_protocol_fallback so CLI args reach the downloader
- apm install gains --ssh / --https (mutually exclusive) and
  --allow-protocol-fallback flags; honours APM_GIT_PROTOCOL and
  APM_ALLOW_PROTOCOL_FALLBACK env vars
- two pre-existing tests in test_auth_scoping.py asserted the legacy
  permissive chain; updated to assert the new strict contract and added a
  coverage test for the allow_fallback escape hatch

Tests: 4029 unit tests pass. Test matrix + integration tests + docs land
in subsequent commits per Waves 3-5.

Refs #778, #328, #661

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(transport): wave-3 unit + integration matrix; fix per-attempt clone env

Wave 2 panel gate (code-review subagent) flagged that
GitHubPackageDownloader._clone_with_fallback decided the clone env
(locked-down vs relaxed) ONCE per dependency from has_token. Under
allow_fallback=True the plan can mix attempts of different use_token
values, so SSH and plain-HTTPS attempts in a mixed chain were running
with GIT_ASKPASS=echo + GIT_CONFIG_GLOBAL=/dev/null +
GIT_CONFIG_NOSYSTEM=1, breaking ssh-agent passphrase prompts and git
credential helpers.

Fix the env per attempt; address an adjacent contract bug; add tests.

* github_downloader._clone_with_fallback: replace the per-dep clone_env
  with a per-attempt _env_for() helper so only token-bearing attempts
  get the locked-down env.
* github_downloader._build_repo_url: treat token="" as an explicit "no
  token" sentinel so plain-HTTPS attempts in a mixed chain genuinely
  run without embedded credentials, letting credential helpers (gh
  auth, Keychain) supply auth. Orchestrator passes "" instead of None
  for use_token=False attempts.
* transport_selection.GitConfigInsteadOfResolver: wrap the lazy
  insteadOf-rewrite cache in a threading.Lock so parallel downloads
  can't double-populate.

Tests:
* tests/unit/test_transport_selection.py (NEW, 30 tests): 14-row
  selection matrix (explicit-strict, shorthand+insteadOf, shorthand
  defaults, CLI prefs, allow_fallback chain, env helpers); resolver
  caching; "must use normal env" contract.
* tests/unit/test_auth_scoping.py: new
  test_allow_fallback_env_is_per_attempt_not_per_dep regression
  asserts auth-HTTPS gets locked-down env, SSH and plain-HTTPS get
  relaxed env, and plain-HTTPS does not embed the token in the URL.
* tests/integration/test_transport_selection_integration.py (NEW, 7
  tests): 2 always-on cases (public shorthand HTTPS; explicit https://
  strict); 5 SSH-required cases (explicit ssh:// strict, bad-host
  no-fallback, insteadOf override, APM_GIT_PROTOCOL=ssh env,
  allow_fallback rescue). Gated on APM_RUN_INTEGRATION_TESTS=1; SSH
  cases auto-skip if no key.
* tests/fixtures/gitconfig_insteadof_to_ssh (NEW): minimal gitconfig
  used by the integration test for the insteadOf-honored case.
* scripts/test-integration.sh: added "Transport Selection" block so
  the integration suite runs in CI.

Full unit suite: 4061 passed (was 4029; +32 net new tests).

Refs #778, #661, #328

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(transport): document strict-by-default transport selection (#778)

Update the four user-facing surfaces affected by the new TransportSelector
contract:

* docs/src/content/docs/guides/dependencies.md:
  - New "Transport selection (SSH vs HTTPS)" section: breaking-change
    callout with the rescue env var, selection matrix, insteadOf
    example, --ssh / --https / APM_GIT_PROTOCOL overrides, and the
    --allow-protocol-fallback escape hatch.
  - Soften the existing "Custom ports preserved" sentence (cross-protocol
    retries are now opt-in).
  - Update the "Other Git Hosts" SSH bullet: SSH is no longer a silent
    fallback; point at explicit URLs or insteadOf.
* docs/src/content/docs/getting-started/authentication.md:
  - Rewrite the "SSH connection hangs" troubleshooting entry: remove the
    now-incorrect "tries SSH then falls back to HTTPS" framing.
  - New "Choosing transport (SSH vs HTTPS)" section with a pointer to
    dependencies.md for the full transport contract.
* docs/src/content/docs/reference/cli-commands.md:
  - Document --ssh / --https / --allow-protocol-fallback on apm install,
    plus APM_GIT_PROTOCOL and APM_ALLOW_PROTOCOL_FALLBACK env vars.
* packages/apm-guide/.apm/skills/apm-usage/dependencies.md (Rule 4 mirror):
  - Same transport contract in skill-resource voice with three runnable
    snippets and a selection matrix.

CHANGELOG: scope new entries to `apm install` only (there is no `apm
add` command in the codebase).

Refs #778

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: revert accidental uv.lock churn (no dependency change in this PR)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs+changelog: address copilot review feedback (#779)

Two findings from the Copilot reviewer on PR #779:

1. Non-ASCII em dash introduced by this PR in the modified `Fields:`
   line of guides/dependencies.md: replace with `--`. The other
   non-ASCII chars Copilot flagged in the file (lines ~150-160 of the
   "nested groups" warning block) are pre-existing and out of scope
   for this PR.
2. CHANGELOG entries for the new transport-selection feature were too
   long and bundled multiple concerns into one bullet. Split into one
   tighter BREAKING entry plus two single-purpose Added entries
   (initial-protocol flags; fallback escape hatch). Each ends in
   `(#778)`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #779 review feedback

- docs(dependencies): pin BREAKING-change caution panel to APM 0.8.13
  (per maintainer review comment)
- transport_selection: collapse explicit `http://` URLs into the plain-HTTPS
  branch so TransportAttempt.scheme stays in {ssh, https} and the downloader
  contract is consistent until #700 lands a first-class HTTP transport
- github_downloader: gate the [!] "Protocol fallback" warning on actual
  scheme change (ssh<->https) rather than label change, so an auth downgrade
  inside a single protocol is not misreported as a protocol switch
- pipeline / request / context / commands: switch
  `allow_protocol_fallback` to `Optional[bool] = None` end-to-end so
  programmatic callers (non-CLI) keep the documented "None => read
  APM_ALLOW_PROTOCOL_FALLBACK env" behavior
- test_auth_scoping: ASCII-only docstring (replace one stray Unicode arrow)

All 4061 unit tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel added a commit that referenced this pull request Apr 20, 2026
…ade (#780) (#781)

* feat(transport): TransportSelector + strict-by-default transport (#778)

Implements the core decision engine for issue #778 'transport selection v1'.
Strict-by-default semantics replace today's silent cross-protocol fallback:
explicit ssh:// and https:// dependencies no longer downgrade to a different
protocol, and shorthand (owner/repo) consults git insteadOf rewrites before
defaulting to HTTPS.

This commit ships Waves 1+2 of the transport-selection plan (per session plan):
- new module src/apm_cli/deps/transport_selection.py with ProtocolPreference,
  TransportAttempt/TransportPlan dataclasses, GitConfigInsteadOfResolver,
  and TransportSelector that returns a typed, strict-by-default plan
- DependencyReference grows explicit_scheme so the selector can distinguish
  user-stated transport from shorthand
- _clone_with_fallback in github_downloader.py now iterates the selector
  plan; per-attempt URL building stays in the orchestrator
- InstallContext / InstallRequest / pipeline / Service threaded with
  protocol_pref + allow_protocol_fallback so CLI args reach the downloader
- apm install gains --ssh / --https (mutually exclusive) and
  --allow-protocol-fallback flags; honours APM_GIT_PROTOCOL and
  APM_ALLOW_PROTOCOL_FALLBACK env vars
- two pre-existing tests in test_auth_scoping.py asserted the legacy
  permissive chain; updated to assert the new strict contract and added a
  coverage test for the allow_fallback escape hatch

Tests: 4029 unit tests pass. Test matrix + integration tests + docs land
in subsequent commits per Waves 3-5.

Refs #778, #328, #661

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(transport): wave-3 unit + integration matrix; fix per-attempt clone env

Wave 2 panel gate (code-review subagent) flagged that
GitHubPackageDownloader._clone_with_fallback decided the clone env
(locked-down vs relaxed) ONCE per dependency from has_token. Under
allow_fallback=True the plan can mix attempts of different use_token
values, so SSH and plain-HTTPS attempts in a mixed chain were running
with GIT_ASKPASS=echo + GIT_CONFIG_GLOBAL=/dev/null +
GIT_CONFIG_NOSYSTEM=1, breaking ssh-agent passphrase prompts and git
credential helpers.

Fix the env per attempt; address an adjacent contract bug; add tests.

* github_downloader._clone_with_fallback: replace the per-dep clone_env
  with a per-attempt _env_for() helper so only token-bearing attempts
  get the locked-down env.
* github_downloader._build_repo_url: treat token="" as an explicit "no
  token" sentinel so plain-HTTPS attempts in a mixed chain genuinely
  run without embedded credentials, letting credential helpers (gh
  auth, Keychain) supply auth. Orchestrator passes "" instead of None
  for use_token=False attempts.
* transport_selection.GitConfigInsteadOfResolver: wrap the lazy
  insteadOf-rewrite cache in a threading.Lock so parallel downloads
  can't double-populate.

Tests:
* tests/unit/test_transport_selection.py (NEW, 30 tests): 14-row
  selection matrix (explicit-strict, shorthand+insteadOf, shorthand
  defaults, CLI prefs, allow_fallback chain, env helpers); resolver
  caching; "must use normal env" contract.
* tests/unit/test_auth_scoping.py: new
  test_allow_fallback_env_is_per_attempt_not_per_dep regression
  asserts auth-HTTPS gets locked-down env, SSH and plain-HTTPS get
  relaxed env, and plain-HTTPS does not embed the token in the URL.
* tests/integration/test_transport_selection_integration.py (NEW, 7
  tests): 2 always-on cases (public shorthand HTTPS; explicit https://
  strict); 5 SSH-required cases (explicit ssh:// strict, bad-host
  no-fallback, insteadOf override, APM_GIT_PROTOCOL=ssh env,
  allow_fallback rescue). Gated on APM_RUN_INTEGRATION_TESTS=1; SSH
  cases auto-skip if no key.
* tests/fixtures/gitconfig_insteadof_to_ssh (NEW): minimal gitconfig
  used by the integration test for the insteadOf-honored case.
* scripts/test-integration.sh: added "Transport Selection" block so
  the integration suite runs in CI.

Full unit suite: 4061 passed (was 4029; +32 net new tests).

Refs #778, #661, #328

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(transport): document strict-by-default transport selection (#778)

Update the four user-facing surfaces affected by the new TransportSelector
contract:

* docs/src/content/docs/guides/dependencies.md:
  - New "Transport selection (SSH vs HTTPS)" section: breaking-change
    callout with the rescue env var, selection matrix, insteadOf
    example, --ssh / --https / APM_GIT_PROTOCOL overrides, and the
    --allow-protocol-fallback escape hatch.
  - Soften the existing "Custom ports preserved" sentence (cross-protocol
    retries are now opt-in).
  - Update the "Other Git Hosts" SSH bullet: SSH is no longer a silent
    fallback; point at explicit URLs or insteadOf.
* docs/src/content/docs/getting-started/authentication.md:
  - Rewrite the "SSH connection hangs" troubleshooting entry: remove the
    now-incorrect "tries SSH then falls back to HTTPS" framing.
  - New "Choosing transport (SSH vs HTTPS)" section with a pointer to
    dependencies.md for the full transport contract.
* docs/src/content/docs/reference/cli-commands.md:
  - Document --ssh / --https / --allow-protocol-fallback on apm install,
    plus APM_GIT_PROTOCOL and APM_ALLOW_PROTOCOL_FALLBACK env vars.
* packages/apm-guide/.apm/skills/apm-usage/dependencies.md (Rule 4 mirror):
  - Same transport contract in skill-resource voice with three runnable
    snippets and a selection matrix.

CHANGELOG: scope new entries to `apm install` only (there is no `apm
add` command in the codebase).

Refs #778

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: revert accidental uv.lock churn (no dependency change in this PR)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs+changelog: address copilot review feedback (#779)

Two findings from the Copilot reviewer on PR #779:

1. Non-ASCII em dash introduced by this PR in the modified `Fields:`
   line of guides/dependencies.md: replace with `--`. The other
   non-ASCII chars Copilot flagged in the file (lines ~150-160 of the
   "nested groups" warning block) are pre-existing and out of scope
   for this PR.
2. CHANGELOG entries for the new transport-selection feature were too
   long and bundled multiple concerns into one bullet. Split into one
   tighter BREAKING entry plus two single-purpose Added entries
   (initial-protocol flags; fallback escape hatch). Each ends in
   `(#778)`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #779 review feedback

- docs(dependencies): pin BREAKING-change caution panel to APM 0.8.13
  (per maintainer review comment)
- transport_selection: collapse explicit `http://` URLs into the plain-HTTPS
  branch so TransportAttempt.scheme stays in {ssh, https} and the downloader
  contract is consistent until #700 lands a first-class HTTP transport
- github_downloader: gate the [!] "Protocol fallback" warning on actual
  scheme change (ssh<->https) rather than label change, so an auth downgrade
  inside a single protocol is not misreported as a protocol switch
- pipeline / request / context / commands: switch
  `allow_protocol_fallback` to `Optional[bool] = None` end-to-end so
  programmatic callers (non-CLI) keep the documented "None => read
  APM_ALLOW_PROTOCOL_FALLBACK env" behavior
- test_auth_scoping: ASCII-only docstring (replace one stray Unicode arrow)

All 4061 unit tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): MARKETPLACE_PLUGIN beats HOOK_PACKAGE in detection cascade (#780)

obra/superpowers (and any Claude Code plugin shipping both hooks/*.json
and agents/skills/commands) was misclassified as HOOK_PACKAGE because
the cascade in detect_package_type() checked _has_hook_json BEFORE the
plugin-evidence branch. The plugin synthesizer (_map_plugin_artifacts)
already maps hooks alongside agents/skills/commands, so MARKETPLACE_PLUGIN
is a strict superset -- swapping the order means hooks still install,
plus everything else that was being silently dropped.

Three deliverables:

A) Surgical detection fix: reorder cascade so MARKETPLACE_PLUGIN is
   checked before HOOK_PACKAGE. Refactored to use a new
   gather_detection_evidence() helper + DetectionEvidence dataclass so
   observability code (warnings, summaries) can reuse the same scan
   without breaking the detect_package_type() public signature.

B) Observability:
   - Add HOOK_PACKAGE to the package-type label table (it was missing
     entirely -- the silent classification path).
   - Update MARKETPLACE_PLUGIN label to mention plugin.json OR
     agents/skills/commands (matches the cascade behaviour).
   - New CommandLogger.package_type_warn() at default visibility.
   - New _warn_if_classification_near_miss() helper fires when a
     HOOK_PACKAGE classification disagrees with directory contents
     (catches near-misses the order swap cannot, e.g. .claude-plugin/
     dir without plugin.json).
   - Wired at both materialization sites (local + cached).

C) Architectural follow-up tracked in plan; will file as separate
   issue after merge for a Visitor / Format-Discovery refactor.

Tests: 17 detection tests + 9 sources-observability tests + 70
command-logger tests pass. Full unit suite (4180 tests) green.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #781 review: broaden detect_package_type with .claude-plugin/ evidence

Per Copilot review on PR #781: the near-miss warning helper had dead
code paths once the cascade was reordered (a HOOK_PACKAGE classification
implies plugin_dirs_present is empty -- otherwise it would be
MARKETPLACE_PLUGIN). Two principled options were offered: simplify the
helper, or broaden detection so the invariants stay consistent. Chose
the latter -- it removes the inconsistency at the source.

Changes:
- Add .claude-plugin/ as first-class plugin evidence in DetectionEvidence
  (new has_claude_plugin_dir field) and in has_plugin_evidence. A Claude
  Code plugin without a plugin.json (manifest-less) now classifies as
  MARKETPLACE_PLUGIN; normalize_plugin_directory already handles the
  missing-manifest case (derives name from directory).
- Drop _warn_if_classification_near_miss helper, its two call sites,
  CommandLogger.package_type_warn method, and the corresponding tests.
  All scenarios it covered are now handled by the cascade itself.
- Add regression test test_claude_plugin_dir_alone_is_plugin_evidence
  asserting that .claude-plugin/ + hooks/ classifies as MARKETPLACE_PLUGIN
  with plugin_json_path=None (matched via directory evidence alone).
- Extend test_obra_superpowers_evidence to assert has_claude_plugin_dir.

Verified: 4175 unit tests pass (excluding the one pre-existing unrelated
failure documented on main).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@arika0093 arika0093 force-pushed the feat/allow-http-request branch 3 times, most recently from dbb6977 to 1796653 Compare April 21, 2026 04:56
Copy link
Copy Markdown
Collaborator

@danielmeppiel danielmeppiel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APM Review Panel -- PR #700

[x] Verdict: REQUEST CHANGES -- one CRITICAL credential-leak finding plus a converging architectural/security issue around HTTP identity. The dual-opt-in design is sound and Sergio's earlier blocker is correctly fixed; the items below are what stand between this and merge.

@arika0093 -- this is a substantial, security-sensitive contribution and the design instincts are right. Five reviewers (python-architect, cli-logging-expert, devx-ux-expert, supply-chain-security-expert; growth-hacker side-channel) converged on a tight set of must-fixes. Thank you for the depth here -- 1338 lines + 30+ tests on a class of feature most projects ship sloppily.

What's excellent

  • Dual opt-in (manifest allow_insecure: true + CLI --allow-insecure) is the right mental model. Matches npm overrides + --legacy-peer-deps. Neither alone is sufficient -- correct.
  • Persistent global config dropped per earlier review feedback. Right call; every install run should be a conscious act.
  • --allow-insecure-host for transitive deps with same-host auto-propagation for direct deps is well-designed. Good FQDN validation prevents flag injection.
  • parse_from_dict() validates allow_insecure is boolean -- prevents YAML type confusion.
  • Doc surface coverage (cli-commands.md, dependencies.md, apm-guide skill resources, CHANGELOG) is exactly the discipline this codebase demands.

Required changes (must land in this PR)

  1. [x] CRITICAL -- HTTP clone leaks credentials in plaintext. github_downloader.py _env_for(use_token=False) strips GIT_ASKPASS and GIT_CONFIG_NOSYSTEM for HTTP attempts, re-enabling system git credential helpers (macOS Keychain, Windows Credential Manager, gh auth). When git clones http://internal.example.com/owner/repo, credential helpers may resolve stored tokens for that host and embed them as plaintext Authorization headers. The existing test test_insecure_http_dep_is_strict_by_default asserts "GIT_ASKPASS" not in env_used -- that test is verifying the vulnerability.

    Fix: HTTP attempts must actively block credential resolution:

    if attempt.scheme == "http":
        env = dict(self.git_env)
        env['GIT_ASKPASS'] = 'echo'
        env['GIT_TERMINAL_PROMPT'] = '0'
        env['GIT_CONFIG_NOSYSTEM'] = '1'
        # Suppress credential helpers explicitly
        env['GIT_CONFIG_COUNT'] = '1'
        env['GIT_CONFIG_KEY_0'] = 'credential.helper'
        env['GIT_CONFIG_VALUE_0'] = ''
        return env

    Update the test to assert credential-helper suppression for HTTP rather than its absence.

  2. [x] BLOCKER -- HTTP/HTTPS identity collision enables transport downgrade. get_unique_key() and get_identity() are scheme-blind, and drift.py documents "Source/host/scheme changes -- not detected." That means https://gitlab.com/team/rules and http://gitlab.com/team/rules collide as the same dep. An attacker submitting a PR that flips the scheme + adds allow_insecure: true produces zero lockfile drift signal. Combined with finding #1, this is a clean transport-downgrade exploit chain.

    Fix: Either (a) get_identity() includes scheme when is_insecure=True, or (b) drift.py treats a is_insecure flip as drift requiring --update. Option (b) is the smaller change and preserves identity stability for non-HTTP deps.

  3. [x] BLOCKER -- Inverted import: install/phases/resolve.py imports from commands/install.py. resolve.py:297-300:

    from apm_cli.commands.install import (
        _check_insecure_dependencies, _collect_insecure_dependency_infos,
        _guard_transitive_insecure_dependencies, _warn_insecure_dependencies,
    )

    Domain code (install/phases/) must never import from CLI command code. Extract the ~200 lines of insecure-policy logic + _InsecureDependencyInfo dataclass into a new src/apm_cli/install/insecure_policy.py. Both commands/install.py and install/phases/resolve.py then import from the new module. Pure move, zero logic change.

  4. [x] BLOCKER -- drift.py propagates is_insecure but not allow_insecure during lockfile replay. Lines +250-252: build_download_ref() copies is_insecure=True but drops allow_insecure. The replayed DependencyReference has is_insecure=True, allow_insecure=False, which _check_insecure_dependencies() rejects. Lockfile replays of HTTP deps are broken (or worse, bypass the check via a different code path).

    if getattr(locked_dep, "is_insecure", False) is True:
        overrides["is_insecure"] = True
        overrides["allow_insecure"] = getattr(locked_dep, "allow_insecure", False)
  5. [x] BLOCKER -- to_canonical() no longer strips http://, breaking the canonical identity contract. Architect + DevX both flagged this. The docstring update silently removes "no https://" and HTTP deps now produce http://host/owner/repo while HTTPS still strips. Every consumer that does dep.to_canonical().split("/") or assumes no :// will silently diverge for HTTP deps. Fix: keep to_canonical() scheme-free. The transport-aware string already exists as to_apm_yml_entry() -- use that for serialization paths and any surface that needs the scheme.

  6. [!] HIGH -- Add LockedDependency.to_dependency_ref() factory. Three sites reconstruct DependencyReference from LockedDependency: _helpers.py:138, plugin_exporter.py:399, lockfile.py:365. This PR already had to fix the missing artifactory_prefix (#614 root cause) AND add is_insecure/allow_insecure at all three. Per the "abstract when 3+ call sites" rule, add a factory method that maps every field once. This is the structural fix Sergio asked about -- the artifactory_prefix patch is a band-aid without it.

  7. [!] HIGH -- Unify the three competing error messages into one canonical recipe. Today a user sees three different errors depending on which gate failed:

    • validation-time ("Pass --allow-insecure"),
    • install-time/manifest-missing ("set allow_insecure: true"),
    • install-time/flag-missing ("Pass --allow-insecure").

    None give the complete recipe. Collapse to one message that always names both requirements:

    [x] http://my-server.example.com/owner/repo -- HTTP dependency (no transport encryption)
        To install:
          1. Set allow_insecure: true on the dep in apm.yml
          2. Pass --allow-insecure to apm install
    

    And use the full URL (with scheme) in every error -- the user needs to see why it's insecure.

  8. [!] HIGH -- Update docs/src/content/docs/enterprise/security.md. This PR introduces an entirely new transport surface and threat model; per the repo rule ("If a code change weakens or contradicts any guarantee in security.md, the doc must be updated in the same PR"), security.md needs an "HTTP (insecure) dependencies" section covering: what allow_insecure does/doesn't protect against, MITM impact on content-hash provenance, credential-helper isolation requirement, the transitive-host boundary model.

  9. [!] HIGH -- Logging routing: dead else: _rich_*() fallback branches. _warn_insecure_dependencies, _guard_transitive_insecure_dependencies, _check_insecure_dependencies repeat if logger: logger.X() else: _rich_X(...) 7 times. Callers always have a logger; the else branches are dead code and bypass DiagnosticCollector if ever triggered. Make logger a required positional arg; drop every fallback branch.

Polish (same PR if cheap; follow-up otherwise)

  1. [i] Library code calls sys.exit(1). _check_insecure_dependencies and _guard_transitive_insecure_dependencies should raise an exception; CLI layer catches and exits. Improves testability and reuse.

  2. [i] _format_insecure_dependency_warning uses jargon ("no transport auth"). Reword:

    [!] Insecure HTTP fetch (unencrypted): http://...
    
  3. [i] Transitive-block error is a policy lecture. Drop the middle sentence; lead with the command. Move the "why" to docs.

  4. [i] apm deps list --insecure column label "HTTP" misleads (values are direct / via <parent>). Rename to Origin.

  5. [i] Unrelated cleanups (config.py refactor, _setup_git_environment SSH cleanup) belong in separate commits for bisectability.

Worth a follow-up issue (don't block this PR)

  • APM_ALLOW_INSECURE=1 env var counterpart for CI ergonomics (mirror APM_ALLOW_PROTOCOL_FALLBACK=1 pattern). Not adding this is fine for v1; tracking issue acceptable.
  • Drift-detection enhancement: warn on first-time HTTP install without lockfile SHA pin (defense-in-depth against the hash-circular-trust limitation).

Merge call

This is solid first-PR work on a security-sensitive feature -- the dual-opt-in design and host scoping are correct. The blockers are concentrated in three areas: credential leakage (#1), identity model (#2, #5), and architectural cleanup (#3, #4, #6). Items #1-#5 are the merge-blockers; #6-#9 should land in the same PR; the rest is polish or follow-up. Please address #1-#9 in this PR and re-request review.

Growth angle

  • Real adoption blocker unlocked: corporate Gitea/Gogs/Artifactory-on-HTTP and air-gapped enterprise users -- the segment that quietly worked around APM's "install from anywhere" claim with manual git clone. Worth a release-notes call-out.
  • First-time NONE-association contributor shipping 1338 lines + 30+ tests on a security feature is the contributor signal worth amplifying once this lands. Suggested release-notes line: "APM now supports private HTTP git hosts with explicit dual-gate opt-in (manifest + CLI). Insecure by default? Never. Insecure when you need it, with a clear audit trail." Repostable hook: "APM doesn't block your infra -- it makes insecure choices visible."
  • Once shipped, security.md becomes the authoritative reference for the threat model -- making the doc update in #8 above doubly load-bearing.

Reviewed via the apm-review-panel skill: python-architect, cli-logging-expert, devx-ux-expert, supply-chain-security-expert; arbitrated by apm-ceo with oss-growth-hacker side-channel.

@arika0093 arika0093 force-pushed the feat/allow-http-request branch from 1796653 to 803a126 Compare April 21, 2026 06:32
arika0093 and others added 4 commits April 21, 2026 07:51
…split)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… 14 stack split)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ck split)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tack split)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
arika0093 and others added 12 commits April 21, 2026 07:51
…em 2)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…item 3)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…em 4)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… item 5)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eview item 6)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…view item 9)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…m 10)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@arika0093 arika0093 force-pushed the feat/allow-http-request branch from b2a0a78 to b8a5c5f Compare April 21, 2026 08:00
@arika0093
Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I addressed all requested changes from this review, rewrote the stack so the commits are easier to map back to the review items, and kept the split granularity intact.

Review item mapping:

  1. HTTP git attempts now suppress credential helpers: b5ce27e
  2. HTTP/HTTPS transport flips now show up as drift: c67cfde
  3. Insecure policy logic was extracted out of CLI command code into install/insecure_policy.py: 9032b9e
  4. Lockfile replay now preserves allow_insecure for HTTP dependencies: 1fca10b
  5. to_canonical() is scheme-free again: caeb13d
  6. LockedDependency.to_dependency_ref() now centralizes the mapping: 85be6d1
  7. HTTP remediation errors were unified into one canonical recipe: f08a716
  8. docs/src/content/docs/enterprise/security.md was updated for the HTTP threat model: 17e40a3
  9. The insecure policy helpers now require a logger and no longer keep dead fallback branches: 32cb21f
  10. Library-level insecure policy helpers now raise exceptions instead of calling sys.exit(1): b067ab7
  11. The insecure HTTP warning wording was tightened: 2dd90af
  12. The transitive HTTP remediation error was shortened and now leads with the command: 783450f
  13. apm deps list --insecure now uses Origin instead of HTTP: b8a5c5f

For review item 14:

  • The unrelated commands/config.py refactor was split into its own commit for bisectability: 333e8f0
  • The _setup_git_environment() / GIT_SSH_COMMAND SSH cleanup mentioned in that item was not introduced by this PR stack. It predates this branch, so there was no PR-local change to split out for that part.

At this point, all review items from this review have been addressed in the branch.

danielmeppiel added a commit that referenced this pull request Apr 21, 2026
)

* feat(transport): TransportSelector + strict-by-default transport (#778)

Implements the core decision engine for issue #778 'transport selection v1'.
Strict-by-default semantics replace today's silent cross-protocol fallback:
explicit ssh:// and https:// dependencies no longer downgrade to a different
protocol, and shorthand (owner/repo) consults git insteadOf rewrites before
defaulting to HTTPS.

This commit ships Waves 1+2 of the transport-selection plan (per session plan):
- new module src/apm_cli/deps/transport_selection.py with ProtocolPreference,
  TransportAttempt/TransportPlan dataclasses, GitConfigInsteadOfResolver,
  and TransportSelector that returns a typed, strict-by-default plan
- DependencyReference grows explicit_scheme so the selector can distinguish
  user-stated transport from shorthand
- _clone_with_fallback in github_downloader.py now iterates the selector
  plan; per-attempt URL building stays in the orchestrator
- InstallContext / InstallRequest / pipeline / Service threaded with
  protocol_pref + allow_protocol_fallback so CLI args reach the downloader
- apm install gains --ssh / --https (mutually exclusive) and
  --allow-protocol-fallback flags; honours APM_GIT_PROTOCOL and
  APM_ALLOW_PROTOCOL_FALLBACK env vars
- two pre-existing tests in test_auth_scoping.py asserted the legacy
  permissive chain; updated to assert the new strict contract and added a
  coverage test for the allow_fallback escape hatch

Tests: 4029 unit tests pass. Test matrix + integration tests + docs land
in subsequent commits per Waves 3-5.

Refs #778, #328, #661

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(transport): wave-3 unit + integration matrix; fix per-attempt clone env

Wave 2 panel gate (code-review subagent) flagged that
GitHubPackageDownloader._clone_with_fallback decided the clone env
(locked-down vs relaxed) ONCE per dependency from has_token. Under
allow_fallback=True the plan can mix attempts of different use_token
values, so SSH and plain-HTTPS attempts in a mixed chain were running
with GIT_ASKPASS=echo + GIT_CONFIG_GLOBAL=/dev/null +
GIT_CONFIG_NOSYSTEM=1, breaking ssh-agent passphrase prompts and git
credential helpers.

Fix the env per attempt; address an adjacent contract bug; add tests.

* github_downloader._clone_with_fallback: replace the per-dep clone_env
  with a per-attempt _env_for() helper so only token-bearing attempts
  get the locked-down env.
* github_downloader._build_repo_url: treat token="" as an explicit "no
  token" sentinel so plain-HTTPS attempts in a mixed chain genuinely
  run without embedded credentials, letting credential helpers (gh
  auth, Keychain) supply auth. Orchestrator passes "" instead of None
  for use_token=False attempts.
* transport_selection.GitConfigInsteadOfResolver: wrap the lazy
  insteadOf-rewrite cache in a threading.Lock so parallel downloads
  can't double-populate.

Tests:
* tests/unit/test_transport_selection.py (NEW, 30 tests): 14-row
  selection matrix (explicit-strict, shorthand+insteadOf, shorthand
  defaults, CLI prefs, allow_fallback chain, env helpers); resolver
  caching; "must use normal env" contract.
* tests/unit/test_auth_scoping.py: new
  test_allow_fallback_env_is_per_attempt_not_per_dep regression
  asserts auth-HTTPS gets locked-down env, SSH and plain-HTTPS get
  relaxed env, and plain-HTTPS does not embed the token in the URL.
* tests/integration/test_transport_selection_integration.py (NEW, 7
  tests): 2 always-on cases (public shorthand HTTPS; explicit https://
  strict); 5 SSH-required cases (explicit ssh:// strict, bad-host
  no-fallback, insteadOf override, APM_GIT_PROTOCOL=ssh env,
  allow_fallback rescue). Gated on APM_RUN_INTEGRATION_TESTS=1; SSH
  cases auto-skip if no key.
* tests/fixtures/gitconfig_insteadof_to_ssh (NEW): minimal gitconfig
  used by the integration test for the insteadOf-honored case.
* scripts/test-integration.sh: added "Transport Selection" block so
  the integration suite runs in CI.

Full unit suite: 4061 passed (was 4029; +32 net new tests).

Refs #778, #661, #328

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(transport): document strict-by-default transport selection (#778)

Update the four user-facing surfaces affected by the new TransportSelector
contract:

* docs/src/content/docs/guides/dependencies.md:
  - New "Transport selection (SSH vs HTTPS)" section: breaking-change
    callout with the rescue env var, selection matrix, insteadOf
    example, --ssh / --https / APM_GIT_PROTOCOL overrides, and the
    --allow-protocol-fallback escape hatch.
  - Soften the existing "Custom ports preserved" sentence (cross-protocol
    retries are now opt-in).
  - Update the "Other Git Hosts" SSH bullet: SSH is no longer a silent
    fallback; point at explicit URLs or insteadOf.
* docs/src/content/docs/getting-started/authentication.md:
  - Rewrite the "SSH connection hangs" troubleshooting entry: remove the
    now-incorrect "tries SSH then falls back to HTTPS" framing.
  - New "Choosing transport (SSH vs HTTPS)" section with a pointer to
    dependencies.md for the full transport contract.
* docs/src/content/docs/reference/cli-commands.md:
  - Document --ssh / --https / --allow-protocol-fallback on apm install,
    plus APM_GIT_PROTOCOL and APM_ALLOW_PROTOCOL_FALLBACK env vars.
* packages/apm-guide/.apm/skills/apm-usage/dependencies.md (Rule 4 mirror):
  - Same transport contract in skill-resource voice with three runnable
    snippets and a selection matrix.

CHANGELOG: scope new entries to `apm install` only (there is no `apm
add` command in the codebase).

Refs #778

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: revert accidental uv.lock churn (no dependency change in this PR)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs+changelog: address copilot review feedback (#779)

Two findings from the Copilot reviewer on PR #779:

1. Non-ASCII em dash introduced by this PR in the modified `Fields:`
   line of guides/dependencies.md: replace with `--`. The other
   non-ASCII chars Copilot flagged in the file (lines ~150-160 of the
   "nested groups" warning block) are pre-existing and out of scope
   for this PR.
2. CHANGELOG entries for the new transport-selection feature were too
   long and bundled multiple concerns into one bullet. Split into one
   tighter BREAKING entry plus two single-purpose Added entries
   (initial-protocol flags; fallback escape hatch). Each ends in
   `(#778)`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #779 review feedback

- docs(dependencies): pin BREAKING-change caution panel to APM 0.8.13
  (per maintainer review comment)
- transport_selection: collapse explicit `http://` URLs into the plain-HTTPS
  branch so TransportAttempt.scheme stays in {ssh, https} and the downloader
  contract is consistent until #700 lands a first-class HTTP transport
- github_downloader: gate the [!] "Protocol fallback" warning on actual
  scheme change (ssh<->https) rather than label change, so an auth downgrade
  inside a single protocol is not misreported as a protocol switch
- pipeline / request / context / commands: switch
  `allow_protocol_fallback` to `Optional[bool] = None` end-to-end so
  programmatic callers (non-CLI) keep the documented "None => read
  APM_ALLOW_PROTOCOL_FALLBACK env" behavior
- test_auth_scoping: ASCII-only docstring (replace one stray Unicode arrow)

All 4061 unit tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(install): MARKETPLACE_PLUGIN beats HOOK_PACKAGE in detection cascade (#780)

obra/superpowers (and any Claude Code plugin shipping both hooks/*.json
and agents/skills/commands) was misclassified as HOOK_PACKAGE because
the cascade in detect_package_type() checked _has_hook_json BEFORE the
plugin-evidence branch. The plugin synthesizer (_map_plugin_artifacts)
already maps hooks alongside agents/skills/commands, so MARKETPLACE_PLUGIN
is a strict superset -- swapping the order means hooks still install,
plus everything else that was being silently dropped.

Three deliverables:

A) Surgical detection fix: reorder cascade so MARKETPLACE_PLUGIN is
   checked before HOOK_PACKAGE. Refactored to use a new
   gather_detection_evidence() helper + DetectionEvidence dataclass so
   observability code (warnings, summaries) can reuse the same scan
   without breaking the detect_package_type() public signature.

B) Observability:
   - Add HOOK_PACKAGE to the package-type label table (it was missing
     entirely -- the silent classification path).
   - Update MARKETPLACE_PLUGIN label to mention plugin.json OR
     agents/skills/commands (matches the cascade behaviour).
   - New CommandLogger.package_type_warn() at default visibility.
   - New _warn_if_classification_near_miss() helper fires when a
     HOOK_PACKAGE classification disagrees with directory contents
     (catches near-misses the order swap cannot, e.g. .claude-plugin/
     dir without plugin.json).
   - Wired at both materialization sites (local + cached).

C) Architectural follow-up tracked in plan; will file as separate
   issue after merge for a Visitor / Format-Discovery refactor.

Tests: 17 detection tests + 9 sources-observability tests + 70
command-logger tests pass. Full unit suite (4180 tests) green.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #781 review: broaden detect_package_type with .claude-plugin/ evidence

Per Copilot review on PR #781: the near-miss warning helper had dead
code paths once the cascade was reordered (a HOOK_PACKAGE classification
implies plugin_dirs_present is empty -- otherwise it would be
MARKETPLACE_PLUGIN). Two principled options were offered: simplify the
helper, or broaden detection so the invariants stay consistent. Chose
the latter -- it removes the inconsistency at the source.

Changes:
- Add .claude-plugin/ as first-class plugin evidence in DetectionEvidence
  (new has_claude_plugin_dir field) and in has_plugin_evidence. A Claude
  Code plugin without a plugin.json (manifest-less) now classifies as
  MARKETPLACE_PLUGIN; normalize_plugin_directory already handles the
  missing-manifest case (derives name from directory).
- Drop _warn_if_classification_near_miss helper, its two call sites,
  CommandLogger.package_type_warn method, and the corresponding tests.
  All scenarios it covered are now handled by the cascade itself.
- Add regression test test_claude_plugin_dir_alone_is_plugin_evidence
  asserting that .claude-plugin/ + hooks/ classifies as MARKETPLACE_PLUGIN
  with plugin_json_path=None (matched via directory evidence alone).
- Extend test_obra_superpowers_evidence to assert has_claude_plugin_dir.

Verified: 4175 unit tests pass (excluding the one pre-existing unrelated
failure documented on main).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(validation): reject shell-string command in MCP stdio entries

Self-defined stdio MCP entries with `command` containing whitespace and
no `args` are now rejected at parse time with a fix-it error pointing at
the canonical `command: <binary>, args: [<token>, ...]` shape.

Previously silently accepted; APM never split `command` on whitespace,
so the loose shape mis-executed downstream. The trap surfaced via #122
(thanks @lirantal) -- users coming from the universal `mcp.json` /
Claude Desktop / Cursor mental model wrote `command: "npx mcp-server-foo"`
and got confused when nothing worked.

Per maintainer steer ("move fast, breaking OK"): error in v1, not warn.
The loose shape was never specified; silent mis-execution is worse than
hard-fail with a clear fix-it. CHANGELOG entry under
Changed (BREAKING).

Closes #806
Refs #122

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(mcp): address PR #809 review panel (cred leak, args:[], type gate, edge cases)

Addresses the APM Review Panel verdict on PR #809.

Must-fix:
- M1: Redact raw command in error 'Got:' framing (cli-log BLOCKER /
  sec HIGH). Echoing 'Got: command={self.command!r}' leaked tokens
  like '--token=ghp_...' to stderr / CI scrollback. Now shows only
  the first token plus argument count. The structured 'Did you mean:'
  suggestion still surfaces user input verbatim because that is the
  copy-paste recovery path.
- M2: Use 'self.args is None' instead of 'not self.args' (arch
  IMPORTANT). Explicit 'args: []' is a deliberate 'no extra args'
  signal (e.g., paired with '/opt/My App/server') and must be
  accepted -- 'not []' incorrectly evaluated truthy and rejected
  legitimate input in a BREAKING change.

Should-fix:
- S1: Whitespace-only command produces a dedicated 'empty or
  whitespace-only' error instead of the degenerate fix-it
  'Did you mean: command: , args: []' (arch + devx IMPORTANT).
- S2: Type gate for non-str command (sec HIGH). YAML
  'command: ["npx", "-y", "x"]' previously bypassed the
  isinstance guard silently and crashed downstream in
  validate_path_segments with an unhandled AttributeError.
- S3: Document rule 4 in manifest-schema.md section 4.2.3 (devx
  IMPORTANT). Spec and code ship together.

Adds 4 regression tests covering each fix. Removes the stray space
before '?' in the fix-it suggestion (cli-log NIT 8).

Follow-ups (not in this PR, to be filed as issues):
- Redact 'command' in MCPDependency.__repr__ (sec MEDIUM, pre-existing)
- Forward MCPDependency validation errors from plugin parser to
  DiagnosticCollector (cli-log IMPORTANT)
- Multi-line Cargo-style error format (cli-log IMPORTANT / devx NIT)
- Shell-metachar warning for stdio command (sec LOW)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(mcp): address PR #809 follow-up tickets in same PR

Per user direction, fold all four CEO-classified follow-up items into
this PR rather than deferring to separate issues.

FU1 -- Redact 'command' in MCPDependency.__repr__ (sec MEDIUM)
  Pre-existing leak: __repr__ echoed 'command={self.command!r}' verbatim
  while carefully redacting env and headers. Now shows only the first
  whitespace-separated token, mirroring the M1 fix.

FU2 -- Surface plugin-parser MCP validation warnings (cli-log IMPORTANT)
  The 'apm' stdlib logger has no handlers configured, so logger.warning
  calls in plugin_parser were silently dropped. Added _surface_warning
  helper that routes through both stdlib logger AND _rich_warning so
  invalid MCP servers are visible without --verbose. Applied to the
  validation-error catch site and the no-command/no-url skip.

FU3 -- Multi-line Cargo-style error format (cli-log IMPORTANT / devx NIT)
  The original 350-char single-line ValueError defeated terminal URL
  detection and the newspaper test. Restructured to:
    'command' contains whitespace in MCP dependency '<name>'.
      Rule: ...
      Got:  command='<first>' (N additional args)
      Fix:  command: <first>
            args: [...]
      See:  https://...
  URL now sits on its own line for click-through; field/rule/got/fix/see
  pattern is scannable per the cli-logging-ux skill's newspaper test.

FU4 -- Shell-metachar warning for stdio command (sec LOW, defense-in-depth)
  Extended _warn_shell_metachars(env, logger) to optionally check
  'command' as well, so 'command: "npx|curl evil.com"' (no whitespace,
  passes the rejection guard) still triggers a warning that MCP stdio
  servers run via execve with no shell. Hooked into the --mcp install
  path via entry.get('command').

Architectural improvement (LOC budget):
  Adding the command-checking branch pushed install.py over the 1525
  invariant ceiling. Per the python-architecture skill's guidance
  ('don't trim cosmetically -- modularize'), extracted the F5 SSRF
  helper, F7 shell-metachar helper, _is_internal_or_metadata_host,
  _SHELL_METACHAR_TOKENS, and _METADATA_HOSTS into a new dedicated
  module: apm_cli/install/mcp_warnings.py. install.py back-binds the
  symbols at module scope so existing test patches against
  apm_cli.commands.install._warn_* keep working unchanged.

  install.py: 1530 -> 1441 LOC (84 under budget, room to breathe).

Tests: 4715/4715 unit + console pass (excludes the known pre-existing
test_user_scope_skips_workspace_runtimes failure on main).

New regression tests:
- test_validate_stdio_error_uses_multiline_cargo_style_format
- test_repr_redacts_command_to_avoid_leaking_credentials

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel and others added 2 commits April 21, 2026 20:33
The combined surface of PR microsoft#700 (allow-insecure CLI flags + policy) and
main's MCP install block (microsoft#810, microsoft#814) pushed commands/install.py over
the 1525 LOC invariant enforced by tests/unit/install/
test_architecture_invariants. Per the test's own guidance, resolved by
extracting -- not trimming.

Conflicts:
- CHANGELOG.md: unioned the PR microsoft#700 Added entry with main's MCP entries.
- packages/apm-guide/.apm/skills/apm-usage/commands.md: merged both flag
  lists into a single --allow-insecure + --mcp row.
- src/apm_cli/commands/install.py: unioned the install() signature and
  docstring examples; kept both the InsecureDependencyPolicyError and
  click.UsageError except branches.
- tests/unit/test_install_command.py: kept TestAllowInsecureFlag and
  TestInstallMcpFlag as sibling classes (each with its own setup/
  teardown).

Architecture (commands/install.py LOC): 1572 -> 1496.
- Extracted F5 SSRF + F7 shell-metachar helpers to new dedicated module
  src/apm_cli/install/mcp_warnings.py (same pattern used in PR microsoft#809).
- commands/install.py re-binds the extracted symbols at module scope
  (warn_ssrf_url -> _warn_ssrf_url, warn_shell_metachars ->
  _warn_shell_metachars, _is_internal_or_metadata_host,
  _SHELL_METACHAR_TOKENS, _METADATA_HOSTS) so existing test patches
  against apm_cli.commands.install._warn_* keep working unchanged.

Tests: 4755/4755 unit + console pass (excludes the known pre-existing
test_user_scope_skips_workspace_runtimes failure on main, unrelated to
this PR or this merge).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR microsoft#809 (shell-string MCP command validation) merged to main after the
previous merge commit on this branch. Re-merged to pick up the new tip.

Conflicts:
- src/apm_cli/install/mcp_warnings.py (add/add): both sides added this
  file. Took main's version -- it is the authoritative microsoft#809 copy and
  includes the FU4 extension (command= parameter on warn_shell_metachars)
  that my earlier manual extraction on this branch did not have.
- src/apm_cli/commands/install.py: kept main's re-bind import ordering
  since the two versions were trivially equivalent.

Tests: 4767/4767 unit + console pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel
Copy link
Copy Markdown
Collaborator

APM Review Panel — Round 2 Verdict (PR #700)

Re-review following @arika0093's commit-by-item response mapping. Panel: Python Architect, CLI Logging, DevX UX, Supply-Chain Security, OSS Growth. Five specialists, full convergence, no disagreements to arbitrate.

Verdict: APPROVE with minor nits. Ready to merge after one small fix.

All 14 items from the previous round are substantively addressed. Evidence verified commit-by-commit.


Items verified resolved

# Blocker Fix commit Verdict
1 Credential-helper leak on HTTP clone b5ce27eb Defence-in-depth: GIT_ASKPASS=echo, GIT_TERMINAL_PROMPT=0, GIT_CONFIG_NOSYSTEM=1, GIT_CONFIG_COUNT/KEY/VALUE triple. All 4 independent channels closed. Verified at github_downloader.py:505-534 + tests test_auth_scoping.py:277-326, 854-859.
2 HTTP/HTTPS identity drift c67cfded XOR is_insecure comparison at drift.py:100-105 catches both upgrade and downgrade. Tests cover both directions + no-drift case.
3 Inverted imports commands→install 9032b9ef install/insecure_policy.py (209 LOC) with correct import direction. Zero residual duplicates in commands/install.py. Layering clean.
4 allow_insecure dropped on lockfile replay 1fca10b9 drift.py:259-263 replays both is_insecure and allow_insecure from lockfile. Legacy lockfiles fail-closed (missing allow_insecure → False → re-prompt).
5 to_canonical() scheme-leak caeb13d9 Scheme-free invariant restored. All consumers audited: display/identity paths scheme-free; only to_apm_yml_entry() keeps scheme where needed.
6 Missing to_dependency_ref() factory 85be6d14 Single source of truth at lockfile.py:204-218. All 3 call sites migrated. Zero hand-rolled DependencyReference( reconstructions outside the factory.
7 Unified error recipe Canonical _format_insecure_dependency_requirements() used across all three paths. See nit N1 below.
8 HTTP threat-model in security.md 17e40a3b New section (L64-82) covers allow_insecure semantics, MITM exposure, content-hash provenance limitation, credential-helper isolation, transitive host model. Sales-enablement quality.
9 Dead _rich_* fallback branches insecure_policy.py has zero _rich_* imports/calls; all three policy functions call logger.warning/error bare (no if logger: guard, no else: fallback).
10 sys.exit(1) in library code b067ab7a insecure_policy.py raises InsecureDependencyPolicyError(RuntimeError); CLI layer at install.py:1329, 1419 owns the exit. Library-level is pure.
11 Jargon in warning copy Warning now reads Insecure HTTP fetch (unencrypted). See nit N1 — sibling jargon persists in error recipe.
12 Transitive-block error insecure_policy.py:172-178 leads with Re-run with --allow-insecure-host HOST. One-step fix, copy-pasteable.
13 apm deps list --insecure column Renamed to "Origin" in Rich and text-fallback tables. Values direct / via <parent>. Docs match.
14 Commit hygiene / bisectability 20 commits, one-per-review-item, descriptive (review item N) tags. 333e8f00 config.py refactor cleanly isolated.

Architecture invariant (install.py <= 1525 LOC) preserved: 1496 LOC after the merge-commit extraction of mcp_warnings.py. Full unit suite 4767/4767 passing.


Nits (should-fix; not merge-blockers)

N1 — Double URL in validation-time path (CLI Logging, one-line fix)
When apm install <http-url> fails the insecure check, the reason text already starts with {url} -- but is passed to logger.validation_fail(package, reason) which prepends {package} -- . Result: [x] http://server/owner/repo -- http://server/owner/repo -- HTTP dependency.... Fix: call logger.error(reason) directly on the insecure-reject path at install.py:292-297, matching the other two failure paths. One-line change.

N2 — Wording inconsistency between warning and error copy (CLI Logging)
Warning says (unencrypted) (good, post-#11). Error recipe at insecure_policy.py:67 still says (no transport encryption). Unify to (unencrypted) for consistent register across both surfaces.

N3 — Context-aware hint when only one gate is missing (DevX)
_check_insecure_dependencies emits the full two-step recipe even when allow_insecure: true is already set on the dep in apm.yml. Users who already edited their manifest see "1. Set allow_insecure: true" redundantly. Conditionally emit only the missing step. Cosmetic; track as fast-follow.

N4 — _build_noninteractive_git_env intent comment (Security)
The env.pop("GIT_ASKPASS", None) at github_downloader.py:514 unconditionally removes the APM default, then conditionally restores it only when suppress_credential_helpers=True. The intent (allow user keychains for HTTPS/SSH fallback, block them for HTTP) is correct but fragile. Add a short comment to prevent a future refactor from inverting the logic. No behaviour change.

N5 — --insecure sample output missing from cli-commands.md (DevX)
The apm deps list reference doc shows a sample output block for default mode but none for --insecure. Add a second block with the "Origin" column to help users preview the output without installing an HTTP dep.


Tracked follow-ups (out of scope; open as issues)

  • F1 — Advisory SHA-pin for HTTP initial fetch (SRI-like expected-sha: in apm.yml) to break the first-fetch circular-trust loop documented in security.md:81. Hardening item.
  • F2is_insecure field not yet documented in lockfile-spec.md schema table. Doc-writer owns this.
  • F3packages/apm-guide/.apm/skills/apm-usage/dependencies.md doesn't mention allow_insecure or the transitive policy (commands.md does). Makes LLM-agent consumers slightly less informed on the dependencies.md page.
  • F4 — Pre-existing back-references in install/presentation/dry_run.py:15 and install/phases/finalize.py:24 violate the strict "no install/commands/" layering. Inherited from refactor(install): modularize install.py into engine package #764, not this PR's regression. Worth a small follow-up extraction.
  • F5 — Host-only changes without a scheme flip still bypass drift detection (documented limitation at drift.py:41-44). Not a regression; worth an issue.

Growth / positioning (side-channel)

PR #700 converts the README's "Install from anywhere" bullet from aspirational into defensible: corporate Gitea / Gogs / Artifactory / air-gapped Artifactory users can now install. The enterprise/security.md threat-model section is a sales-enablement asset that answers the enterprise-review question before it's asked. The apm deps list --insecure audit surface compounds that.

Contributor-velocity narrative: @arika0093 (first-time external contributor, NONE-association) shipped a security-sensitive 1,300+ LOC feature with 204 new test functions, through two panel reviews, one atomic commit per review item. This is the contributor-funnel story APM should amplify in the release post — it sets the quality bar for future complex contributions.

Recommended release-notes hook:

APM now installs from HTTP mirrors — and makes every insecure choice visible. Corporate Gitea, Gogs, and air-gapped Artifactory users can apm install --allow-insecure with per-host transitive controls, credential-helper suppression, and apm deps list --insecure auditability.

README "Install from anywhere" bullet should be updated in a follow-up to mention HTTP mirrors with explicit opt-in.


Conflict resolution

Merged cleanly against main across two rounds (PR #809 landed mid-review):

Full suite green on HEAD. No functional changes from these merge commits — pure conflict resolution.


Final call

Merge-ready. Single actionable ask before merge: fix N1 (one-line: swap logger.validation_faillogger.error on the insecure-reject path). N2–N5 can ship as a tiny follow-up. F1–F5 → open as issues.

Outstanding work by @arika0093. Thank you for seeing this through two review rounds with the care each commit shows.


Panel: python-architect, cli-logging-expert, devx-ux-expert, supply-chain-security-expert, oss-growth-hacker. Synthesis via apm-review-panel skill. Conflicts resolved in c8bcc700 + 35d64631.

@danielmeppiel danielmeppiel self-requested a review April 21, 2026 18:47
Comment thread tests/unit/test_install_command.py Fixed
Round-2 apm-review-panel findings on PR microsoft#700. Five specialists converged
on APPROVE with minor nits; this commit applies the actionable ones.

N1 (cli-logging): fix double-URL rendering in the validation-time path.
`_format_insecure_dependency_requirements` already embeds the full URL in
the reason string, but `logger.validation_fail(package, reason)` prepends
"{package} -- " and produced output like
"[x] http://host/repo -- http://host/repo -- HTTP dependency...". Switch
to `logger.error(reason)` on the insecure-reject branch to match the
other two failure paths. `invalid_outcomes` still tracks the package so
the later validation summary sees it.

N2 (cli-logging): unify jargon register. Warning copy already says
"(unencrypted)" post-review microsoft#11; align the error recipe from
"(no transport encryption)" to "(unencrypted)" so both surfaces use the
same wording.

N3 (devx): context-aware remediation steps. `_check_insecure_dependencies`
previously emitted the full two-step recipe in both failure branches,
even when the user had already performed step 1 (manifest edit). The
factory now takes `missing_dep_allow` and `missing_cli_flag` keyword
args and only renders the missing step(s). The add-time validation
path keeps the default of both-steps since the dep is not yet in
apm.yml. Tests updated to match the new per-branch assertions.

N4 (security): document the intentional two-stage credential-helper
policy in `_build_noninteractive_git_env`. The pop-then-conditionally-
restore of GIT_ASKPASS is correct but fragile; an explicit docstring
prevents a future refactor from inverting the logic and leaking
credentials over plaintext HTTP or blocking system keychains on
HTTPS/SSH fallback.

N5 (doc-writer): add `apm deps list --insecure` sample output to
cli-commands.md showing the bold-red `Origin` column with `direct`
and `via <parent>` example rows.

F2 (doc-writer): document `is_insecure` and `allow_insecure` fields in
lockfile-spec.md section 4.2, including replay semantics and legacy
fail-closed behaviour.

F3 (doc-writer): add "HTTP dependencies (opt-in)" section to the
apm-guide dependencies.md skill resource so LLM agents consuming the
skill have the dual-opt-in model, example manifest, example CLI
invocation, and cross-references to commands.md and the enterprise
security guide.

Full unit suite: 4767/4767 passing (one deselect unrelated).
`install.py` at 1497 LOC (under the 1525-LOC invariant).

Panel verdict: microsoft#700 (comment 4290979672)

Co-authored-by: arika0093 <arika0093@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel
Copy link
Copy Markdown
Collaborator

Follow-up on the panel verdict: I've pushed 2c7e81bf addressing N1-N5, F2, and F3 directly on top of your branch so the PR stays a single-author squash-merge under your name. Co-authorship trailers preserve the full attribution.

Nits addressed

  • N1 -- Fixed the double-URL render. install.py now calls logger.error(reason) on the insecure-reject path instead of logger.validation_fail (which prepends the package name, producing pkg -- url -- HTTP dependency...).
  • N2 -- Unified the jargon register: error recipe now says (unencrypted) to match the warning copy.
  • N3 -- _format_insecure_dependency_requirements now takes missing_dep_allow / missing_cli_flag kwargs and only emits the steps the user actually needs. If allow_insecure: true is already on the dep entry, the error only asks for --allow-insecure (and vice versa). The CLI add-time path keeps both steps by default. Unit tests updated to assert the per-branch wording.
  • N4 -- Added an explicit docstring to _build_noninteractive_git_env documenting the intentional two-stage credential-helper policy (pop-then-conditionally-restore of GIT_ASKPASS), preventing a future refactor from inverting the logic.

Follow-ups addressed (doc-writer agent)

  • N5 -- Added an apm deps list --insecure sample output block to docs/.../reference/cli-commands.md showing the Origin column.
  • F2 -- Documented is_insecure and allow_insecure in docs/.../reference/lockfile-spec.md with replay semantics and legacy fail-closed behaviour.
  • F3 -- Added an "HTTP dependencies (opt-in)" section to packages/apm-guide/.apm/skills/apm-usage/dependencies.md so LLM agents consuming the skill know the dual opt-in model.

Validation

  • Full unit suite: 4767/4767 passing (one pre-existing deselect unrelated to this PR).
  • install.py at 1497 LOC -- under the 1525-LOC architecture invariant.

Remaining follow-ups (F1 advisory SHA-pin, F4 install/ -> commands/ inherited back-references, F5 host-only drift) are out of scope and should be filed as separate issues. Ready for maintainer merge whenever CI gates clear.

Thank you for the iteration quality -- the one-commit-per-review-item hygiene made this second round fast and high-confidence.

CodeQL flagged `"http://gitlab.company.internal" in arg` at
test_install_command.py:963 as a potentially unsafe URL substring check:
the hostname could appear at an arbitrary position in the checked URL
(e.g. inside a path segment, query value, or userinfo) rather than as
the actual host, producing a false-positive match.

Replace substring checks with `urllib.parse.urlsplit` and compare
`scheme` and `netloc` explicitly:

- test_explicit_http_generic_host_tries_http_first (the flagged case):
  parse each subprocess-arg URL and require scheme == "http" AND
  netloc == "gitlab.company.internal".
- test_generic_host_falls_back_to_https_when_ssh_fails (sibling with
  the same pattern at line 929): same treatment for the HTTPS arg;
  keep the SSH SCP-form check as `"git@git.example.org:" in arg` since
  SCP-style URLs are not parseable by urlsplit.

Both tests still assert the same contract: explicit `http://` does not
fall back to SSH, and SSH failure falls back to HTTPS on the right host.

Co-authored-by: arika0093 <arika0093@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel merged commit 5f9b5e4 into microsoft:main Apr 21, 2026
10 checks passed
danielmeppiel added a commit that referenced this pull request Apr 21, 2026
- Bump version to 0.9.0 in pyproject.toml and uv.lock
- Move CHANGELOG [Unreleased] entries into [0.9.0] - 2026-04-21
- Add fresh empty [Unreleased] header
- Consolidate the two scattered ### Changed subsections into one
- Backfill missing entries: build-time self-update policy (#675),
  APM Review Panel skill instrumentation (#777)
- Backfill PR refs on a few entries that referenced issue numbers
- Promote the previously-orphaned 'apm install --mcp' / VS Code adapter /
  init Next Steps lines to consistent (#PR) attribution
- Remove stray blank line inside the ### Added block
- Credit external contributors inline (@arika0093 #700, @joostsijm #675,
  @edenfunf #788)

Highlights of 0.9.0:

BREAKING CHANGES
- MCP entry validation hardened (#807)
- Strict-by-default transport selection (#778)
- Whitespace stdio MCP commands now rejected at parse time (#809)

ADDED
- apm install --mcp for declarative MCP server addition (#810)
- --registry flag and MCP_REGISTRY_URL for custom MCP registries (#810)
- HTTP-dependency support via --allow-insecure dual opt-in (#700)
- apm install --ssh / --https flags for transport selection (#778)
- Multi-target apm.yml + --target flag (#628)
- Marketplace UX: view, outdated, validate (#514)
- Build-time self-update policy for package-manager distros (#675)
- APM Review Panel + 4 specialist personas (#777)

This PR is release machinery and is intentionally excluded from the
[0.9.0] section per .github/instructions/changelog.instructions.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
danielmeppiel added a commit that referenced this pull request Apr 21, 2026
The 0.9.0 section had 30 bullets where 9 PRs were over-split:
#810 (5 bullets), #514 (5), #700 (2), #778 (3), #638 (2), #808 (paragraph
listing every drift fix), plus one orphan ### Changed block holding a
single sub-point of the #778 BREAKING entry.

Per .github/instructions/changelog.instructions.md: 'One line per PR:
concise description ending with (#PR_NUMBER). Combine related PRs into a
single line when they form one logical change.'

Result: 23 entries, one per PR, terse imperative summaries with the
user-facing impact up front. Migration / 'Closes' / 'Previously...' prose
moved off the changelog (it belongs in the PR body and CHANGELOG-linked
docs). Section is now scannable as headlines instead of essays.

No code changes; v0.9.0 tag already published on prior commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@arika0093 arika0093 deleted the feat/allow-http-request branch April 21, 2026 23:36
danielmeppiel pushed a commit that referenced this pull request Apr 30, 2026
Leftover implementation plan from PR #700 architectural fix work; the
content lived only as a developer scratchpad and was never wired into
docs, CI, or runtime. Strip the few stale 'plan.md section X' comment
references in policy_target_check phase + tests at the same time.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Support installation from git repositories hosted over HTTP only

5 participants