Skip to content

perf: apm deps update re-downloads packages even when SHA is unchanged #496

@danielmeppiel

Description

@danielmeppiel

Problem

apm deps update delegates to _install_apm_dependencies(update_refs=True), which correctly re-resolves git refs to their latest commit SHAs (API call). However, when the resolved SHA matches the existing lockfile SHA, the engine still:

  1. Re-clones/fetches the package (unnecessary network + I/O)
  2. Re-integrates all primitives (unnecessary file writes)
  3. Re-computes content hashes (unnecessary CPU)

This happens because update_refs=True bypasses the lockfile entirely at three choke points in install.py:

  • Line 1154: if lockfile_path.exists() and not update_refs: -- skips loading lockfile
  • Line 1725: if ... and not update_refs and not ref_changed: -- skips SHA match check
  • Line 1736: (is_cacheable and not update_refs) -- skips cache hit

Expected behavior

After resolving refs (API call -- always necessary), compare the resolved SHA with the lockfile SHA. If they match, skip download and integration. Only proceed with download + integration when the SHA has actually changed.

Proposed approach

Separate "resolution" from "download" in the update_refs path:

  1. Load the lockfile even when update_refs=True (but only for SHA comparison, not to skip resolution)
  2. After ref resolution, compare resolved SHA vs lockfile resolved_commit
  3. If match: skip download, mark as cached, skip integration (or make integration content-aware)
  4. If different: download, integrate, update lockfile as today

This would also benefit apm install when re-running on an already-installed project.

Impact

For projects with many dependencies that are already at their latest refs, apm deps update currently re-downloads and re-integrates everything. With this optimization, it would only make lightweight API calls to verify SHAs, making the common case (nothing changed) near-instant.

Context

Discovered during E2E testing of PR #493. The update command works correctly -- this is purely a performance optimization.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions