Skip to content

Implement HTTP Range requests for resumable model downloads#67

Draft
Copilot wants to merge 3 commits intoadd-resumable-downloadsfrom
copilot/integrate-http-range-requests
Draft

Implement HTTP Range requests for resumable model downloads#67
Copilot wants to merge 3 commits intoadd-resumable-downloadsfrom
copilot/integrate-http-range-requests

Conversation

Copy link

Copilot AI commented Nov 14, 2025

Interrupted model pulls now resume from the last downloaded byte using HTTP Range requests. When ./model-cli pull ai/smollm2:135M-Q4_0 is interrupted and re-run, it continues from where it left off instead of restarting.

Implementation

Resumable layer wrapper (pkg/distribution/distribution/resumable.go)

  • Wraps v1.Image and v1.Layer to intercept layer downloads
  • Detects existing .incomplete files and their byte offsets
  • Issues HTTP requests with Range: bytes=offset- header
  • Falls back to full download on HTTP 416/200 responses

Distribution client (pkg/distribution/distribution/client.go)

  • Stores HTTP transport for Range request authentication
  • Wraps remote models with resumableImage before store write
  • Parses references to construct registry blob URLs

Store layer (pkg/distribution/internal/store/blobs.go)

  • Opens .incomplete files in append mode when detected
  • Writes resumed data directly to existing partial files

Behavior

First attempt (interrupted):
  GET /v2/repo/blobs/sha256:abc... → 50MB downloaded → ^C

Second attempt (resumed):
  GET /v2/repo/blobs/sha256:abc...
  Range: bytes=52428800-         → remaining 450MB downloaded
  HTTP 206 Partial Content

Gracefully degrades to full download when Range requests unsupported.

Original prompt

Integrate HTTP Range requests for resumable downloads

There's part of the reference implementation in:

the reference implementation in @distribution/pull_v2.go

When someone executes:

./model-cli pull ai/smollm2:135M-Q4_0

And that pull gets interrupted. If we pull again, it should resume where we left off.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits November 14, 2025 10:57
- Add resumableImage and resumableLayer wrappers to support HTTP Range requests
- Modify WriteBlob to append to incomplete files when resuming
- Wrap remote models with resumable support before writing to store
- Support both HTTP 206 (Partial Content) and HTTP 200 (fallback) responses
- Automatically fall back to full download if Range request fails

Co-authored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com>
Co-authored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com>
Copilot AI changed the title [WIP] Integrate HTTP range requests for resumable downloads Implement HTTP Range requests for resumable model downloads Nov 14, 2025
Copilot AI requested a review from ericcurtin November 14, 2025 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants