transport: add parallel transport and improve resumable transport

[xenoscopic]

This PR implements an auto-parallelizing transport to support model pulls. It doesn't wire anything up within this repository - that will happen downstream. This PR also refactors the existing resumable transport and its tests so that code is shared across the resumable and parallel implementations.

The parallel downloader currently uses temporary files on disk to support chunks in a way that's transparent to the http.RoundTripper interface (see transport/internal/bufferfile/fifo.go).

Most of this PR was written by AI, but it has been human-supervised, human-reviewed, and human-tested.

[Copilot]

Pull Request Overview

This PR implements a parallel HTTP transport for improved model pull performance, alongside enhancements to the existing resumable transport. Key changes include:

• New parallel transport implementation that automatically splits large downloads into concurrent byte-range requests
• Refactored resumable transport with improved error handling and code organization
• Shared common utilities extracted to reduce code duplication
• Comprehensive test coverage using a shared testing framework

Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
transport/parallel/transport.go	Core parallel transport implementation with concurrent byte-range downloading
transport/resumable/transport.go	Refactored resumable transport with improved error handling and shared utilities
transport/internal/common/http_utils.go	Shared HTTP utilities for range parsing and header handling
transport/internal/testing/fake_transport.go	Common test framework for transport testing
transport/internal/bufferfile/fifo.go	FIFO buffer implementation for temporary file-based streaming
tools/benchmarks/parallelget/main.go	Command-line benchmark tool for comparing sequential vs parallel downloads

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[xenoscopic]

Performance comparison on Docker Offload:

# ./pglinux 'https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/57/57d4442d2a611cf9d54d118361ce29c3ae4db4a36d9e82328b706f6cb3604d2c/data
?<redacted>'
Benchmarking HTTP GET performance for: https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/57/57d4442d2a611cf9d54d118361ce29c3ae4db4a36d9e82328b706f6cb3604d2c/data?<redacted>
Configuration: chunk-size=1048576 bytes, max-concurrent=4

Running non-parallel benchmark...
  Progress: [██████████████████████████████] 100.0% (4661213792/4661213792 bytes)
✓ Non-parallel: 4661213792 bytes in 2m26.335300079s (30.38 MB/s)
Running parallel benchmark...
  Progress: [██████████████████████████████] 100.0% (4661213792/4661213792 bytes)
✓ Parallel: 4661213792 bytes in 1m28.94439037s (49.98 MB/s)
Validating response consistency...
✓ Responses match perfectly

============================================================
PERFORMANCE COMPARISON
============================================================
🚀 Parallel was 1.65x faster than non-parallel
⏱️  Time saved: 57.390909709s (39.2%)

Detailed timing:
  Non-parallel: 2m26.335300079s
  Parallel:     1m28.94439037s
  Difference:   -57.390909709s

Performance comparison on Starlink:

$ ./parallelget 'https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/83/8334b850b7bd46238c16b0c550df2138f0889bf433809008cc17a8b05761863e/data?<redacted>'
Benchmarking HTTP GET performance for: https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/83/8334b850b7bd46238c16b0c550df2138f0889bf433809008cc17a8b05761863e/data?<redacted>
Configuration: chunk-size=1048576 bytes, max-concurrent=4

Running non-parallel benchmark...
  Progress: [██████████████████████████████] 100.0% (1915305312/1915305312 bytes)
✓ Non-parallel: 1915305312 bytes in 55.236737542s (33.07 MB/s)
Running parallel benchmark...
  Progress: [██████████████████████████████] 100.0% (1915305312/1915305312 bytes)
✓ Parallel: 1915305312 bytes in 54.093693708s (33.77 MB/s)
Validating response consistency...
✓ Responses match perfectly

============================================================
PERFORMANCE COMPARISON
============================================================
🚀 Parallel was 1.02x faster than non-parallel
⏱️  Time saved: 1.143043834s (2.1%)

Detailed timing:
  Non-parallel: 55.236737542s
  Parallel:     54.093693708s
  Difference:   -1.143043834s

[ilopezluna]

I have tested the branch and verified that parallel transport works 👏👏.
I don’t have enough knowledge to give a thorough review, but I approve to unblock merging and to confirm that the parts I tested work as expected.

[xenoscopic]

I'm going to merge this for now so that we can start pushing on the monorepo migration. This change won't bring in any new behavior - the parallel transport isn't wired up to anything by default. We will inherit some refinements and refactors to the existing resumable transport, but we have good test coverage for those.