Skip to content

fix(libstore/filetransfer): restart source before upload retries#15047

Merged
Ericson2314 merged 1 commit into
NixOS:masterfrom
lovesegfault:fix-15023
Jan 22, 2026
Merged

fix(libstore/filetransfer): restart source before upload retries#15047
Ericson2314 merged 1 commit into
NixOS:masterfrom
lovesegfault:fix-15023

Conversation

@lovesegfault
Copy link
Copy Markdown
Member

Motivation

When an upload fails with a transient HTTP error (e.g., S3 rate limiting
with HTTP 503), retries would fail with "curl error: Failed to open/read
local data from file/application" because the upload source was already
exhausted from the previous attempt.

Restart the source in init() to ensure it's at the beginning for both
first attempts (no-op) and retries (necessary fix).

Context

Fixes: #15023


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

Comment thread doc/manual/rl-next/fix-upload-retry.md Outdated
@edolstra edolstra added the backport 2.33-maintenance Automatically creates a PR against the branch label Jan 22, 2026
When an upload fails with a transient HTTP error (e.g., S3 rate limiting
with HTTP 503), retries would fail with "curl error: Failed to open/read
local data from file/application" because the upload source was already
exhausted from the previous attempt.

Restart the source in init() to ensure it's at the beginning for both
first attempts (no-op) and retries (necessary fix).

Fixes: NixOS#15023
@Ericson2314 Ericson2314 added this pull request to the merge queue Jan 22, 2026
Merged via the queue into NixOS:master with commit 73a727f Jan 22, 2026
14 checks passed
@internal-nix-ci
Copy link
Copy Markdown

Successfully created backport PR for 2.33-maintenance:

@lovesegfault lovesegfault deleted the fix-15023 branch January 26, 2026 17:13
philiptaron added a commit to philiptaron/nixpkgs that referenced this pull request Feb 7, 2026
## Bug fixes

- Fix destruction of DerivationBuilder implementations (NixOS/nix#15072)
- Don't report cancelled goals as failures (NixOS/nix#14972)
- Fix `linux` build on fresh `glibc` and `gcc` (NixOS/nix#15011)

## S3 binary cache improvements

- Add AWS SSO support for S3 authentication (NixOS/nix#14645)
- Respect `AWS_PROFILE` environment variable (NixOS/nix#14645)
- Add STS support for default profile (NixOS/nix#14645)
- Skip `Accept-Encoding` header for S3 SigV4 requests (NixOS/nix#15048)
- Restart source before upload retries (NixOS/nix#15047)
- Route AWS CRT logs through Nix logger (NixOS/nix#15059)

The glibc 2.42 build fix patch is dropped as it is now included upstream.

https://github.com/NixOS/nix/releases/tag/2.33.2
brittonr pushed a commit to brittonr/nix that referenced this pull request Apr 1, 2026
fix(libstore/filetransfer): restart source before upload retries
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.33-maintenance Automatically creates a PR against the branch documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nix 2.33 nix copy to S3 suddenly rate limited

3 participants