[v0.25 backport] fix cache export error handling#6267
Closed
thaJeztah wants to merge 2 commits into
Closed
Conversation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com> (cherry picked from commit 99c6887) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't fail whole cache export on subbranch error. This behavior changed in v0.25, but while before error was not returned, the cache chains were either too agressively dropped or the whole exported cache chain got corrupted. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com> (cherry picked from commit 77aa49d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes cache export error handling for the case where a blob that is part of the imported cache, but doesn't actually participate in the current build has been removed in the cache source, and fails when in it transferred to new cache on export.
Two commits. Technically, either of them solves the issue of the build failing. First commit doesn't drop any cache metadata but is specific to the error caused by the blob being removed. The second one works on any export error in the cache chain, but causes the branch that the error affects to be dropped (that may cause the parents to be dropped if there are no valid input combinations left).
This is a regression in v0.25 from cache refactor. The reason it worked before is this nil return https://github.com/moby/buildkit/blob/v0.24.0/solver/exporter.go#L201 from subchain export error. Note that there was no similar return for "subexporters". There is also "exist" check in
marshalRemotein both versions, but in this specific case it is not reached. While v0.24 there was no error, when this case was hit thereturn nillikely corrupted the exported cache chains (at least that is what happens in my reproducer). This PR seems to work correctly, blob is skipped but the rest of the updated cache is still perfectly valid.