Skip to content

storage: wait for tar-split completion to avoid reader race#801

Open
jankaluza wants to merge 1 commit intocontainers:mainfrom
jankaluza:tar
Open

storage: wait for tar-split completion to avoid reader race#801
jankaluza wants to merge 1 commit intocontainers:mainfrom
jankaluza:tar

Conversation

@jankaluza
Copy link
Copy Markdown
Member

Switch to NewInputTarStreamWithDone and explicitly wait for the tar-split goroutine to finish before returning.

This prevents races with uncompressed.Close() while tar-split is still reading.

Fixes: #148

@github-actions github-actions Bot added storage Related to "storage" package common Related to "common" package image Related to "image" package labels Apr 29, 2026
@packit-as-a-service
Copy link
Copy Markdown

Packit jobs failed. @containers/packit-build please check.

@jankaluza jankaluza force-pushed the tar branch 2 times, most recently from cc2e066 to ccf4fb2 Compare April 29, 2026 09:59
Copy link
Copy Markdown
Member

@giuseppe giuseppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

restarted the failed test, the failure seems unrelated

@jankaluza
Copy link
Copy Markdown
Member Author

@jankaluza
Copy link
Copy Markdown
Member Author

@mtrmac , your review would be appreciated, since you know the context of this change and you also reviewed the tar-split part of this change.

Copy link
Copy Markdown
Contributor

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread storage/pkg/chunked/compressor/compressor.go Outdated
Comment thread storage/pkg/chunked/compressor/compressor.go Outdated
Comment thread storage/layers.go Outdated
Comment thread storage/layers.go Outdated
Comment thread storage/layers.go Outdated
Switch to `NewInputTarStreamWithDone` and explicitly wait for
the tar-split goroutine to finish before returning.

This prevents races with `uncompressed.Close()` while tar-split is still
reading.

Fixes: containers#148

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
Copy link
Copy Markdown
Contributor

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise.

Please rebase, after #809 that should fix the failing test.

_, err = io.Copy(io.Discard, tsReader)
require.NoError(t, err)
require.NoError(t, <-done)
require.NoError(t, tsReader.Close())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the generally correct order is close (to definitely abort the goroutine), <-done (to wait for it). Also in the other test.

Comment thread storage/pkg/chunked/compressor/compressor.go
@TomSweeneyRedHat
Copy link
Copy Markdown
Member

LGTM
after the rebase and @mtrmac concerns are addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common Related to "common" package image Related to "image" package storage Related to "storage" package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Race condition in applyDiffWithOptions

4 participants