Skip to content

cache: write blob to the content store via bufio during conversion#2601

Merged
tonistiigi merged 1 commit into
moby:masterfrom
ktock:conversion-bufio
Feb 8, 2022
Merged

cache: write blob to the content store via bufio during conversion#2601
tonistiigi merged 1 commit into
moby:masterfrom
ktock:conversion-bufio

Conversation

@ktock
Copy link
Copy Markdown
Collaborator

@ktock ktock commented Feb 7, 2022 •
edited
Loading

Writing blobs to content store should be wrapped with bufio to avoid performance regression on containerd worker.

latency of exporting layers

This converts a zstd image to gzip.

Dockerfile

FROM ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd

command

buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/ctx2 --local dockerfile=/tmp/ctx2 \
               --output type=oci,dest=/tmp/out.tar,oci-mediatypes=true,compression=gzip,force-compression=true

containerd worker

branch time to export layers
master 69.5s
PR 12.2s
full log (master) 
# buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/ctx2 --local dockerfile=/tmp/ctx2 \
               --output type=oci,dest=/tmp/out.tar,oci-mediatypes=true,compression=gzip,force-compression=true
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 96B done
#1 DONE 0.2s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.2s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 ...

#4 [auth] ktock/tomcat:pull token for ghcr.io
#4 DONE 0.0s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 DONE 1.7s

#5 [1/1] FROM ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74 6.1s done
#5 DONE 0.6s

#6 exporting to oci image format
#6 exporting layers
#6 exporting layers 69.5s done
#6 exporting manifest sha256:3625a6de03f11ab53479975bd056c2674f37bb6c20aa5c4147b6644bfc7efb04 0.0s done
#6 exporting config sha256:30d6e2ab67c2843a7d2989d98172538bd34cce799daccfc45912f4005dcc5a93 0.0s done
#6 sending tarball
#6 sending tarball 6.7s done
#6 DONE 76.3s
full log (PR) 
# buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/ctx2 --local dockerfile=/tmp/ctx2 \
               --output type=oci,dest=/tmp/out.tar,oci-mediatypes=true,compression=gzip,force-compression=true
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 96B done
#1 DONE 0.2s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.2s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 ...

#4 [auth] ktock/tomcat:pull token for ghcr.io
#4 DONE 0.0s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 DONE 1.6s

#5 [1/1] FROM ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74 4.8s done
#5 DONE 0.6s

#6 exporting to oci image format
#6 exporting layers
#6 exporting layers 12.2s done
#6 exporting manifest sha256:3625a6de03f11ab53479975bd056c2674f37bb6c20aa5c4147b6644bfc7efb04 0.0s done
#6 exporting config sha256:30d6e2ab67c2843a7d2989d98172538bd34cce799daccfc45912f4005dcc5a93 0.0s done
#6 sending tarball
#6 sending tarball 5.5s done
#6 DONE 17.8s

OCI worker

branch time to export layers
master 14.9s
PR 12.0s
full log (master) 
# buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/ctx2 --local dockerfile=/tmp/ctx2 \
               --output type=oci,dest=/tmp/out.tar,oci-mediatypes=true,compression=gzip,force-compression=true
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 96B done
#1 DONE 0.2s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.2s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 ...

#4 [auth] ktock/tomcat:pull token for ghcr.io
#4 DONE 0.0s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 DONE 1.8s

#5 [1/1] FROM ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74 8.8s done
#5 DONE 0.6s

#6 exporting to oci image format
#6 exporting layers
#6 exporting layers 14.9s done
#6 exporting manifest sha256:3625a6de03f11ab53479975bd056c2674f37bb6c20aa5c4147b6644bfc7efb04 0.0s done
#6 exporting config sha256:30d6e2ab67c2843a7d2989d98172538bd34cce799daccfc45912f4005dcc5a93 0.0s done
#6 sending tarball
#6 sending tarball 2.8s done
#6 DONE 17.8s
full log (PR) 
# buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/ctx2 --local dockerfile=/tmp/ctx2 \
               --output type=oci,dest=/tmp/out.tar,oci-mediatypes=true,compression=gzip,force-compression=true
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 96B done
#1 DONE 0.2s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.2s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 ...

#4 [auth] ktock/tomcat:pull token for ghcr.io
#4 DONE 0.0s

#3 [internal] load metadata for ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd
#3 DONE 1.6s

#5 [1/1] FROM ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74
#5 resolve ghcr.io/ktock/tomcat:10.0.0-jdk15-openjdk-buster-zstd@sha256:89b5f6e1d8672f9140f28dd31e1b51c8727b9b83515cd1133d19b881028a9e74 4.9s done
#5 DONE 0.6s

#6 exporting to oci image format
#6 exporting layers
#6 exporting layers 12.0s done
#6 exporting manifest sha256:3625a6de03f11ab53479975bd056c2674f37bb6c20aa5c4147b6644bfc7efb04 0.0s done
#6 exporting config sha256:30d6e2ab67c2843a7d2989d98172538bd34cce799daccfc45912f4005dcc5a93 0.0s done
#6 sending tarball
#6 sending tarball 2.8s done
#6 DONE 14.9s

@ktock ktock marked this pull request as draft February 7, 2022 06:26
@ktock ktock marked this pull request as ready for review February 7, 2022 06:35
tonistiigi
tonistiigi reviewed Feb 7, 2022
View reviewed changes
Comment thread cache/converter.go Outdated
}
var zw io.WriteCloser = w
var compress io.WriteCloser
bufW := bufio.NewWriterSize(w, 128*1024)
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we use a pool for this. Probably not very meaningful but I think it's a good practice to do it for bigger buffers.

Copy link
Copy Markdown
Collaborator Author

@ktock ktock Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the comment, fixed.

tonistiigi
tonistiigi reviewed Feb 7, 2022
View reviewed changes
Comment thread cache/converter.go
if err := compress.Close(); err != nil { // Flush the writer
return nil, err
}
if err := zw.Close(); err != nil { // Flush the writer
Copy link
Copy Markdown
Member

@tonistiigi tonistiigi Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be called twice now

Copy link
Copy Markdown
Collaborator Author

@ktock ktock Feb 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the comment, fixed.

@ktock
cache: write blob to the content store via bufio during conversion
2d00773 
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
@ktock ktock mentioned this pull request Feb 7, 2022
Merged
@tonistiigi tonistiigi merged commit fb38dd3 into moby:master Feb 8, 2022
@ktock ktock deleted the conversion-bufio branch February 8, 2022 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tonistiigi tonistiigi tonistiigi approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ktock @tonistiigi