Unify windows snapshotters & add support to store scratch layers in different directory#27
Merged
ambarve merged 5 commits intokevpar:fork/masterfrom Nov 22, 2021
Merged
Conversation
Both LCOW & WCOW snapshotters have many functions that are identical and some other functions have common code. This change refactors those snapshotters to remove duplicate code. This refactor will also help when adding other features like storing scratch snapshots in a different location. Signed-off-by: Amit Barve <ambarve@microsoft.com>
Some containers do heavy IO to the scratch, in case of LCOW/WCOW the scratch itself is a VHD that is stored on the same disk as that of other snapshots (including the image layers). If containerd is pulling multiple images and one or more containers are doing heavy IO to the scratch at the same time, that causes disk contention and related issues. This changes allows the user to set a config (either in containerd.toml or in the container configuration) that specifies another directory inside which the scratch of all containers (or that specific container) should be stored. Signed-off-by: Amit Barve <ambarve@microsoft.com>
This was referenced Oct 18, 2021
Collaborator
|
It'd be nice to have a PR description here (even if the commits go into detail) |
dcantah
reviewed
Oct 25, 2021
dcantah
reviewed
Oct 25, 2021
dcantah
reviewed
Oct 25, 2021
dcantah
reviewed
Oct 25, 2021
dcantah
reviewed
Oct 25, 2021
dcantah
reviewed
Oct 26, 2021
| ic.Meta.Platforms = append(ic.Meta.Platforms, ocispec.Platform{ | ||
| OS: "linux", | ||
| Architecture: "amd64", | ||
| Architecture: runtime.GOARCH, |
Collaborator
There was a problem hiding this comment.
Any reason for this change?
Collaborator
Author
There was a problem hiding this comment.
I think in upstream they changed this and when I was comparing the changes with upstream I must have copied that. I can change it back.
Signed-off-by: Amit Barve <ambarve@microsoft.com>
Collaborator
Author
dcantah
reviewed
Nov 4, 2021
anmaxvl
reviewed
Nov 11, 2021
helsaawy
reviewed
Nov 11, 2021
helsaawy
reviewed
Nov 15, 2021
Signed-off-by: Amit Barve <ambarve@microsoft.com>
Signed-off-by: Amit Barve <ambarve@microsoft.com>
Collaborator
|
lgtm |
anmaxvl
approved these changes
Nov 22, 2021
ambarve
added a commit
that referenced
this pull request
Jan 25, 2022
…ifferent directory (#27) (#30) * Refactor windows snapshotters Both LCOW & WCOW snapshotters have many functions that are identical and some other functions have common code. This change refactors those snapshotters to remove duplicate code. This refactor will also help when adding other features like storing scratch snapshots in a different location. * Support to override scratch snapshot location Some containers do heavy IO to the scratch, in case of LCOW/WCOW the scratch itself is a VHD that is stored on the same disk as that of other snapshots (including the image layers). If containerd is pulling multiple images and one or more containers are doing heavy IO to the scratch at the same time, that causes disk contention and related issues. This change allows the user to set a config (either in containerd.toml or in the container configuration) that specifies another directory inside which the scratch of all containers (or that specific container) should be stored. Signed-off-by: Amit Barve <ambarve@microsoft.com> (cherry picked from commit 333a72b) Signed-off-by: Amit Barve <ambarve@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
In some cases it is desirable to store scratch VHDs of WCOW/LCOW containers on a different volume. Right now the scratch VHDs of containers and the image layers are all stored under the same directory (
snapshots). If there are containers doing heavy IO to their scratch disks and containerd is also pulling and extracting several images at the same time then the disk on which all of these snapshots are stored starts hitting IO limits and these snapshot operations take long time than usual to complete.Solution:
Current approach is to provide a config option to containerd that will specify path to a different directory (which can be on a different volume) on which all scratch snapshots should be stored. We achieve this by creating the directory for scratch snapshots under this new path but we create a symlink to this directory under the default
snapshotsdirectory.Commit description:
The change mentioned above needed to be done for both WCOW and LCOW snapshotters and it seems that both WCOW and LCOW snapshotters have a lot of duplicated code. The first commit adds a new common base snapshotter for both WCOW & LCOW snapshotters.
Second commit actually adds the changes required for supporting storing snapshots in a different directory.