build support#1
Closed
ndeloof wants to merge 48 commits into
Closed
Conversation
The Linux kernel never sets the Inheritable capability flag to anything other than empty. Moby should have the same behavior, and leave it to userspace code within the container to set a non-empty value if desired. Reported-by: Andrew G. Morgan <morgan@kernel.org> Signed-off-by: Samuel Karp <skarp@amazon.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This should help with CI being unstable when generating the types (due to Go randomizing order). Unfortunately, the (file) names are a bit ugly, but addressing that in a follow-up. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Stéphane Este-Gracias <sestegra@gmail.com>
Finish the refactor which was partially completed with commit 34536c4, passing around IdentityMapping structs instead of pairs of []IDMap slices. Existing code which uses []IDMap relies on zero-valued fields to be valid, empty mappings. So in order to successfully finish the refactoring without introducing bugs, their replacement therefore also needs to have a useful zero value which represents an empty mapping. Change IdentityMapping to be a pass-by-value type so that there are no nil pointers to worry about. The functionality provided by the deprecated NewIDMappingsFromMaps function is required by unit tests to to construct arbitrary IdentityMapping values. And the daemon will always need to access the mappings to pass them to the Linux kernel. Accommodate these use cases by exporting the struct fields instead. BuildKit currently depends on the UIDs and GIDs methods so we cannot get rid of them yet. Signed-off-by: Cory Snider <csnider@mirantis.com>
The wrapResponseError() utility converted some specific errors, but in doing so, could hide the actual error message returned by the daemon. In addition, starting with 38e6d47, HTTP status codes were already mapped to their corresponding errdefs types on the client-side, making this conversion redundant. This patch removes the wrapResponseError() utility; it's worth noting that some error-messages will change slightly (as they now return the error as returned by the daemon), but may cointain more details as before, and in some cases prevents hiding the actual error. Before this change: docker container rm nosuchcontainer Error: No such container: nosuchcontainer docker container cp mycontainer:/no/such/path . Error: No such container:path: mycontainer:/no/such/path docker container cp ./Dockerfile mycontainer:/no/such/path Error: No such container:path: mycontainer:/no/such docker image rm nosuchimage Error: No such image: nosuchimage docker network rm nosuchnetwork Error: No such network: nosuchnetwork docker volume rm nosuchvolume Error: No such volume: nosuchvolume docker plugin rm nosuchplugin Error: No such plugin: nosuchplugin docker checkpoint rm nosuchcontainer nosuchcheckpoint Error response from daemon: No such container: nosuchcontainer docker checkpoint rm mycontainer nosuchcheckpoint Error response from daemon: checkpoint nosuchcheckpoint does not exist for container mycontainer docker service rm nosuchservice Error: No such service: nosuchservice docker node rm nosuchnode Error: No such node: nosuchnode docker config rm nosuschconfig Error: No such config: nosuschconfig docker secret rm nosuchsecret Error: No such secret: nosuchsecret After this change: docker container rm nosuchcontainer Error response from daemon: No such container: nosuchcontainer docker container cp mycontainer:/no/such/path . Error response from daemon: Could not find the file /no/such/path in container mycontainer docker container cp ./Dockerfile mycontainer:/no/such/path Error response from daemon: Could not find the file /no/such in container mycontainer docker image rm nosuchimage Error response from daemon: No such image: nosuchimage:latest docker network rm nosuchnetwork Error response from daemon: network nosuchnetwork not found docker volume rm nosuchvolume Error response from daemon: get nosuchvolume: no such volume docker plugin rm nosuchplugin Error response from daemon: plugin "nosuchplugin" not found docker checkpoint rm nosuchcontainer nosuchcheckpoint Error response from daemon: No such container: nosuchcontainer docker checkpoint rm mycontainer nosuchcheckpoint Error response from daemon: checkpoint nosuchcheckpoint does not exist for container mycontainer docker service rm nosuchservice Error response from daemon: service nosuchservice not found docker node rm nosuchnode Error response from daemon: node nosuchnode not found docker config rm nosuchconfig Error response from daemon: config nosuchconfig not found docker secret rm nosuchsecret Error response from daemon: secret nosuchsecret not found Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Operations performed on overlay network sandboxes are handled by dispatching operations send through a channel. This allows for asynchronous operations to be performed which, since they are not called from within another function, are able to operate in an idempotent manner with a known/measurable starting state from which an identical series of iterative actions can be performed. However, it was possible in some cases for an operation dispatched from this channel to write a message back to the channel in the case of joining a network when a sufficient volume of sandboxes were operated on. A goroutine which is simultaneously reading and writing to an unbuffered channel can deadlock if it sends a message to a channel then waits for it to be consumed and completed, since the only available goroutine is more or less "talking to itself". In order to break this deadlock, in the observed race, a goroutine is now created to send the message to the channel. Signed-off-by: Martin Dojcak <martin.dojcak@lablabs.io> Signed-off-by: Ryan Barry <rbarry@mirantis.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Elias Koromilas <elias.koromilas@gmail.com>
oci: inheritable capability set should be empty
client: remove wrapResponseError()
Mount (accessible) host devices in `--privileged` rootless containers
vendor buildkit v0.10.0
The second patch release for containerd 1.6 includes a fix for [CVE-2022-24769](GHSA-c9cp-9c75-9v8c). Notable Updates - **Fix the inheritable capability defaults** ([GHSA-c9cp-9c75-9v8c](GHSA-c9cp-9c75-9v8c)) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
includes a fix for CVE-2022-24769. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The moby#42511 PR removed layer store indexing by OS but this comment was left behind Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Remove comment that is no longer relevant
update containerd binary and vendor to v1.6.2
api: swagger: use explicit definitions, use inline examples, and fix indentation.
- use pkg/errors for errors and fix error-capitalisation - remove one redundant call to logDeprecatedWarning() (we're already skipping deprecated drivers in that loop). - rename `list` to `priorityList` for readability. - remove redundant "skip" for the vfs storage driver, as it's already excluded by `scanPriorDrivers()` - change one debug log to an "info", so that the daemon logs contain the driver that was configured, and include "multiple prior states found" error in the daemon logs, to assist in debugging failed daemon starts. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, we only printed a warning if a storage driver was deprecated. The
intent was to continue supporting these drivers, to allow users to migrate
to a different storage driver.
This patch changes the behavior; if the user has no storage driver specified
in the daemon configuration (so if we try to detect the previous storage
driver based on what's present in /var/lib/docker), we now produce an error,
informing the user that the storage driver is deprecated (and to be removed),
as well as instructing them to change the daemon configuration to explicitly
select the storage driver (to allow them to migrate).
This should make the deprecation more visible; this will be disruptive, but
it's better to have the failure happening *now* (while the drivers are still
there), than for users to discover the storage driver is no longer there
(which would require them to *downgrade* the daemon in order to migrate
to a different driver).
With this change, `docker info` includes a link in the warnings that:
/ # docker info
Client:
Context: default
Debug Mode: false
Server:
...
Live Restore Enabled: false
WARNING: The overlay storage-driver is deprecated, and will be removed in a future release.
Refer to the documentation for more information: https://docs.docker.com/go/storage-driver/
When starting the daemon without a storage driver configured explicitly, but
previous state was using a deprecated driver, the error is both logged and
printed:
...
ERRO[2022-03-25T14:14:06.032014013Z] [graphdriver] prior storage driver overlay is deprecated and will be removed in a future release; update the the daemon configuration and explicitly choose this storage driver to continue using it; visit https://docs.docker.com/go/storage-driver/ for more information
...
failed to start daemon: error initializing graphdriver: prior storage driver overlay is deprecated and will be removed in a future release; update the the daemon configuration and explicitly choose this storage driver to continue using it; visit https://docs.docker.com/go/storage-driver/ for more information
When starting the daemon and explicitly configuring it with a deprecated storage
driver:
WARN[2022-03-25T14:15:59.042335412Z] [graphdriver] WARNING: the overlay storage-driver is deprecated and will be removed in a future release; visit https://docs.docker.com/go/storage-driver/ for more information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The registry package contained code to automatically set the CertsDir() path, based on wether or not the daemon was running in rootlessmode. In doing so, it made use of the `pkg/rootless.RunningWithRootlessKit()` utility. A recent change in de6732a added additional functionality in the `pkg/rootless` package, introducing a dependency on `github.com/rootless-containers/rootlesskit`. Unfortunately, the extra dependency also made its way into the docker cli, which also uses the registry package. This patch introduces a new `SetCertsDir()` function, which allows the default certs-directory to be overridden, and updates the daemon to configure this location during startup. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Only minor changes since v0.14.6, such as upgrading Go to 1.18: rootless-containers/rootlesskit@v0.14.6...v1.0.0 Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
libnetwork/overlay:fix sandbox deadlock
daemon: require storage-driver to be set if the driver is deprecated
This adds an additional "Swarm" header to the _ping endpoint response, which allows a client to detect if Swarm is enabled on the daemon, without having to call additional endpoints. This change is not versioned in the API, and will be returned irregardless of the API version that is used. Clients should fall back to using other endpoints to get this information if the header is not present. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
update RootlessKit to v1.0.0
API: add "Swarm" header to _ping endpoint
Allow STOPSIGNAL instruction in commit change
…ndency_in_cli registry: remove dependency on rootlesskit, add `SetCertsDir()`
full diff: moby/buildkit@8d45bd6...d7744bc Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
vendor: github.com/moby/buildkit d7744bc
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ndeloof
force-pushed
the
containerd-store
branch
4 times, most recently
from
35d82c0 to
58da2bd
Compare
ndeloof
force-pushed
the
containerd-store
branch
from
58da2bd to
749c755
Compare
rumpl
reviewed
Jun 29, 2022
Comment on lines
+65
to
+70
| wo.ImageStore = opt.ContainerdCli.ImageService() | ||
| wo.ContentStore = opt.ContainerdCli.ContentStore() | ||
| wo.LeaseManager = opt.ContainerdCli.LeasesService() | ||
| wo.Snapshotter = snapshot.FromContainerdSnapshotter(ctd.DefaultSnapshotter, opt.ContainerdCli.SnapshotService(ctd.DefaultSnapshotter), &opt.IdentityMapping) | ||
| wo.Applier = opt.ContainerdCli.DiffService() | ||
| wo.Differ = opt.ContainerdCli.DiffService() |
Owner
There was a problem hiding this comment.
Build works if you remove these lines, they are already correctly set in containerd.NewWorkerOpt
rumpl
pushed a commit
that referenced
this pull request
Aug 22, 2023
I had a CI run fail to "Upload reports":
Exponential backoff for retry #1. Waiting for 4565 milliseconds before continuing the upload at offset 0
Finished backoff for retry #1, continuing with upload
Total file count: 211 ---- Processed file moby#160 (75.8%)
...
Total file count: 211 ---- Processed file moby#164 (77.7%)
Total file count: 211 ---- Processed file moby#164 (77.7%)
Total file count: 211 ---- Processed file moby#164 (77.7%)
A 503 status code has been received, will attempt to retry the upload
##### Begin Diagnostic HTTP information #####
Status Code: 503
Status Message: Service Unavailable
Header Information: {
"content-length": "592",
"content-type": "application/json; charset=utf-8",
"date": "Mon, 21 Aug 2023 14:08:10 GMT",
"server": "Kestrel",
"cache-control": "no-store,no-cache",
"pragma": "no-cache",
"strict-transport-security": "max-age=2592000",
"x-tfs-processid": "b2fc902c-011a-48be-858d-c62e9c397cb6",
"activityid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-tfs-session": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-e2eid": "49a48b53-0411-4ff3-86a7-4528e3f71ba2",
"x-vss-senderdeploymentid": "63be6134-28d1-8c82-e969-91f4e88fcdec",
"x-frame-options": "SAMEORIGIN"
}
###### End Diagnostic HTTP information ######
Retry limit has been reached for chunk at offset 0 to https://pipelinesghubeus5.actions.githubusercontent.com/Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-22.04-systemd%2Fbundles%2Ftest-integration%2FTestInfoRegistryMirrors%2Fd20ac12e48cea%2Fdocker.log
Warning: Aborting upload for /tmp/reports/ubuntu-22.04-systemd/bundles/test-integration/TestInfoRegistryMirrors/d20ac12e48cea/docker.log due to failure
Error: aborting artifact upload
Total file count: 211 ---- Processed file moby#165 (78.1%)
A 503 status code has been received, will attempt to retry the upload
Exponential backoff for retry #1. Waiting for 5799 milliseconds before continuing the upload at offset 0
As a result, the "Download reports" continued retrying:
...
Total file count: 1004 ---- Processed file moby#436 (43.4%)
Total file count: 1004 ---- Processed file moby#436 (43.4%)
Total file count: 1004 ---- Processed file moby#436 (43.4%)
An error occurred while attempting to download a file
Error: Request timeout: /Y2huPMnV2RyiTvKoReSyXTCrcRyxUdSDRZYoZr0ONBvpl5e9Nu/_apis/resources/Containers/8331549?itemPath=integration-reports%2Fubuntu-20.04%2Fbundles%2Ftest-integration%2FTestCreateWithDuplicateNetworkNames%2Fd47798cc212d1%2Fdocker.log
at ClientRequest.<anonymous> (/home/runner/work/_actions/actions/download-artifact/v3/dist/index.js:3681:26)
at Object.onceWrapper (node:events:627:28)
at ClientRequest.emit (node:events:513:28)
at TLSSocket.emitRequestTimeout (node:_http_client:839:9)
at Object.onceWrapper (node:events:627:28)
at TLSSocket.emit (node:events:525:35)
at TLSSocket.Socket._onTimeout (node:net:550:8)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7)
Exponential backoff for retry #1. Waiting for 5305 milliseconds before continuing the download
Total file count: 1004 ---- Processed file moby#436 (43.4%)
And, it looks like GitHub doesn't allow cancelling the job, possibly
because it is defined with `if: always()`?
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
rumpl
pushed a commit
that referenced
this pull request
Jun 25, 2024
…f v1.5.4 full diffs: - protocolbuffers/protobuf-go@v1.31.0...v1.33.0 - golang/protobuf@v1.5.3...v1.5.4 From the Go security announcement list; > Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in > the google.golang.org/protobuf/encoding/protojson package which could cause > the Unmarshal function to enter an infinite loop when handling some invalid > inputs. > > This condition could only occur when unmarshaling into a message which contains > a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown > option is set. Unmarshal now correctly returns an error when handling these > inputs. > > This is CVE-2024-24786. In a follow-up post; > A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown > option is set (as well as when unmarshaling into any message which contains a > google.protobuf.Any). There is no UnmarshalUnknown option. > > In addition, version 1.33.0 of google.golang.org/protobuf inadvertently > introduced an incompatibility with the older github.com/golang/protobuf > module. (golang/protobuf#1596) Users of the older > module should update to github.com/golang/protobuf@v1.5.4. govulncheck results in our code: govulncheck ./... Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities... === Symbol Results === Vulnerability #1: GO-2024-2611 Infinite loop in JSON unmarshaling in google.golang.org/protobuf More info: https://pkg.go.dev/vuln/GO-2024-2611 Module: google.golang.org/protobuf Found in: google.golang.org/protobuf@v1.31.0 Fixed in: google.golang.org/protobuf@v1.33.0 Example traces found: #1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek #2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read #3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal Your code is affected by 1 vulnerability from 1 module. This scan found no other vulnerabilities in packages you import or modules you require. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
rumpl
pushed a commit
that referenced
this pull request
Jun 25, 2024
api: Make EnableIPv6 optional (impl #1 - pointer-based)
rumpl
pushed a commit
that referenced
this pull request
Dec 30, 2024
contains a fix for CVE-2024-45338 / https://go.dev/issue/70906, but it doesn't affect our codebase: govulncheck -show=verbose ./... Scanning your code and 1260 packages across 211 dependent modules for known vulnerabilities... ... Vulnerability #1: GO-2024-3333 Non-linear parsing of case-insensitive content in golang.org/x/net/html More info: https://pkg.go.dev/vuln/GO-2024-3333 Module: golang.org/x/net Found in: golang.org/x/net@v0.32.0 Fixed in: golang.org/x/net@v0.33.0 Your code is affected by 0 vulnerabilities. This scan also found 0 vulnerabilities in packages you import and 1 vulnerability in modules you require, but your code doesn't appear to call these vulnerabilities. full diff: golang/net@v0.32.0...v0.33.0 Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This introduce (partial) support for build
merge upstream master to get buildkit dependency bump