container: --gpus support #1714

tiborvass · 2019-03-05T04:10:32Z

Signed-off-by: Tibor Vass tibor@docker.com

Most common usecase: docker run --gpus all
Otherwise can be further specified: docker run --gpus 2,driver=nvidia,capabilities=compute

opts/gpus.go

codecov-io · 2019-03-12T22:30:21Z

Codecov Report

Merging #1714 into master will increase coverage by 0.17%.
The diff coverage is 67.18%.

@@            Coverage Diff             @@
##           master    #1714      +/-   ##
==========================================
+ Coverage   56.21%   56.38%   +0.17%     
==========================================
  Files         307      308       +1     
  Lines       21225    21241      +16     
==========================================
+ Hits        11931    11977      +46     
+ Misses       8427     8390      -37     
- Partials      867      874       +7

opts/gpus.go

tiborvass · 2019-03-19T19:03:54Z

Ping @thaJeztah @vdemeester @cpuguy83 @silvin-lubecki I revendored and it's ready for review.

cli/command/container/opts.go

RenaudWasTaken · 2019-03-19T22:17:38Z

Looks good to me!

thaJeztah · 2019-03-20T15:35:05Z

This can be rebased now (moby vendor bump was merged)

cli/command/container/opts.go

tiborvass · 2019-03-20T21:19:02Z

Ping @thaJeztah @silvin-lubecki @vdemeester updated

opts/gpus.go

thaJeztah

Some nits, notes, and suggestions

Some follow-ups needed to add Bash-completion and reference docs (I know we want to get this in before the release, so I'd be ok with doing that in a follow-up)

opts/gpus.go

thaJeztah · 2019-03-20T22:07:43Z

opts/gpus.go

+
+// Type returns the type of this option
+func (o *GpuOpts) Type() string {
+	return "gpuRequest"


perhaps gpu-options this is what's shown in --help output; also looks like we don't use camelBack there

I'm putting gpu-request

cli/command/container/opts.go

tiborvass · 2019-03-20T22:39:02Z

@thaJeztah @silvin-lubecki thank you for the review, i updated the PR accordingly.

thaJeztah

two more ideas, but LGTM

cli/command/container/opts.go

opts/gpus.go

tiborvass · 2019-03-20T22:55:29Z

@thaJeztah updated

thaJeztah

LGTM

vdemeester

SGTM, but @tiborvass can I HAZ tests on gpus.go ? 👼

tiborvass · 2019-03-21T18:08:15Z

@vdemeester added a couple. The most important one is --gpus all

Signed-off-by: Tibor Vass <tibor@docker.com>

vdemeester

LGTM 🐯

mviereck · 2019-08-21T14:17:24Z

It seems that the --gpus option is tied to NVIDIA hardware only and fails with other GPU vendors.
It seems that the --gpus option is tied to NVIDIA docker images only and fails with other images.

Am I right? If yes, this new option is a shame.
Please add GPU support for all GPU vendors. Seriously.

EDIT:
Sorry, was intended to be posted in #1200

RenaudWasTaken · 2019-08-21T18:43:22Z

Hello @mviereck !

The --gpus option is implemented more as a plugin than as a "vendor-lock-in". For exemple it doesn't work until you install the nvidia-container-toolkit (in other words the nvidia plugin).

We have plans to make it more like other plugins (docker plugin install), so that it's easier for everyone to build their own plugins. So far we have lacked time to do so. If you want to help drive this effort we'd be happy to help you (e.g: through review)!
The way the interface for GPUs has been designed should easily map to a REST API.

mviereck · 2019-08-23T14:23:32Z

@RenaudWasTaken Thank you for your friendly response!

If you want to help drive this effort we'd be happy to help you (e.g: through review)!

I am not experienced in Go, so I cannot contribute code directly, sorry.
I have opened a new ticket on this topic with some proposals, maybe you want to look at it, too: #2063

GordonTheTurtle added the status/0-triage label Mar 5, 2019

tiborvass mentioned this pull request Mar 5, 2019

Add DeviceRequests to HostConfig to support NVIDIA GPUs moby/moby#38828

Merged

tiborvass commented Mar 5, 2019

View reviewed changes

opts/gpus.go Outdated Show resolved Hide resolved

tiborvass commented Mar 5, 2019

View reviewed changes

opts/gpus.go Show resolved Hide resolved

tiborvass force-pushed the nvidia-gpu branch from 2ee6ae8 to a73299c Compare March 12, 2019 22:30

tiborvass force-pushed the nvidia-gpu branch 2 times, most recently from 632b386 to 9621d3f Compare March 14, 2019 05:43

tiborvass changed the title ~~[WIP] Add --gpu support~~ [WIP] Add --gpus support Mar 14, 2019

tiborvass force-pushed the nvidia-gpu branch from 9621d3f to 164785c Compare March 19, 2019 19:00

tiborvass changed the title ~~[WIP] Add --gpus support~~ container: --gpus support Mar 19, 2019

tiborvass marked this pull request as ready for review March 19, 2019 19:01

tiborvass force-pushed the nvidia-gpu branch from 164785c to e0c6513 Compare March 19, 2019 19:02

tiborvass commented Mar 19, 2019

View reviewed changes

opts/gpus.go Outdated Show resolved Hide resolved

thaJeztah reviewed Mar 19, 2019

View reviewed changes

cli/command/container/opts.go Outdated Show resolved Hide resolved

thaJeztah added status/2-code-review and removed status/0-triage labels Mar 20, 2019

thaJeztah reviewed Mar 20, 2019

View reviewed changes

cli/command/container/opts.go Outdated Show resolved Hide resolved

tiborvass force-pushed the nvidia-gpu branch 4 times, most recently from 308c073 to 3d2b393 Compare March 20, 2019 21:17

silvin-lubecki reviewed Mar 20, 2019

View reviewed changes

opts/gpus.go Outdated Show resolved Hide resolved

silvin-lubecki reviewed Mar 20, 2019

View reviewed changes

opts/gpus.go Outdated Show resolved Hide resolved

silvin-lubecki reviewed Mar 20, 2019

View reviewed changes

opts/gpus.go Outdated Show resolved Hide resolved

thaJeztah requested changes Mar 20, 2019

View reviewed changes

cli/command/container/opts.go Outdated Show resolved Hide resolved

opts/gpus.go Outdated Show resolved Hide resolved

tiborvass force-pushed the nvidia-gpu branch from a64b404 to 3dfc04f Compare March 20, 2019 22:55

tiborvass force-pushed the nvidia-gpu branch from 3dfc04f to 1996fb3 Compare March 20, 2019 23:03

thaJeztah approved these changes Mar 20, 2019

View reviewed changes

vdemeester reviewed Mar 21, 2019

View reviewed changes

tiborvass force-pushed the nvidia-gpu branch from 1996fb3 to 76f3ffc Compare March 21, 2019 18:07

tiborvass force-pushed the nvidia-gpu branch from 76f3ffc to 6e0c0e4 Compare March 21, 2019 18:11

container: --gpus support

1ba368a

Signed-off-by: Tibor Vass <tibor@docker.com>

tiborvass force-pushed the nvidia-gpu branch from 6e0c0e4 to 1ba368a Compare March 21, 2019 20:14

vdemeester approved these changes Mar 21, 2019

View reviewed changes

tiborvass added status/4-merge and removed status/2-code-review labels Mar 21, 2019

thaJeztah merged commit 059c085 into docker:master Mar 21, 2019

GordonTheTurtle added this to the 19.03.0 milestone Mar 21, 2019

collabnix mentioned this pull request May 9, 2019

Support for NVIDIA GPUs under Docker Compose docker/compose#6691

Closed

schra mentioned this pull request Aug 21, 2019

Support for Docker's --gpus option mviereck/x11docker#180

Closed

mviereck mentioned this pull request Aug 23, 2019

Option --gpus failes with AMD and Intel GPUs #2063

Closed

jamesdbrock mentioned this pull request Aug 27, 2019

Use runtime nvidia jupyterhub/dockerspawner#244

Closed

h1nk mentioned this pull request Dec 27, 2019

Generic Driver: Add an additional create sub-command option to automatically install the NVIDIA Container Toolkit docker-archive-public/docker.machine#4780

Open

tonistiigi mentioned this pull request Mar 18, 2020

Support RUN --gpus ... or equivalent. docker/buildx#239

Closed

mneilly-et mentioned this pull request Sep 10, 2020

Update run_and_time.sh to support docker 19.03 and newer mlcommons/inference#727

Closed

tonistiigi mentioned this pull request Feb 6, 2021

Support RUN --gpus ... or equivalent. moby/buildkit#1436

Closed

0x2b3bfa0 mentioned this pull request Sep 17, 2022

nvidia-docker2 might not be needed iterative/terraform-provider-iterative#674

Closed

BMurri mentioned this pull request Jun 4, 2024

Preview: GPU support microsoft/ga4gh-tes#717

Merged

container: --gpus support #1714

container: --gpus support #1714

Uh oh!

Conversation

tiborvass commented Mar 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-io commented Mar 12, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

tiborvass commented Mar 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

RenaudWasTaken commented Mar 19, 2019

Uh oh!

thaJeztah commented Mar 20, 2019

Uh oh!

Uh oh!

tiborvass commented Mar 20, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thaJeztah Mar 20, 2019

Choose a reason for hiding this comment

Uh oh!

tiborvass Mar 20, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tiborvass commented Mar 20, 2019

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tiborvass commented Mar 20, 2019

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

vdemeester left a comment

Choose a reason for hiding this comment

Uh oh!

tiborvass commented Mar 21, 2019

Uh oh!

vdemeester left a comment

Choose a reason for hiding this comment

Uh oh!

mviereck commented Aug 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RenaudWasTaken commented Aug 21, 2019

Uh oh!

mviereck commented Aug 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

tiborvass commented Mar 5, 2019 •

edited

Loading

codecov-io commented Mar 12, 2019 •

edited

Loading

tiborvass commented Mar 19, 2019 •

edited

Loading

mviereck commented Aug 21, 2019 •

edited

Loading