create_container not support --gpus param

[ffteen]

docker version: 19.03
I want to set --gpus all when create container ,but found docker-py not support this param.

[jcsirot]

Hello @ffteen thank you for the report

[msadri70]

any progress on this issue?

[Bluemi]

I think one hacky way, though not very reliable, is to use the low-level Api and overwrite the host configuration. Since I only tried to follow the docker cli code in go, I'm not sure how reliable/portable this solution is. It works on my machine and I thought it might help someone until the official support is implemented.

The following code is a modification of the original DockerClient.containers.create() function, that adds a DeviceRequest to the host configuration and otherwise works exactly like the original function:

import docker
from docker.models.images import Image
from docker.models.containers import _create_container_args

def create_with_device_request(client, image, command, device_request=None, **kwargs):
    if isinstance(image, Image):
        image = image.id
    kwargs['image'] = image
    kwargs['command'] = command
    kwargs['version'] = client.containers.client.api._version
    create_kwargs = _create_container_args(kwargs)

    # modification to the original create function
    if device_request is not None:
        create_kwargs['host_config']['DeviceRequests'] = [device_request]
    # end modification

    resp = client.api.create_container(**create_kwargs)
    return client.containers.get(resp['Id'])

# Example usage
device_request = {
    'Driver': 'nvidia',
    'Capabilities': [['gpu'], ['nvidia'], ['compute'], ['compat32'], ['graphics'], ['utility'], ['video'], ['display']],  # not sure which capabilities are really needed
    'Count': -1,  # enable all gpus
}

container = create_with_device_request(docker.from_env(), 'nvidia/cuda:9.0-base', 'nvidia-smi', device_request, ...)

I think the cli client sets the NVIDIA_VISIBLE_DEVICES environment variable, so it's probably a good idea to do the same with environment={'NVIDIA_VISIBLE_DEVICES': 'all'} as parameter of the create_with_device_request() call.
This enables all available gpus. You could modify this with different device_requests:

# enable two gpus
device_request = {
    'Driver': 'nvidia',
    'Capabilities': ...,
    'Count': 2,  # enable two gpus
}

# enable gpus with id or uuid
device_request = {
    'Driver': 'nvidia',
    'Capabilities': ...,
    'DeviceIDs': ['0', 'GPU-abcedfgh-1234-a1b2-3c4d-a7f3ovs13da1']  # enable gpus with id 0 and uuid
}

The environment parameter should then look like {'NVIDIA_VISIBLE_DEVICES': '0,1'} respectively {'NVIDIA_VISIBLE_DEVICES': '0,GPU-xxx'}

[ffteen]

I‘m not sure which capabilities are really needed too！

Does create_service support device request param？

I use nvidia runtime instead.

[Bluemi]

As far as I can tell, services.create() does not support device requests.

Setting runtime='nvidia' is definitely the better approach, if possible.
The problem I had was, that I use the nvidia-container-toolkit which does not require to install the nvidia-runtime, so setting nvidia runtime leads to Error: unknown runtime specified nvidia, while using --gpus=all works as expected.

Is there a better way to use nvidia-gpus with the nvidia-container-toolkit?

[hnine999]

I have a change (that appears to work) that allows the "gpus" option in my fork. I'd like to create a PR for it, but when running the tests, this error (which is unrelated to the change) occurs:

tests/integration/api_service_test.py:379:53: F821 undefined name 'BUSYBOX'
Makefile:92: recipe for target 'flake8' failed

Is there a package that needs to be installed to fix this?

[shin-]

@hnine999 No, that's an error on our end - we'll fix it shortly. Feel free to submit your PR in the meantime!

[jamesdbrock]

The PR from @hnine999 is #2419

[rAm1n]

Hi - Any update with this feature?