Feature proposal: cdi mode, add something close to the no-cgroups feature in legacy mode

[oOraph]

Proposal implemented here:
#1541

[oOraph]

Unless there is some way I missed to get some close behaviour with (jit)-cdi ?

[elezar]

@oOraph in order to assess the proposal properly, could you expand the issue description to describe the intented use cases that are not covered by the current implementation. Here errors that you're seeing would be useful.

[oOraph]

Sure:

Use case: same as the no-cgroups flags in legacy mode (= with the hook) -> create dev nodes, setup environment, mount shared libraries but do not give access to devices yet

When we spawned a container in legacy mode adding:

[...]
[nvidia-container-cli]
[...]
no-cgroups = true
[...]
[nvidia-container-runtime]
mode = "legacy"

to the nvidia-container-runtime config.toml file

The device access was not handled by the nvidia-container-runtime on container startup. So when spawning a container with NVIDIA_VISIBLE_DEVICES="all"

The runtime then made sure that:

• The dev nodes were created,
• The required shared libraries (mostly cuda/nvml ones) to interract with the gpu were mounted

But the device accesses were not granted yet

This was really convenient: then we could later grant gpu accesses to the container by editing the cgroup device whitelist afterward

But now with the cdi mode this no-cgroups behaviour is gone, there is no way to get the dev nodes created and libs mounted without getting the device rwm permissions granted alongside

here:

nvidia-container-toolkit/internal/edits/device.go

Line 74 in 4b63c42

return &specs.DeviceNode{

[elezar]

Thanks for the clarification. I will have a look at the proposal with this in mind. Could you expand on the use case for managing the cgroups yourselves?