virtcontainers: apply devices constraints

[devimc]

Apply devices constraints to the container in the virtual machine

Depends-on: github.com/kata-containers/agent#352

fixes #656

Signed-off-by: Julio Montes julio.montes@intel.com

[opendev-zuul]

Build failed (third-party-check pipeline) integration testing with
OpenStack. For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

• kata-runsh : RETRY_LIMIT in 8m 15s

[amshinde]

@devimc Lets verify this change works with device passthrough.

[bergwolf]

The comment is outdated even before this PR.

[devimc]

good catch! thanks

[katacontainersbot]

PSS Measurement:
Qemu: 169741 KB
Proxy: 4054 KB
Shim: 9115 KB

Memory inside container:
Total Memory: 2043460 KB
Free Memory: 2003556 KB

[devimc]

@amshinde do you know if currently we have a device passthrough test?

https://github.com/kata-containers/tests/blob/master/integration/docker/run_test.go#L85-L138

[opendev-zuul]

Build failed (third-party-check pipeline) integration testing with
OpenStack. For information on how to proceed, see
http://docs.openstack.org/infra/manual/developers.html#automated-testing

• kata-runsh : FAILURE in 32m 54s

[katacontainersbot]

PSS Measurement:
Qemu: 167558 KB
Proxy: 4175 KB
Shim: 8804 KB

Memory inside container:
Total Memory: 2043460 KB
Free Memory: 2003440 KB

[xindazhao]

Hi, is there any difference in the use of docker command by using device constrain? I use following command to enable INTEL GPU device support, with this commit added in, the GPU device node "/dev/dri/card0" and "/dev/dri/renderD128" can be seen in container, but returns error if open() the device node. Without this commit, everything works well.
What changes do I have to make in the command line to adapt to device constrains?

docker run -it --runtime=kata-runtime --rm --device=/dev/vfio/0 -v /dev:/dev ubuntu /bin/bash

[devimc]

@xindazhao this patch is to honour the devices cgroup, for example if you want to grant access to /dev/dri/card0 then you have to use --device /dev/dri/card0 in the command line. The problem with vfio devices is that once they are unbind'd from the host other devices like --device /dev/dri/card0 are no more visible, hence docker run --runtime kata-runtime --device /dev/dri/card0 -it --rm ubuntu bash fails with following error docker: Error response from daemon: linux runtime spec devices: error gathering device information while adding custom device "/dev/dri/card0": no such file or directory.
@xindazhao I have some ideas to fix the issue your are facing, is there a way to know which devices were bind'd or are part of a vfio device?, for example if you passthrough /dev/vfio/0, how I can identify that /dev/dri/card0 and /dev/dri/renderD128 were created because of /dev/vfio/0 was attached?

[xindazhao]

There is no direct relationship between the VFIO node and GPU device node. In the GVT-g (VFIO Mdev) scenario, multiple virtual GPUs share a physical GPU. We append different VFIO nodes (/dev/vfio/0, /dev/vfio/1 /dev/vfio/2 ....) to the docker command line, but in the guest VM, all the GPU device nodes that you see might be the same "/dev/dri/card0" and "/dev/dri/renderD128"

[sboeuf]

@devimc what's the story on this PR? What is the blocker to make CI passing so that we can move to a mergeable state?

[devimc]

if device constrains are applied, we'll face issues with vfio devices see #701 (comment)