Add support for host-created EmptyDir

[evanfoster]

Is your feature request related to a problem? Please describe.

Kubernetes EmptyDirs are sometimes used as a way to share data from the host to a guest. The work done in #1485 drastically improved performance for most use-cases, but broke host->guest sharing of files.

Describe the solution you'd like

A flag in the configuration that disables guest-created EmptyDirs, allowing users to fall back to slower, host-created ones.

Describe alternatives you've considered

Unfortunately, there aren't really any other good solutions to the problem my org is using EmptyDir to solve. There are multi-tenancy issues with host volumes, and our use-case relies on linking container logs on the host to a logging container in a pod.

Additional context

I've already implemented this and my org is running a fork with a disable_guest_empty_dir flag, but I'd like to gauge upstream's interest in this. It's a very small changeset (+42 -11) and would allow folks to use EmptyDir with Kata in this way again.

Before raising this feature request

Have you looked at the limitations document?

Yes.

[evanfoster]

Please assign this to me.

[devimc]

done, thanks

[devimc]

cc @awprice

[grahamwhaley]

I suspect we should discuss if we want this to be a pod annotation rather than another config file option - we are trying to reduce/move away from/limit our config file bloat. And, this feels more like a per-pod choice than a whole stack choice? /cc @amshinde

[awprice]

👍 This change seems reasonable to me.

[amshinde]

If we were to add this, I think we need to support this as an annotation in addition to the config option to allow configuring this on a per pod basis.
@evanfoster Do volumes based on hostPath not fit your requirements?

[evanfoster]

Unfortunately not. We evaluated hostPath, but it's not really compatible with how we run our clusters (for example, hostPath breaks multi-tenancy).

I'm absolutely not opposed to adding an annotation as well. Being able to set the default to host-created or guest-created emptyDirs and then allowing users to selectively override per pod would be really nice.

[evanfoster]

We actually have a perfect use-case for that annotation. Since the config option is disable_guest_empty_dir and is under the runtime section, would the appropriate annotation be io.katacontainers.config.runtime.disable_guest_empty_dir ?

[amshinde]

We actually have a perfect use-case for that annotation. Since the config option is disable_guest_empty_dir and is under the runtime section, would the appropriate annotation be io.katacontainers.config.runtime.disable_guest_empty_dir ?

Yes, sounds right.

[evanfoster]

Running into a snag with this. The annotation needs to be applied at container creation time, but it's only being passed to the runtime during sandbox creation time. Is there a decent way to persist the annotation from sandbox to container creation? I've been digging, but I haven't found anyone else doing so yet.