feature request: more control over the docker build context

[majelbstoat]

I have a monorepo with shared code. Because docker doesn't follow symlinks and never will, I have to copy that shared code into the right place before building the image for each service. There's currently no facility for doing that with skaffold. I'm using VSCode's Cloud Code extension, so running a wrapper script isn't straightforward, but in any case, I'd like the build to be fully described in the definition, not be dependent on some external glue.

#1441 describes one approach to solving this, which would be great, but I don't need the full flexibility of a custom script.

What I'd like to see is the ability to control which parts of the context are put into the tarball that is sent to Docker. Being able to specify paths, and mappings would be sufficient for my use-case. For example:

context:
  root: .
  paths:
    - mapRef:
        from: services/mercury/src
        to: .
    - mapRef:
        from: shared/nodejs/src
        to: ./lib

This would create a tarball that had files from mercury at the root, and files from the shared source at ./lib. Ideally, it would do this without actually copying the shared files into the mercury service in the local file system.

The absence of the path key would cause behaviour as currently is, i.e. everything beneath the context.

[corneliusweig]

Would it be an option to create hard links instead of soft links? This should work with docker, however it's probably not feasible if you need to sync whole directories.

[majelbstoat]

Yeah, hard links cannot be created for directories, so unfortunately not.

[corneliusweig]

Yeah, hard links cannot be created for directories, so unfortunately not.

So recursively creating hard links is also not an option?

cp -rl from to

[majelbstoat]

The code is shared to multiple different locations in the monorepo, and the benefit of symlinks is whenever a new shared library is added, it is immediately available to all the services. Running a script each time a new file is added complicates the developer workflow. I already have a hacky script to glue bits and pieces together, replacing it with another isn't an improvement :)

(With the rise of lerna et al, this is not an uncommon pattern, btw. Most organisations with a monorepo and micro services will probably come across some form of this.)

[balopat]

I think it would be interesting to provide support for a flexible Docker context creation.
Skaffold could assemble the tar file based on the context spec and pipe it to docker build.
It is big enough though that I think a good design proposal would be beneficial before someone jumps in to implement it.

[tstromberg]

Thank you for the idea!

I'm closing this issue as it's been open a while, and it is not clear if it's still an open issue. No one has recently stated an interest in addressing this, but if you feel strongly about it, please feel free to add a comment or send us a PR!