Optional volumes control with ENV

[sielaq]

Hi,

I'm not sure if this is possible currently to solve with ENV variable.
I need to have docker-compose yaml that has optional volume mounting.
For example I could mount local version of docker binary into the container on demand.

so I can't use:
export OPT_PATH=/usr/local/bin/docker

  volumes:
   - ${OPT_PATH}:${OPT_PATH}

because if variable is not set - it generates errors.

I can't use ENV variable for the whole line either - since it breaks yaml structure:
export VOLUME_DOCKER='- "/usr/local/bin/docker:/usr/local/bin/docker" '

...
  volumes:
    ${VOLUME_DOCKER}

If only there was an additional flag like ignore or disable,
I would be able to describe the mount point,
and enable it on demand and still have a one common deployment plan:
export FLAG=ignore:

  volumes:
   - /tmp:/tmp:ignore
   - /usr/local/bin/docker:/usr/local/bin/docker:ro:${FLAG}

[shin-]

Hi!

I would actually handle this simply by having a docker-compose.dev.yml file with an override of your service that just contains the volume binding you want to use. That way, if you need to use the host binding, you would run docker-compose up -f docker-compose.yml -f docker-compose.dev.yml - and if not, you can just run docker-compose up. Does that make sense? You can see more info on extending compose files here: https://docs.docker.com/compose/extends/

HTH

[sielaq]

Hi,

Thanks for an idea, but I wanted to avoid having multiple yml files,
to not to make stuff more complex, and follow 12factor patterns to use ENV that configure all the things.

[sielaq]

@shin- Might be I should describe my problem a bit so you can understand me better.
I'm doing a specific project https://github.com/eBayClassifiedsGroup/PanteraS
PaaS in a one container box -this is kind of "Pod" if you use kubernetes nomenclature, but done with docker.
So I'm using supervisor inside container and describe which services should be run (master or slave related)
outside container using ENV variables.

Most problematic to set up, are ports (depends which service supposed to be run)
and volumes (depends of use-cases), so I have to make an own template
https://github.com/eBayClassifiedsGroup/PanteraS/blob/master/docker-compose.yml.tpl
and realize it with hacky way - since in the way it is now would not work directly.

I have more than 6 agnostic "moving parts" and combining them with -f foo.yml -f bar.yml ... would be very complex for the end user - especially user doesn't need to know which element is needed.
If only each line in yml would be able to be skipped, so ENV would be able to skip it,
this will much help prepare a proper configuration.

[sielaq]

@shin- sorry for nagging you, can you take a look at my last comment ?

[kopax]

any update on this ?

[mccredie]

I did the following and it seems to be working for me. Note that this only works on docker-compose format 2.1 but not 3.

I created two files. docker-compose.yml and docker-compose.base.yml

docker-compose.yml contains something like this:

version: 2.1
services:
  app:
    extends:
      file: docker-compose.base.yml
      service: "app_${BUILD_ENV-prod}"
    image: someimage:latest
    ports:
        - 8080:8080
    volumes:
        - /some/path:/some/other/path

And docker-compose.base.yml contains something like this:

version: 2.1
services:
  app_prod: {}
  app_dev:
    volumes:
      - /some/path/src: /some/other/path/src
    environment:
      DEBUG: 1

Running BUILD_ENV=dev docker-compose up will launch the dev version with the optional volumes mounted. docker-compose up will launch without the additional volumes.

[kilianc]

Thanks for an idea, but I wanted to avoid having multiple yml files,
to not to make stuff more complex, and follow 12factor patterns to use ENV that configure all the things.

I need to quote this. Having multiple files adds complexity. In my case I had to do dirty hacks like export VOLUME=./:/opt/.dummy to avoid mounting the volume.

Generally speaking @shin- I completely understand where you're coming from and that there is a way to override the yaml feeding multiple files but still, optional parameters would be very very nice to have.

I can apply the same example to cache_from. It works perfectly fine for CI but for local development it misses the cache and there is no way to specify local + image. It would be nice to disable with env interpolation!

[nyurik]

I had another usecase for this: Google cloud allows VMs to have "temporary local SSDs", and my startup script sets up environment variables depending on if the temporary partition exists or not (I need the same setup to work in multiple environments, with and without the temp partition). So some instances of docker-compose.yaml should mount the extra partition when it exists, but continue working properly when they do not.

[DomHudson]

I would also like to see this. It's generally considered risky to have different configurations for different deployments as it's a common source of errors between environments.

[chris-crone]

@sielaq We're looking to solve this problem in docker/app. We don't have an ETA for that particular feature to land as we're currently implementing CNAB support.

[gustavovalverde]

This would be a great feature!! It's so hard to set dynamic environments for users without this.

[gustavovalverde]

@kilianc Do you have an example on how to use this?

I need to quote this. Having multiple files adds complexity. In my case I had to do dirty hacks like export VOLUME=./:/opt/.dummy to avoid mounting the volume.