Environment Variables Not Being Passed to Containers Properly

[alunduil]

I use environment variable declarations like the following:

  environment:
   - API_DEBUG=true
   - DATASTORE_URL=http://${ETCD_PORT_2379_TCP_ADDR}:${ETCD_PORT_2379_TCP_PORT}
   - KUBERNETES_URL=http://${KUBERNETESAPISERVER_PORT_8080_TCP_ADDR}:${KUBERNETESAPISERVER_PORT_8080_TCP_PORT}

With compose-1.5 this seems to pass through empty strings:

• DATASTORE_URL=http://:
• KUBERNETES_URL=http://:

How do we get the correct behaviour of passing these variables through as written (without any needless mangling)?

This breaks our ability to use compose to deploy testing environments that function correctly.

[dnephin]

This was the only breaking change in 1.5.0 (unfortunate, but there wasn't really any way around it). The release notes include a warning about it.

You can escape the $ with another $: http://docs.docker.com/compose/compose-file/#variable-substitution

So $${ETCD_PORT_2379_TCP_ADDR}, etc

[dnephin]

Although, I think in your case, you could also use the link name?

environment:
    DATASTORE_URL=http://etcd:2379
    KUBERNETES_URL=http://kubernetes:8080

[alunduil]

Interesting. I'll have to give that a shot.

Why was the new behavior required? It doesn't seem like it's actually of use unless you need to pass through values (which doesn't seem like a case that would come up very frequently).

The release notes also indicate that only specific fields are affected by this change (see quote below). Is it a bug that environment variables are also affected by this?

With the introduction of variable substitution support in the Compose file, any Compose file that uses an environment variable ($VAR or ${VAR}) in the command: or entrypoint: field will break.

[dnephin]

Why was the new behavior required?

Support for variables in the Compose file was one of our most requested features.

It allows users to customize the compose file for different users or environments without having to edit the file directly.

The release notes also indicate that only specific fields are affected by this change

That's my fault. We expected that the only place users would have existing environment variables for the container would be the command and entrypoint fields, but apparently environment could have also contained them.

It's not a bug, every field now supports variables.

[alunduil]

@dnephin Can you provide a summary or links to the requests with use cases for the interpolation? I'm curious as to how people will be using this feature (in case we want to integrate those strategies).

[dnephin]

The PR is #1765, there are ~15-20 linked issues, most of which talk about use cases.

[alunduil]

For future reference, using the service name in the docker-compose file does not work. Also, using the link variables doesn't appear to actually link containers anymore. Is there more information that must be passed with the new docker-compose (or docker-1.9.0) to ensure that links work as written?

[dnephin]

Using the project name as an environment variable works, as long as you set it in the host environment.

Links are networking are different implementations and they don't work together. If you use --x-networking to use the new networking feature it will put the hostname into /etc/hosts, but I don't think it will set environment variables for you, because those have been deprecated for a while now. That's why I was suggesting using the hostname and static port.

[alunduil]

@dnephin where do I pass --x-networking so I can use these magical hostnames?

Are you suggesting that the docker link mechanism is deprecated and using the given environment variables provided by linking will go away? I'm probably just reading your statement incorrectly.

[dnephin]

http://docs.docker.com/compose/env/

http://docs.docker.com/compose/compose-file/#links or
http://docs.docker.com/compose/networking/

both set the hostname (but unfortunately to different things right now #2312)

[dnephin]

As of docker 1.10 these ETCD_PORT_2379_TCP_ADDR link variables are completely removed (they've been deprecated for a while now).

Use the hostname provided by links (in /etc/hosts in docker 1.9, and from dns in docker 1.10).