docker stack deploy command doesn't support multiple docker-compose.yml

[Sachinbihade]

Here is command which I am firing.

env $(cat .env | grep ^[A-Z] | xargs) $(cat .override_env | grep ^[A-Z] | xargs) docker stack deploy -c docker-compose.yml -c docker-compose.override.yml test_services

Expectation is it should take all base configs from first compose file i.e docker-compose.yml and then environment specific config from second compose file i.e. docker-compose.override.yml
as per the documentation https://docs.docker.com/engine/reference/commandline/stack_deploy/

But when I ran this i get error as:
Error response from daemon: rpc error: code = 3 desc = ContainerSpec: image reference must be provided.
This means its not merging two docker compose files, and picking up just the second one.
FYI. If I run only with first docker compose then it works.
Here are both the files:
docker-compose.yml

version: '3.1'	  
services:
  test1-service::
    image: "${DOCKER_REGISTRY}${DOCKER_APP_PATH}${IMAGE_PATH}"
    environment:
      - XMX_VALUE=${XMX_VALUE}
      - MAX_METASPACE_SIZE=${MAX_METASPACE_SIZE}     
    volumes:
      - ${HOST_LOG_DIR}:${FILE_PATH}
    deploy:
      mode: replicated
      replicas: 3
      placement:
        constraints: [node.role == worker]
      resources:
        limits:
          memory: ${MEMORY_LIMIT}
        reservations:
          memory: 500M
      update_config:
        parallelism: 1
        delay: 30s
      restart_policy:
        condition: on-failure
        max_attempts: 3
        window: 120s
    ports:
      - ${TEST_PORT}:${TEST_PORT}

docker-compose.override.yml

version: '3.1'

services:
 test1-service:
    environment:
      - ${TEST1_PORT}=${TEST1_PORT}
    ports:
      - ${TEST1_PORT}=${TEST1_PORT}

I am expecting both TEST_PORT and TEST1_PORT should get added as port mapping.
But as I mentioned above the command ignores first docker compose, so throws error as image not found.
Thanks,
Sachin

[Sachinbihade]

can any please provide update on this?

[stephane303]

I confirm there is a ploblem with multiple docker-compose.yml files.
If I use the following:

docker-compose.exe -f docker-compose.yml -f docker-compose.deploy.yml config > stack.yml

and then

docker stack deploy --compose-file stack.yml mystack

Everything is working fine.

But I would expect the following to have the same exact behaviour:

docker stack deploy --compose-file docker-compose.yml --compose-file docker-compose.deploy.yml sylvia-edunil

Unfortunately I get an error:

service phpfpm-71-edunil: undefined volume "edltmp/"

as if only one file is used, as described by @Sachinbihade .

[jaskirat8]

I am also facing same issue in 17.09 but not in 18.09

[carlfischerjba]

Here's an example where the first file prints "foo" and the second file overrides the command to print "bar" instead.

docker-compose.yml:

version: '3.5'
services:
  web:
    image: busybox
    command: sh -c "echo foo; sleep infinity"

docker-compose.prod.yml:

version: '3.5'
services:
  web:
    command: sh -c "echo bar; sleep infinity"

Works as expected with docker-compose. The second file correctly overrides the first.

$ docker-compose up
web_1  | foo
$ docker-compose -f docker-compose.yml -f docker-compose.prod.yml up
web_1  | bar

But not with docker stack deploy. The second file appears to have no effect.

$ docker stack deploy -c docker-compose.yml test
$ docker service logs test_web
test_web.1.sz8rgk40qzll@...    | foo
$ docker stack rm test
$ docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml test
$ docker service logs test_web
test_web.1.r8ce2hpk3p1l@...    | foo

Inspecting the container shows that the command from the second file has been appended to the command from the first file instead of overriding it completely. In this case it fails silently but with a different combination of commands we would get a syntax error.

$ docker inspect test_web -f '{{.Spec.TaskTemplate.ContainerSpec.Args}}'
[sh -c echo foo; sleep infinity sh -c echo bar; sleep infinity]

Here's my Docker version.

$ docker version
Client:
 Version:           18.09.5
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        e8ff056
 Built:             Thu Apr 11 04:43:34 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.0
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       aeac9490dc
  Built:            Wed Jul 17 18:14:16 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.5
  GitCommit:        bb71b10fd8f58240ca47fbb579b9d1028eea7c84
 runc:
  Version:          1.0.0-rc6+dev
  GitCommit:        2b18fe1d885ee5083ef9f0838fee39b62d653e30
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[carlfischerjba]

Possibly related to #1582.

[thaJeztah]

I think this was implemented in #569, so let me close this ticket, but feel free to comment if there was something remaining