docker-compose 1.27.3 config depends_on breaks stack deploy

[dfresh613]

Description of the issue

When we apply docker-compose config to a docker-compose file with depends_on, the end result is not deployable by docker stack..

Steps to reproduce the issue

Example docker-compose file:

services:
  test:
    depends_on:
    - test2
    image: a/fakeimage:latest
  test2:
    image: a/fakeimage:latest
version: '3.4'

Use docker-compose -f <example-config> config > new_config

The result is:

services:
  test:
    depends_on:
      test2:
        condition: service_started
    image: a/fakeimage:latest
  test2:
    image: a/fakeimage:latest
version: '3.4'

Now, attempting to deploy that new compose config, docker stack deploy fails:

docker stack deploy -c configout.yml iservices
services.test.depends_on must be a list

This behavior does not occur in 1.26.x or 1.27.2

docker-ce == 19.03.5
docker-compose ==1.27.3

[ulyssessouza]

Hello @dfresh613.

Actually the new output is compliant with the new compose-spec.
Could you please explain your usecase of using the output of docker-compose config as input for docker stack deploy?

[dfresh613]

We have a docker-compose file to deploy an application that may be deployed either as a stack, or with docker-compose, easiest to maintain one config file for both, and consistently apply updates using other compose files docker-compose config -f <file1> -f <file2> config

Regardless our use case, this is a change that doesn't seem like it should have been made in a patch (.3) release.

[samueldeklund]

We are having a similar issue with piping docker-compose config output to docker stack deploy command.

We use several different environment files and select one based on which server we are deploying to, and use the --env-file parameter to inject those values into the yml file. But now that the presence of the deploy key throws an error instead of passing it through (even suppressing the error with --log-level doesn't re-enable output), our workflow is broken.

[crossdot]

I think docker-compose should respect version specified in docker-compose.yml
For 2.* it should be object map with conditions
For 3.* it should be list

[dfresh613]

What I find quite funny, and correct me if I'm wrong-- But it seems this change originally failed the existing unit tests which ensures depends_on matches type list: https://ci-next.docker.com/public/blue/organizations/jenkins/compose/detail/PR-7761/1/pipeline/#step-89-log-867

And then, in the next commit these screaming unit tests were "updated" by deleting them, or changing their validation criteria: fa72078

[SMFloris]

Indeed, we find the same issue. We have multiple compose files (one for prod, one for dev), where we deploy something like this (we make use of the interpolation of docker-compose config):

docker-compose -f docker-compose.prod.yml -f docker-compose.dev.yml config | docker stack deploy

This ensures that we run our containers as close as possible to their prod counterparts locally, the only differences being the cpu allocations, scales, labels and volumes mount.

There are multiple issues that are preventing me from even testing our stack on my laptop (i.e. #7774). I'm going to be working from my laptop in a week from now, but can't even deploy my stack since I have the newer version that came with docker. The main selling point of docker stack and docker-compose is that you can use almost the same files for both. And this has been broken so this should be treated as a regression in my opinion.

@ulyssessouza, as I saw regarding #7774, docker stack doesn't care about compose-spec in their latest 3.9 schema, even though in the compose-spec repository its being advertised that it has an implementation of compose-spec. This is untruthful. One should expect that as more and more people upgrade to Docker 3.0.5 for Mac, many more bugs like this will pop up.

Basically, I hope this was just a roadmap mismatch (i.e. the docker cli team wasn't fast enough to implement the compose spec and you guys were a bit more ambitious). This, thus, becomes a release problem. One should not release a set of tools that should be compatible with each-other that aren't and break your flow and scripts.

[SMFloris]

For everyone having issues with docker-compose config and piping the output into docker stack deploy, there is this open issue ( docker/cli#2365 ) for which I have done a PR. Feel free to post your own use cases there in order for the Docker team to gather more data in order for them to make the right call regarding that open issue.

I did my best to make sure that the configs are merged/interpolated the same way and that the output piped back to docker stack deploy is always valid. Feel free to test that as well if you have the time.

[dfresh613]

@SMFloris that is awesome, I definitely agree that functionality is needed directly from the stack command.

That sounds feasible for a longer term solution- but I would still like to know what the planned next steps are for the current 1.27.x line of docker-compose since:

• This code changed failed unit tests, and is a regression, significantly changing expected behavior
• This backward-compatibility breaking regression was introduced in a patch release (.3), which is not best practice.
• This oversight essentially removed supported functionality in a way that was not formally deprecated, announced, or documented, in a release version that is supposed to be stable.

I believe this should be addressed directly within the 1.27.x line. Namely an updated patch release 1.27.4 which corrects the regression and passes originally defined unit tests covering 1.27.x.

[ndeloof]

IMHO this is not a regression nor a desirable behavior for docker-compose config to remove depends_on set by input. This command has not been intended to act as a pre-processor for docker stack deploy, but to validate and render the actual compose model after applying overrides and interpolations. It's not designed to "convert a v2 file into v3", which actually doesn't make much sense, especially now with the compose-spec having dropped the version field.

If your origin compose file do include a depends_on, then I assume this is on purpose and your application code does rely on dependent services being up before it run, and then won't run correctly by stack deploy - if it does, then depends_on is just not required. Please elaborate if you have another use-case that would make this legitimate.