Dependent services should not be re-created when re-creating depending services

[viranch]

Consider following docker-compose.yml:

web:
    build: .
    links:
        - db
db:
    image: mysql

I run the services.

$ docker-compose up -d
Creating project_db_1...
Creating project_web_1...
$

When I want to restart (recreate) my web service after some changes, I issue:

$ docker-compose up -d web
Recreating project_db_1...
Recreating project_web_1...
$ 

I would normally expect only my web service to start without interrupting the depending services.

But it doesn't happen so. The dependent services should not be re-created when recreating depending services. Dependent should only be started if not running already.

On the other hand, if I re-create dependent service, the depending service should be re-created, which does not happen.

$ docker-compose up -d db
Recreating project_db_1...
$ 

The web service then loses the link and has no idea of the new db service.

[aanand]

The dependent services should not be re-created when recreating depending services. Dependent should only be started if not running already.

Yes - assuming no configuration has changed for the upstream service, it shouldn't be recreated. I'm working on this in #1345. For now, use --no-deps:

$ docker-compose up -d --no-deps web

On the other hand, if I re-create dependent service, the depending service should be re-created

Arguably, yes, although there should be a flag to prevent it.

[luhn]

I came here to report this bug. I'm running 1.4.0rc2 and I want love the smart-recreate from #1345, but it almost always breaks my applications because it's inverting the dependency graph. This behavior existed before #1345, too, if you upped only a single container at a time.

For example, here's a simple compose file I set up:

a:
  image: nginx

b:
  image: nginx
  links:
    - a

c:
  image: nginx
  links:
    - a

Now, let's say we update a:

$ docker-compose up -d --force-recreate a
Recreating tmp_a_1...

B and C, which are dependent on A, should have been recreated, but they weren't. Now the links from B to A and C to A are broken:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS               NAMES
0cddf85e5d74        nginx               "nginx -g 'daemon of   2 minutes ago       Up 2 minutes        80/tcp, 443/tcp     tmp_a_1
c1d1add241f2        nginx               "nginx -g 'daemon of   3 minutes ago       Up 3 minutes        80/tcp, 443/tcp     tmp_b_1
a40f1198bd7b        nginx               "nginx -g 'daemon of   3 minutes ago       Up 3 minutes        80/tcp, 443/tcp     tmp_c_1
$ docker inspect 0cddf85e5d74 | grep IPAddress  # This is A's IP
        "IPAddress": "172.17.0.64",
        "SecondaryIPAddresses": null,
$ docker exec c1d1add241f2 cat /etc/hosts  # B has the wrong IP address for A
172.17.0.63 c1d1add241f2
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.61 a 00b69811c461 tmp_a_1
172.17.0.61 a_1 00b69811c461 tmp_a_1
172.17.0.61 tmp_a_1 00b69811c461
$ docker exec a40f1198bd7b cat /etc/hosts  # So does C
172.17.0.62 a40f1198bd7b
127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.61 a 00b69811c461 tmp_a_1
172.17.0.61 a_1 00b69811c461 tmp_a_1
172.17.0.61 tmp_a_1 00b69811c461

In a similar vein, if we were to start from scratch and up B, A would be upped too, breaking the link from C to A. Updates will bubble up the wrong way on the dependency graph, breaking all links along the way. I can't emphasize enough the severity of this bug. It will be made especially prominent by the release of the new smart-recreate feature, since a single updated container could break an entire deployment. This should be resolved before 1.4 leaves RC.

Thanks for reading. I appreciate the work you guys are doing with this. I love how simple and straightforward it is. The "cloud" ecosystem needs more software like this.

[aanand]

It will be made especially prominent by the release of the new smart-recreate feature, since a single updated container could break an entire deployment.

Are you talking about this scenario?

$ docker-compose up -d
Creating tmp_a_1...
Creating tmp_b_1...
Creating tmp_c_1...

(...change configuration for a...)

$ docker-compose up -d

Because if so, here's what will happen:

$ docker-compose up -d
Recreating tmp_a_1...
Recreating tmp_b_1...
Recreating tmp_c_1...

No broken links.

If you don't pass a list of services to docker-compose up, it will recreate services whose configuration has changed or which (transitively) depend on services whose configuration has changed.

I don't think we make this clear in the docs, but we should.

[luhn]

Ah, I thought up -d --force-recreate a would behave the same as up -d after having modified A. It does seem like the latter works as intended. I apologize for the mistake.

However, the behavior of up -d --force-recreate a seems wrong, especially since it's breaking my linked containers. It seems like it should behave the same as up -d after having modified A.

[dnephin]

This should be fixed in 1.4.x.

@luhn The default should work for you now, but if you need to force a recreate like that, I think using --no-deps will do what you want.

[viranch]

@dnephin I'm backporting the fix for this issue to older versions (in my fork). Can you point me to the commit(s) that fix this?

[dnephin]

I think you'll want at least #1399, #1960, #1939. Not sure if there are others