logs doesn't exit if the container is already stopped

[dnephin]

Related to #1858 (cc @aanand, @mnowster since this just came up)

If we attach the log printer to a container that has already exited, it never gets the "end of log" condition and docker-compose logs doesn't exist (the same is true for up).

In a way, this is related to the problem described in #1115, where logs for containers created by scale are not picked up.

I think we can fix both of those issues by adjusting the way we find containers to log. Instead of sending in a list of containers, we can do something like this:

1. start a thread which tails events (related to Proposal: docker-compose events #1510)
2. when it receives a "create" event that matches the service name we want to print logs for, create a new thread to consume logs for that container
3. (in the case of up) start the containers
4. proceed with log tailing until all containers have exited (which will change the exit condition as well I think)

[thaJeztah]

May be completely off the mark here, but wasn't this the intended behavior? I.e., stay attached to the stopped container's log stream, and start outputting once the container is started.

I just dropped in here, so I may be misunderstanding the issue 😊

[aanand]

This is an interesting idea, and it would be great to solve the problem of picking up new containers.

It's a bit odd to have one thread both attaching to and starting containers, but that's the only way I can see to avoid the race condition (if attaching and starting were happening in different threads, synchronising them would be a nightmare).

@thaJeztah The issue is that the order of API calls is currently create, start, attach. If a container exits "immediately", i.e. before we've attached to it, up will hang. This is going to happen a lot more now that up waits for all containers before exiting.

I suppose in non-detached mode, rather than having a listener thread, we can do it all synchronously:

1. run the same command as in interactive mode to (re)create the containers
2. start the containers
3. exit

[aanand]

Thinking about it, the race condition seems fundamentally inescapable: if I've got docker-compose up running in one terminal, and I do a docker-compose scale in another, then the latter process can always create and start a container that has exited by the time the first process has gotten around to attaching to it.

I don't know what we can do about that. Worse, it's probably a common scenario - a container could exit immediately for any number of reasons.

[dnephin]

@aanand If we start tailing the event stream before we start any containers I believe that it's ok if the container has exited (as long as it hasn't been removed). The logs will still be there on exit.

We'll know to attach to the log stream when we get a start event, and we'll know to disconnect when we get a stopped event.

As long as the event stream is properly ordered (which I believe it is), we have a correct sequence of events we can operate on, without any race conditions.

I guess there is always the case where we run docker-compose logs and we want to view logs for existing containers. There's still a race if the container starts and stops between the time we start listening to events and the time we query for all existing containers. In that case it's probably less critical because we didn't actually ask any containers to start. It's not any different from if we had run docker-compose logs after the container finished running.