-
Notifications
You must be signed in to change notification settings - Fork 5.7k
Description
Description
Compose watch mode may currently stop services that link other services without ever restarting:
- The idea is that a service (think a front proxy) links any another service (think a backend).
- Compose watch mode will correctly identify that the backend service needs to be recreated/restarted (e.g. using
action: rebuild). - It will correctly stop the front service before stopping the backend service.
- It will correctly restart the backend, but fails to restart the front service afterwards.
- The application is now in a broken state ("half up"), the backend can no longer be reached without a front proxy.
I've specifically encountered this when using legacy links between services, but the same behavior can be reproduced when using depends_on with restart: true. It's my understanding legacy links would basically act like depends_on with restart: true, despite restart: false being the default otherwise.
I think it's debatable if the front proxy service needs to be restarted in this example. In either case, it should either be restarted completely or not restarted at all. I don't think stopping the service without restarting it again would ever be intended behavior.
The problem occurred as part of a bigger application using multiple services, but I've been able to reproduce this in a most basic compose.yaml file with no external dependencies except a public image. The resulting file may look a bit contrived, but I hope this should be enough to reproduce this without any specific service configurations.
Possibly related, I've checked docker compose up -a reports the affected services as "exited". Using docker compose restart backend does restart the backend and the affected services as expected. However, the running docker compose up --watch session will not pick up the affected services again. This appears to work fine if I do not trigger a rebuild first.
I've checked for similar reports and stumbled upon #11695 and #11813, but believe they may point to different, but possibly related, issues.
The recent changes regarding watch mode and watching multiple services as implemented in #12469 and others have been a real game-changer for me personally and a larger number of projects I'm working on, so keep up the great work! 👍
Steps To Reproduce
Basic reproducer:
services:
nok:
image: httpd:2.4-alpine
links:
- backend
backend:
build:
dockerfile_inline: |
FROM httpd:2.4-alpine
develop:
watch:
- path: compose.yaml
action: rebuildReproducer showing two broken services + 2 working ones as possible workarounds:
services:
nok:
image: httpd:2.4-alpine
links:
- backend
same:
image: httpd:2.4-alpine
depends_on:
backend:
condition: service_started
restart: true
ok:
image: httpd:2.4-alpine
depends_on:
- backend
alright:
image: httpd:2.4-alpine
depends_on:
backend:
condition: service_started
restart: false
backend:
build:
dockerfile_inline: |
FROM httpd:2.4-alpine
develop:
watch:
- path: compose.yaml
action: rebuildStart watch mode in foreground:
docker compose up --watchChange compose.yaml file to trigger rebuild:
touch compose.yamlLog output shows backend service is correctly rebuilt, but affected front proxy services stop:
⦿ Rebuilding service(s) ["backend"] after changes were detected...
⦿ service(s) ["backend"] successfully built
nok-1 | [Thu May 08 13:30:52.184586 2025] [mpm_event:notice] [pid 1:tid 123450472430344] AH00492: caught SIGWINCH, shutting down gracefully
same-1 | [Thu May 08 13:30:52.189356 2025] [mpm_event:notice] [pid 1:tid 130455114877704] AH00492: caught SIGWINCH, shutting down gracefully
nok-1 exited with code 0
same-1 exited with code 0
backend-1 | [Thu May 08 13:30:53.516278 2025] [mpm_event:notice] [pid 1:tid 128908057438984] AH00492: caught SIGWINCH, shutting down gracefully
backend-1 exited with code 0
backend-1 has been recreated
You can see both nok and same are stopped without restarting.
Compose Version
Docker Compose version v2.36.0
Docker Environment
Client:
Version: 26.1.3
Context: default
Debug Mode: false
Plugins:
compose: Docker Compose (Docker Inc.)
Version: v2.36.0
Path: /usr/local/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 33
Running: 11
Paused: 0
Stopped: 22
Images: 1537
Server Version: 26.1.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version:
runc version:
init version:
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-59-generic
Operating System: Ubuntu 24.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 30.54GiB
Name: me
ID: 519acebf-0404-4fc7-a89d-afcb3bcf9b15
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Anything else?
No response