Log output missing with docker-compose run

[Evesy]

Description

Using docker-compose run on 2.2.3, log output from the container (either some of it, or all of it) is missing. This issue is not present on 2.2.2
This occurs when running in our CI environment, haven't been able to reproduce on my own machine.

Steps to reproduce the issue:

1. We're running /bin/bash -c "docker-compose run -u 1000 infra ./plan.sh on our CI agent
   2.plan.sh is a bash script inside the docker container that also calls other programs (e.g. terraform)

docker-compose.yml:

version: '3.2'

services:
  infra:
    hostname: infra
    image: "eu.gcr.io/at-artefacts/ci-docker-terraform-gcp:${GO_DEPENDENCY_LABEL_DOCKER_TERRAFORM:-latest}"
    network_mode: host
    entrypoint: /bin/bash
    environment:
      - PROJECT
      - ENV
    privileged: true
    volumes:
      - ~/.config:/home/go/.config
      - ./terraform:/app/terraform

Describe the results you received:
Using 2.2.3 the output is as per below:

Using bucket <>
Cleaning up temporary files...
Symlinking temporary environment files...

The output we do see is directly from echo statements in the executed script. However, the actual logs contain much more, and can be seen via docker logs when running.

We also have seen occurrences where no output is returned.

Describe the results you expected:
Using 2.2.2 the output is not missing anything:

Using bucket terraform-at-signals-platform-nonprod in project at-signals-platform-nonprod
Cleaning up temporary files...
Symlinking temporary environment files...
- cert-manager in modules/iam
- elasticsearch-snapshots in modules/iam
- hot-nodes-20210504 in modules/elastic-pool
- k8-elastic-cluster in modules/container-cluster
- kubernetes-external-dns in modules/iam
- redis-signals in modules/cloud-redis-instance
- velero-backups in modules/iam
- warm-nodes-20210504 in modules/elastic-pool
Initializing modules...

Initializing the backend...
Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
....
....

Additional:
Running docker-compose run with -T looks to, in the cases we're able to easily reproduce, fix the issue too in the latest version. However as mentioned above this hasn't previously been needed on older versions of compose.

Output of docker compose version:

2.2.3

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., 0.0.0+unknown)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b638
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.170+
 Operating System: Alpine Linux v3.14 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 10
 Total Memory: 15.64GiB
 Name: gocd-agent-compose-v2-5888988fb-hcwjf
 ID: GHWC:JLQZ:OWHA:4Q3V:XSSP:VGEJ:CGUK:MUW3:PS57:TE7M:YZHR:IH7X
 Docker Root Dir: /storage/docker/gocd-agent-compose-v2-5888988fb-hcwjf
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine
 Default Address Pools:
   Base: 10.207.212.0/22, Size: 24

Additional environment details:

CI environment

[tommilligan]

Looks like introduced in 7b64399 as a deliberate fix.

I suppose this is bringing the command in line with the documented behaviour, and to standardise it with exec. Still find it surprising that a long-standing behaviour was changed in a patch release though (broke CI for us too)

[ndeloof]

That's indeed sad we had to remove the TTY auto-detection
Long terms, I'd expect we can share run and exec code with docker/cli so we benefit for a more stable and battle-tested codebase to manage such corner-cases, see docker/cli#3436

[Evesy]

Looks like introduced in 7b64399 as a deliberate fix.

I suppose this is bringing the command in line with the documented behaviour, and to standardise it with exec. Still find it surprising that a long-standing behaviour was changed in a patch release though (broke CI for us too)

Nice spot.

So whilst the documentation has always been By default docker compose run allocates a TTY when not specified, looking at the previous code, the actual logic has been that it will autodetect for you, and if you're not running in a TTY, --no-TTY would be appended for you

[Evesy]

@ndeloof To confirm, the course of action now when upgrading to >=2.2.3 would be to explicitly add --no-TTY to our docker-compose run commands when running in our CI environment, which would replicate the old behaviour?

[tommilligan]

So whilst the documentation has always been By default docker compose run allocates a TTY when not specified, the actual logic has been that it will autodetect for you, and if you're not running in a TTY, --no-TTY would be appended for you

Agreed - and the behaviour from docker-compose v1 was auto-allocation, so I didn't see this as a breaking change when upgrading to v2.

To confirm, the course of action now when upgrading to >=2.2.3 would be to explicitly add --no-TTY to our docker-compose run commands when running in our CI environment, which would replicate the old behaviour?

I added --no-TTY yesterday to anything that's a script and uses docker compose run, and this fixed our overnight CI build issues.

[ndeloof]

TTY auto-detection (as a common use-case on CI servers) has been restored by #9258