v2.3.x config yaml output contains ports as strings

[jsoriano]

Description

docker-compose config formats the published port as a string, this can be a problem when parsing this output.

This starts to happen in 2.3.0.

In documentation ports appear as numbers.

Steps to reproduce the issue:

1. Define a project with exposed ports.
2. Run docker-compose config.
3. published port is a string, while target port is a number.

Can be reproduced for example with this minimal docker-compose.yml:

version: "2.3"
services:
  web:
    image: "nginx"
    ports:
      - "8080:8080"

Describe the results you received:

$ docker-compose config
name: foo
services:
  web:
    image: nginx
    networks:
      default: null
    ports:
    - mode: ingress
      target: 8080
      published: "8080"
      protocol: tcp
networks:
  default:
    name: foo_default

Describe the results you expected:

$ docker-compose config
name: foo
services:
  web:
    image: nginx
    networks:
      default: null
    ports:
    - mode: ingress
      target: 8080
      published: 8080
      protocol: tcp
networks:
  default:
    name: foo_default

Additional information you deem important (e.g. issue happens only occasionally):

It seems to happen since 2.3.0, in 2.2.3 the result is the expected one.

Output of docker compose version:

Docker Compose version v2.3.3

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 327
 Server Version: 20.10.7
 Storage Driver: btrfs
  Build Version: Btrfs v4.4
  Library Version: 101
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.0-171-generic
 Operating System: Ubuntu 18.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.44GiB
 Name: voyager
 ID: R7WQ:DRKJ:F2ZK:EC7J:V3RF:4GRC:GPI6:AEDN:6GL4:VU3V:LU2U:4YE5
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

[designermonkey]

This has broken our build system. We run:

(echo "version: '3.8'"; docker compose -f traefik/docker-compose.yaml convert) | docker stack deploy --with-registry-auth --compose-file - traefik
services.server.ports.0.published must be a integer

Our docker desktop version:

Version
4.6.1 (76265)
Engine: 20.10.13
Compose: v2.3.3
Credential Helper: 0.6.4
Kubernetes: v1.22.5
Snyk: v1.827.0

[designermonkey]

Docker compose is now on version 2.4.1, and Docker Desktop at 4.7.1

I am stuck at holding docker desktop at v4.5.0 to prevent this from happening.

I'm not complaining but would love to know if there's any idea on when this will be addressed?

[trajano]

Docker compose is now on version 2.4.1, and Docker Desktop at 4.7.1

I am stuck at holding docker desktop at v4.5.0 to prevent this from happening.

I thinking it may be best to use docker-compose until 21.x.x of Docker comes out , it will also remove that (echo "version:..." hack you had in your build system (nice idea BTW)

[rafipiccolo]

FYI

This is my script to get a swarm compatible compose file from a "docker compose config"

It is now in production and working with latest docker compose (2.6.0)
(It's a shame I had to create a script but now it works nice)

Docker compose -f swarm.yml config | node reparecompose.js | docker stack deploy --prune --with-registry-auth -c - swarm

The reparecompose.js :

import fs from 'fs';

let s = fs.readFileSync(0, 'utf8');
s += `version: "3.7"\n`;
s = s.replace(/published: "(\d+)"/g, "published: $1");
s = s.replace(/^name: .*\n/, "");

console.log(s);

It does only 3 changes in the file.
Did you notice that other changes were necessary to have a working stack. for me that was enough.

[danielporto]

This is really annoying. Any clean fix available or must I use an older version?

[designermonkey]

How do we get this to be a priority for the docker dev team?

I upgraded today to 4.11.1 and it is still an issue, so again, reverting to 4.5.0.

I'm getting closer and closer to giving up with Docker.

[glours]

Hey 👋
Compose is strictly following the Compose Specification here.
The published port(s) can be a port range that's why a string is used instead of a uint32 for the target attribute
Mirantis now owns Swarm and the stack CLI, the best option here is to ask them:

• if they using compose-go and are up to date.
• if they can support a string value for the published port

Docker Compose and Stack have 2 different purposes and should not be use together, even if they have been sharing the same file format at one point.
There is an open issue for the support of the Compose specification in the stack command

[rafipiccolo]

@designermonkey, @danielporto i provided a script in this thread to "make docker config great again"

The other viable way is to set your env from your .env file before deploying.
This completely allows you to get rid of docker compose config.
From my understanding, this is the expected way for docker people.
found it here : moby/moby#29133

# put all .env in the real environment
export $(cat .env) > /dev/null 2>&1; 
# run stack, knowing that all variables are going to be populated by current env
docker stack deploy -c docker-compose.yml STACK_NAME

[designermonkey]

Being a simple end user, I didn't know that different aspects of the same bundled software are owned by different people. Thanks @glours for letting us know that.

Docker Compose and Stack have 2 different purposes and should not be used together

I can't agree with this however as up until very recently they were completely compatible and just ignored each other's differences. It's very stressful to be using what people consider a single application 'docker' in production and it not even be compatible with itself.

Also, thanks @rafipiccolo I will have to investigate that idea more.

[jsoriano]

Hey @glours, thanks for your reply.

Mirantis now owns Swarm and the stack CLI, the best option here is to ask them:

I don't use swarm. The issue happens with docker-compose as downloaded from https://github.com/docker/compose/releases, starting on 2.3.0.

Compose is strictly following the Compose Specification here.

I find this a bit confusing. This specification doesn't say anything about the type of published, and even in these examples they appear as numbers (an unquoted number in yaml is a number, not a string). These discrepancies make difficult to build software or CI systems that rely on different versions of docker-compose.

If current one is the expected behaviour, I think there are still some inconsistencies:

• In the long form, docker-compose config returns target as number, but published as string. Is target supposed to be a number while published is a string?
• In most documents I have been able to find, including the Compose Specification, published appears as number.

Using port ranges would be a reason to use strings instead of numbers, but in this case docker-compose decomposes the configuration in multiple ports, with the same discrepancies. For example for a port definition like 8080-8081:8080-8081, docker-compose config returns this:

    ports:
    - mode: ingress
      target: 8080
      published: "8080"
      protocol: tcp
    - mode: ingress
      target: 8081
      published: "8081"
      protocol: tcp