Incompatible volume bind option for `stack deploy` from compose generated config

[ktnr]

Description

I am generating stack.yml files from compose files to use with docker deploy stack.

traefik-compose.yml

version: '3.8'
services:
  proxy:
    image: traefik:v2.10
    networks:
      - default
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command:
      - --providers.docker
      - --providers.docker.exposedbydefault=false
      - --providers.docker.swarmmode
      - --accesslog
      - --log
      - --api
    deploy:
      placement:
        constraints:
          - node.role == manager

With docker compose -f traefik-compose.yml config > traefik-stack.yml:

name: test-project
services:
  proxy:
    command:
    - --providers.docker
    - --providers.docker.exposedbydefault=false
    - --providers.docker.swarmmode
    - --accesslog
    - --log
    - --api
    deploy:
      placement:
        constraints:
        - node.role == manager
    image: traefik:v2.10
    networks:
      default: null
    volumes:
    - type: bind
      source: /var/run/docker.sock
      target: /var/run/docker.sock
      bind:
        create_host_path: true
networks:
  default:
    name: test-project_default

Then I get:

docker stack deploy -c traefik-stack.yml traefik-test
Creating network test-project_default
Creating service traefik-test_proxy
failed to create service traefik-test_proxy: Error response from daemon: path /var/run/docker.sock is mounted on / but it is not a shared mount.

When I remove

name: test-project

and

      bind:
        create_host_path: true

the stack deploys successfully. Setting create_host_path: false also fails.

The incompatible name property has been raised in moby/moby#44972. But the incompatible volume bind option is not yet documented.

Reproduce

As described above.

Expected behavior

The stack should deploy successfully without errors.

docker version

Client: Docker Engine - Community
 Cloud integration: v1.0.33
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:52:17 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Desktop
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /usr/local/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     /usr/local/lib/docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     /usr/local/lib/docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.12.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 107
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: vcaadxh2nw1fycl74ni0ehb3o
  Is Manager: true
  ClusterID: kihepcl08kumphz63neahi8pq
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 10.0.0.2
  Manager Addresses:
   10.0.0.2:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.10.16.3-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.49GiB
 Name: docker-desktop
 ID: 9d4d71c2-c912-4ecb-a17a-6e57ddea5cdd
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional Info

Curiously, when I generate the stack.yml and deploy with

docker stack config -c traefik-compose.yml > traefik-stack.yml 
docker stack deploy -c traefik-stack.yml traefik-test

or when I outright deploy from the compose file with

docker stack deploy -c traefik-compose.yml traefik-test

the stack deploys successfully.

But this only works if there are no environment variables in the compose file that need to be substituted, like

    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?Variable not set}

which is why directly deploying from compose files is not a convenient option. If there's an environment variable present:

invalid interpolation format for services.proxy.networks.[]: "required variable TRAEFIK_PUBLIC_NETWORK is missing a value: Variable not set"; you may need to escape any $ with another $

[ktnr]

Related? docker/compose#10323

[ktnr]

When I manually add propagation: private or propagation: rprivate

      bind:
        propagation: rprivate
        create_host_path: true

the stack deploys successfully. Doesn't work with slaveor shared.

What is the default option that is used for compose up? Here it says that rprivate is the default, so I guess the same goes for compose up.

[ktnr]

The default bind options are generated here I guess:
https://github.com/docker/compose/blob/ff3984e6093672ede542f842148996b36f7d1fab/pkg/compose/create.go#L939-L972

May be related, it added CreateHostPath: true,: docker/compose#9514

[cpuguy83]

1. create-host-path should be false here, you do not want anything to create docker.sock
2. Probably compose should leave the propagation alone if its not set by the user but seems like they always provide a propagation mode

[ktnr]

1. create_host_path: true does nothing if a file/folder already exists at the specified path
2. The default propagation mode is rprivate, it was previously just not explicitly put in the file by compose config. Adding it was a result of Device Bind Mount Disconnecting from Container Directory for-mac#6317 May it be necessary to achieve compatibility with https://github.com/compose-spec ?

[cpuguy83]

It does nothing if it is there sure, but if it is not there you do not want it to create the path.
So true is an incorrect value here.