[BUG] "pip install docker-compose" fails on debian 12

[florianschroen]

Description

docker-compose python lib can not be installed on debian 12.

latest version: docker-compose 1.29.2

Error

[...]/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`

the full error trace can be found on the "steps to reproduce" part.

Steps To Reproduce

prepare debian 12 docker container

$ docker run --rm -ti debian:12
root@452b3b6bb194:/# apt update
root@452b3b6bb194:/# apt install python3 python3-pip python3-venv

try install the "normal way"

root@452b3b6bb194:/# pip install docker-compose
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

reference for this: https://forum.ansible.com/t/breaking-changes-with-pip-and-system-packages/2177

try install in venv

root@452b3b6bb194:/# python3 -m venv pyenv
root@452b3b6bb194:/# . pyenv/bin/activate
(pyenv) root@452b3b6bb194:/# pip install docker-compose
Collecting docker-compose
  Downloading docker_compose-1.29.2-py2.py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.8/114.8 kB 1.7 MB/s eta 0:00:00
Collecting PyYAML<6,>=3.10
  Downloading PyYAML-5.4.1.tar.gz (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 kB 5.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!
      
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
      
              This deprecation is overdue, please update your project and remove deprecated
              calls to avoid build errors in the future.
      
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
      
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/pyenv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/pyenv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/pyenv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 318, in run
          self.find_sources()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 326, in find_sources
          mm.run()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 548, in run
          self.add_defaults()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 586, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 113, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/tmp/pip-build-env-y1jupotd/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

2nd try: install without venv, forcing possible break of system packages

(pyenv) root@452b3b6bb194:/# deactivate            
root@452b3b6bb194:/# PIP_BREAK_SYSTEM_PACKAGES=1 pip install docker-compose
Collecting docker-compose
  Using cached docker_compose-1.29.2-py2.py3-none-any.whl (114 kB)
Collecting PyYAML<6,>=3.10
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!
      
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
      
              This deprecation is overdue, please update your project and remove deprecated
              calls to avoid build errors in the future.
      
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
      
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/command/egg_info.py", line 318, in run
          self.find_sources()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/command/egg_info.py", line 326, in find_sources
          mm.run()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/command/egg_info.py", line 548, in run
          self.add_defaults()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/command/egg_info.py", line 586, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/command/sdist.py", line 113, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/tmp/pip-build-env-m5fxnal8/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Compose Version

No response

Docker Environment

$ docker info
Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 22
  Running: 1
  Paused: 0
  Stopped: 21
 Images: 230
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-88-generic
 Operating System: Linux Mint 21
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.44GiB
 Name: machine
 ID: 2P4G:3LEB:HHON:PXG3:WWD6:2ZWR:4CS4:34AY:WT4F:Q6QO:ZRHO:MHCZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: foo
 Experimental: false
 Insecure Registries:
  192.168.6.10:5000
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[Jack-Dane]

I believe this issue is mentioned here: yaml/pyyaml#724

The issue doesn't occur with a lower version of PyYaml (5.3.1) that is still supported by docker-compose. So a workaround would be:

pip install pyyaml==5.3.1
then:
pip install docker-compose

But please be aware of the vulnerability in PyYaml before version 5.4: https://www.cvedetails.com/cve/CVE-2020-14343/. (So follow at your own risk).

[Jack-Dane]

There has been a workaround added for 6.0.1. A fix would be to support PyYaml 6.0.1, as it doesn't include the vulnerability.

yaml/pyyaml#702

I have just seen this duplicate GitHub issue, it is unlikely this will get resolved because the Python docker-compose package is EOL. #11140

[thaJeztah]

Yes, v1 is EOL; docker compose v2 is installed as part of the docker cli, and from your docker info looks to be installed already;

compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

You can run docker compose (two words) instead of docker-compose to use compose v2

[florianschroen]

Thank you guys!

Your information pointed me the right direction for my inital problem via ansible.

The reference ansible-collections/community.docker#672 (comment) helped with a workaround until community.docker v3.5.0 gets released. (which should contain module docker_compose_v2 module).

[rraymondgh]

I bit the bullet and shifted to using shell commands for docker compose. I tried multiple versions of python for building venv pip install docker-compose nothing worked. Looking forward to a release that works

[thaJeztah]

Compose v1 is EOL so no new release for the python version will be made.

It's possible to run compose "standalone" by making an alias to the docker-compose CLI plugin; for example;

sudo ln -s  /usr/libexec/docker/cli-plugins/docker-compose /usr/bin/docker-compose

See https://docs.docker.com/compose/install/standalone/