[BUG] "TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'" in docker_container.running

[Tangresh]

Description
The docker_container and docker_network state modules encounter an exception in the HTTP library code:

----------
          ID: prometheus_network
    Function: docker_network.present
        Name: prometheus
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/opt/saltstack/salt/extras-3.10/docker/api/client.py", line 214, in _retrieve_server_version
                  return self.version(api_version=False)["ApiVersion"]
                File "/opt/saltstack/salt/extras-3.10/docker/api/daemon.py", line 181, in version
                  return self._result(self._get(url), json=True)
                File "/opt/saltstack/salt/extras-3.10/docker/utils/decorators.py", line 46, in inner
                  return f(self, *args, **kwargs)
                File "/opt/saltstack/salt/extras-3.10/docker/api/client.py", line 237, in _get
                  return self.get(url, **self._set_request_timeout(kwargs))
                File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
                  return self.request("GET", url, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
                  resp = self.send(prep, **send_kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
                  r = adapter.send(request, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
                  resp = conn.urlopen(
                File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
                  response = self._make_request(
                File "/opt/saltstack/salt/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
                  conn.request(
              TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
              
              During handling of the above exception, another exception occurred:
              
              Traceback (most recent call last):
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/state.py", line 2381, in call
                  ret = self.states[cdata["full"]](
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1245, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1260, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1293, in wrapper
                  return f(*args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/states/docker_network.py", line 524, in present
                  network = __salt__["docker.inspect_network"](name)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1245, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1260, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/modules/dockermod.py", line 5376, in inspect_network
                  response = _client_wrapper("inspect_network", network_id)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/modules/dockermod.py", line 438, in wrapper
                  __context__["docker.client"] = _get_client(timeout=timeout, **kwargs)
                File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/modules/dockermod.py", line 398, in _get_client
                  ret = docker.APIClient(**client_kwargs)
                File "/opt/saltstack/salt/extras-3.10/docker/api/client.py", line 197, in __init__
                  self._version = self._retrieve_server_version()
                File "/opt/saltstack/salt/extras-3.10/docker/api/client.py", line 221, in _retrieve_server_version
                  raise DockerException(
              docker.errors.DockerException: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'
     Started: 09:56:24.795515
    Duration: 34.378 ms
     Changes:

Setup

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD
• classic packaging
• onedir packaging
• used bootstrap to install

Ubuntu 22.04 VM in Virtualbox, setup with Vagrant
Salt version 3006.4 installed through official Repo

Steps to Reproduce the behavior

prometheus_network:
  docker_network.present:
    - name: prometheus

Docker version 24.0.7 is installed through their official package repo.
The state is executed through salt-call state.apply

Expected behavior
Docker network should be present. No python exceptions.

Versions Report

salt-call --versions-report

Salt Version:
          Salt: 3006.4
 
Python Version:
        Python: 3.10.13 (main, Oct  4 2023, 21:54:22) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: 5.0.3
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.13.12
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: ubuntu 22.04.2 jammy
        locale: utf-8
       machine: x86_64
       release: 5.15.0-88-generic
        system: Linux
       version: Ubuntu 22.04.2 jammy

The salt master is version 3005.3

Additional context

The problem does NOT exist with Salt 3006.3.

The exceptions seem to be related to these issues and changes:

• 5c236ea
• urllib3 v2 incompatibility docker/docker-py#3113
• Make compatible with requests 2.29.0 and urllib3 2.0 docker/docker-py#3116

Output of /opt/saltstack/salt-pip list:

Package            Version
------------------ ---------
certifi            2023.7.22
cffi               1.14.6
charset-normalizer 3.2.0
cheroot            8.5.2
CherryPy           18.6.1
contextvars        2.4
cryptography       41.0.4
distro             1.5.0
docker             5.0.3
idna               2.8
immutables         0.15
importlib-metadata 6.0.0
jaraco.classes     3.2.1
jaraco.collections 3.4.0
jaraco.functools   2.0
jaraco.text        3.5.1
Jinja2             3.1.2
jmespath           1.0.1
looseversion       1.0.2
MarkupSafe         2.1.2
more-itertools     5.0.0
msgpack            1.0.2
packaging          22.0
pip                22.3.1
portend            2.4
psutil             5.8.0
pycparser          2.21
pycryptodomex      3.9.8
pyOpenSSL          23.2.0
python-dateutil    2.8.1
python-gnupg       0.4.8
pytz               2022.1
PyYAML             6.0.1
pyzmq              23.2.0
relenv             0.13.12
requests           2.31.0
rpm-vercmp         0.1.2
salt               3006.4
setproctitle       1.3.2
setuptools         65.7.0
six                1.16.0
tempora            4.1.1
timelib            0.2.5
urllib3            2.0.6
websocket-client   1.6.4
wheel              0.41.2
zc.lockfile        1.4
zipp               3.6.0

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[lkubb]

docker/docker-py#3113

This is an issue with the docker library, not Salt.

[Tangresh]

I had the impression that docker-py was shipped with Salt, since /opt/saltstack/salt/salt-pip list printed a version for the docker package. Apparently that is not the case and I don't understand how docker-pip works.

The problem can be avoided by just installing a newer version of docker-py.