Between 1.27.0 and 1.27.4 the handling of volume config option "device" has changed

[maaarghk]

Description of the issue

I have a compose file with a section like this:

volumes:
  app_data:
    driver_opts:
      type: none
      device: "${PWD}/../../www/"
      o: bind

On docker-compose 1.27.0 a volume is created like so:

$ docker inspect --format='{{ .Options.device }}' composeproject_app_data
/home/mark/Code/composeproject/ops/dev/../../www/

After upgrading to 1.27.4 the following error is raised after trying to bring the containers up:

ERROR: Configuration for volume app_data specifies "device" driver_opt /home/mark/Code/composeproject/www, but a volume with the same name uses a different "device" driver_opt (/home/mark/Code/composeproject/ops/dev/../../www/). If you wish to use the new configuration, please remove the existing volume "composeproject_app_data" first

In other words, 1.27.4 is resolving the "../.." in the path whereas 1.27.0 is not.

This behaviour should only change with a new compose file definition version.

Of course you will also need to worry about people who just freshly installed 1.27.4 coming in and whinging like me that backwards compatibility was broken between .4 and .5 when you make the fix :) 🏅

Context information (for bug reports)

Output of docker-compose version
Broken:

docker-compose version 1.27.4, build unknown
docker-py version: 4.3.1
CPython version: 3.8.5
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Working:

docker-compose version 1.27.0, build unknown
docker-py version: 4.3.1
CPython version: 3.8.5
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Output of docker version

Client:
 Version:           19.03.12-ce
 API version:       1.40
 Go version:        go1.14.5
 Git commit:        48a66213fe
 Built:             Sat Jul 18 01:33:21 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.12-ce
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.14.5
  Git commit:       48a66213fe
  Built:            Sat Jul 18 01:32:59 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.1.m
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828.m
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Steps to reproduce the issue

1. Have a volume mount specified as per above with relative paths
2. Start with 1.27.0
3. Upgrade
4. Fail to start with 1.27.4

Observed result

Containers don't come up

Expected result

Containers come up

Additional information

Manjaro Linux on kernel 5.9-rc3

[B4RteQPl]

I can confirm that I'm playing with this bug since upgrade to 1.27.4

[LaXiS96]

I believe issue #7893 is somewhat related.

Around compose 1.27.3 or .4 something changed and prevented me from successfully deploying from Windows to a Linux Docker host.

I don't have concrete proof as I decided to just stick to the old version (and it was over 3 months ago), but the situation was like this:

• docker-compose.yml included a volume with device: /mnt/sda3/data
• the error reported after running docker-compose up included something like C:\mnt\sda3\data
  The prepending of "C:" and conversion to Windows-style path separators tells me something is wrong. The /mnt/sda3/data is specific to and valid for the Linux Docker host I was deploying to.

It doesn't look like any recent commit has addressed this problem, I only see "Remove path checks for bind mounts" in the 1.27.4 release changelog but it is not clear where this is reflected in the code.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.

[LaXiS96]

This issue was closed but I don't believe the problem has been fixed yet. I still have to try the latest v1.29.2 but I have done some digging and discovered some things.

The "bad" commit is 8c81a9d, which does "Enable relative paths for driver_opts.device" in v1.27.3.

In v1.29.2 the piece of code changed somewhat:

compose/compose/config/config.py

Lines 477 to 498 in 5becea4

	if 'driver_opts' in config:
	config['driver_opts'] = build_string_dict(
	config['driver_opts']
	)
	device = format_device_option(entity_type, config)
	if device:
	config['driver_opts']['device'] = device
	return mapping
	def format_device_option(entity_type, config):
	if entity_type != 'Volume':
	return
	# default driver is 'local'
	driver = config.get('driver', 'local')
	if driver != 'local':
	return
	o = config['driver_opts'].get('o')
	device = config['driver_opts'].get('device')
	if o and o == 'bind' and device:
	fullpath = os.path.abspath(os.path.expanduser(device))
	return fullpath

but it doesn't seem to have fixed the problem, as it still uses os.path.abspath on the client to expand the path, which will give incorrect results when the "local" filesystem is local to the server/Docker host, not the client/docker-compose.

This looks like an oversight in testing and I don't think it's such an unusual usecase (deploying from a Windows client to a Linux Docker host - NOT Docker Desktop)...
I'd really like an experienced compose developer to chime in... @aiordache could you please comment on this? What was the original need for adding the relative path handling?