Validate Compose file and produce better error messages

[aanand]

There are a lot of ways to confuse Compose with a badly-formed docker-compose.yml. This file should be checked when it is read and good error messages should be produced to help users fix the problem.

This could be done by producing a schema for the file, like was done in #1348.

These sorts of things should be checked:

• Top-level object is a dict
• Each key (a service name) maps to a dict (Improve error when service is not a dict #128)
• String config options (e.g. image) - are numbers allowed? When? Don't accept YAML's eager interpretation of values (e.g. true) which should be quoted (docker-compose does not pass environment variables true/false unchanged #1788)
• List config options (e.g. volumes) either must be a list, or we helpfully wrap single values in a list for you
• String-or-list config options (e.g. command, entrypoint)
• Dictionary config options (e.g. environment)
• Format of various options (e.g. volumes - see Error when specifying permission in volume #260)

References: #117 #127

[wiliamsouza]

fig.yml

mysql:
  image: wiliamsouza/mysql
  ports: "3306:3306"
  volumes:
    - volumes/log:/var/log/mysql
    - volumes/lib:/var/lib/mysql
    - volumes/conf.d:/etc/mysql/conf.d
  environment:
    - MYSQL_ROOT_PASSWORD: rootpass
    - MYSQL_DATABASE: mydb
    - MYSQL_USER: myuser
    - MYSQL_PASSWORD: mypass

The port should be in the format:

  ports: 
   - "3306:3306"

fig up mysql
Recreating dockermysql_mysql_1...
Traceback (most recent call last):
  File "/home/wiliam/.local/bin/fig", line 9, in <module>
    load_entry_point('fig==0.3.2', 'console_scripts', 'fig')()
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/main.py", line 39, in main
    command.sys_dispatch()
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/docopt_command.py", line 21, in sys_dispatch
    self.dispatch(sys.argv[1:], None)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/command.py", line 30, in dispatch
    super(Command, self).dispatch(*args, **kwargs)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/docopt_command.py", line 24, in dispatch
    self.perform_command(*self.parse(argv, global_options))
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/command.py", line 47, in perform_command
    return super(Command, self).perform_command(options, *args, **kwargs)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/docopt_command.py", line 27, in perform_command
    handler(command_options)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/cli/main.py", line 301, in up
    (old, new) = self.project.recreate_containers(service_names=options['SERVICE'])
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/project.py", line 115, in recreate_containers
    (s_old, s_new) = service.recreate_containers()
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/service.py", line 155, in recreate_containers
    (old_container, new_container) = self.recreate_container(c, **override_options)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/service.py", line 178, in recreate_container
    new_container = self.create_container(**options)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/service.py", line 129, in create_container
    container_options = self._get_container_options(override_options, one_off=one_off)
  File "/home/wiliam/.local/lib/python2.7/site-packages/fig/service.py", line 251, in _get_container_options
    all_ports = container_options.get('ports', []) + self.options.get('expose', [])
TypeError: cannot concatenate 'str' and 'list' objects

[jneen]

Have we considered designing a custom syntax that enforces this stuff at a syntactic level? Perhaps something like

web {
  build .
  command python app.py
  link db
  port 8000:8000
}

db {
  image postgres
}

[jneen]

I'm happy to put together a proposed implementation if yall think this is a direction we'd like to go.

[aanand]

Appealing - but personally, I'd much rather maintain a few lines of validation code than a custom language grammar + parser, as flawed as YAML is.

On Sat, Aug 9, 2014 at 11:26 AM, Jeanine Adkisson
notifications@github.com wrote:

I'm happy to put together a proposed implementation if yall think this is a direction we'd like to go.

Reply to this email directly or view it on GitHub:
#129 (comment)

[jneen]

Ok. My feeling is that validation is not going to be anywhere near complete with just a few lines, and parsing is not actually that hard, especially for such a dirt-simple grammar. But it's your call.

[jneen]

Oh, and it'd probably just be a small amount of pyparsing, not a whole flex/bison stack or something, that would indeed be way overkill.

[jneen]

... and in terms of being a tool for humans, we would do well to make the format obvious by design, which yaml doesn't really accomplish. But yes, I will actually stop talking about this now.

[d11wtq]

👎 on introducing a new grammar. It would break backwards compatibility and YAML is a pretty universal format. We actually do things with our fig.yml in our deployment tooling for example. I don't want to have to build a parser into our deployment tooling. YAML is flexible enough that we don't have to update the grammar as we introduce new features. It just requires that we check the data types for the keys we want to use.

[nubs]

As a user of fig, I also would prefer the YAML. In my experience there is very rarely need for something beyond a simple structured configuration using a standard format (JSON, YAML, XML (ugh!), etc.).