Fig data-only containers

[ajrouvoet]

Using this simple configuration:

data:
  image: base/arch
  volumes:
    - /var/lib/mysql
  command: true

mysql:
  build: .
  volumes_from:
    - data

I would expect that the mysql data is persisted on the data-only container 'data'. But writing anything to /var/lib/mysql is NOT persisted across runs. Doing this with plain docker does work.

Am I overlooking something, or is this either a bug or a feature of fig?

The fig commands I'm trying to use are:

fig run mysql init
fig run mysql start

The mysql image has an entrypoint that takes init and start command. The first one populates /var/lib/mysql, the second one starts the mysql server with /var/lib/mysql as it's datadir. Both run, but the second one fails because /var/lib/mysql is empty.

The following is a minimal example of it failing:

fig run mysql /bin/bash -c "touch /var/lib/mysql/testfile && ls -l /var/lib/mysql"
fig run mysql /bin/bash -c "ls -l /var/lib/mysql"

Where the first ls -l shows that the file was created, where as the second one returns empty.

Using docker 1.3.0 and fig 1.0.0

[dnephin]

Could you include the fig commands you're using as well? I believe I've seen this work

[ajrouvoet]

Ofcourse, added both my goal and the minimal test. Thank you for taking a look so soon!

[ajrouvoet]

The minimal example can be made more minimal ofcourse, since you don't really need the mysql image. Replacing the build: . with image: base/arch would work as well.

[dnephin]

I think I was able to reproduce this. I had to run fig up -d data to get it to recreate the container, otherwise it will just use the last container (which didn't have the volume). After doing that it worked.

I think docker will also silently ignore volumes if the directory doesn't exist in the container.

I ran docker inspect <container id> on the containers to see which volumes were attached.

[mhubig]

I had the same issues, updating to 1.0.1 fixed it ...

[ajrouvoet]

Confirming that @dnephin's proposed solution worked for me as a workaround.

[TwoD]

I'm having a similar issue as fig rebuilds my db container every time I fig up some other container linking to it. However, it does not rebuild any containers linked to by db in turn.

I've got all my database files mounted as host volumes so I don't really lose any data when this happens, but it's still annoying as other containers rely on db too.

[wid]

On fig up the doc tells
By default if there are existing containers for a service, fig up will stop and recreate them (preserving mounted volumes with volumes-from), so that changes in fig.yml are picked up.

I think we should add a parameter to fig.yml: no-recreate. This parameters should tell that fig up must never recreate this container.

[SvenDowideit]

please don't use --no-something style negative flags - --no-recreate=false is hard to write about - and yes, it will happen :)

[TwoD]

A parameter like that would not fix the underlying bug, which assumes linked containers should be recreated too when specifying only one or a few containers to up. What would be the reason to do that?

I get that Fig recreates changed containers, and/of those you specify in the fig up command, but why recreate any other containers those are linking to when they don't need to? Especially without also recreating other containers also linking there. (Please don't make it do that, it would be even worse and make fig up servicename pointless.)

That makes no sense and breaks connections from other existing containers linked to the same container as those are not recreated. Recreating one website container forcibly recreates my DB container and thus breaks all other sites or node using the same DB containers until I manually recreate those too.

[dnephin]

This issue is quite similar to #447, I've proposed a solution there (which doesn't use negative flags).

[wid]

Actually I don't think #447 is the same need but I agree on negative flags.

Maybe we could tag a container as "permanent: true" or "data-container: true" so that this container is never changed or re-created by fig and require manual docker action.