Add data volumes for the stackstorm container

[warrenvw]

The stackstorm container doesn't survive a make down and make up intact. In particular, any packs you install will be missing from /opt/stackstorm/packs, yet exist in the databases because we persist storage used by the dependent services.

At a minimum, we should add data volumes for /opt/stackstorm/packs.

What about adding:

• /etc/st2
• /etc/mistral

Should any of these NOT be created? What other volume related changes should we make now?

[shusugmt]

These too:

• /opt/stackstorm/configs
• /opt/stackstorm/virtualenvs

I'm not sure adding those directories under /etc to VOLUME is a right approach. If you want to customize those, then you can

• place the config file outside the container, then bind mount those files
• or, prepare the script which modifies those configurations and place that into /st2-docker/entrypoint.d/customize-config.sh (support for /entrypoint.d/ is not yet implemented though: see Roadmap #22)

Either way, you can put them under git and have control of it.

[cognifloyd]

I think that, by default, all of the containers created with docker-compose should have the same behavior. right now Mongo and Postgres data is preserved (due to volumes) while stackstorm data is not (except the logs). Then, allow customization of the docker-compose setup as needed.

By default then, preserve these in volumes:

• /opt/stackstorm/configs
• /opt/stackstorm/packs
• /opt/stackstorm/virtualenvs

Regarding the /etc files. These ones would need to be auto-bound (in the default case, disabled via the customize-config.sh or similar):

• /etc/st2/htpasswd
• /etc/st2/st2.conf
• /etc/st2/keys/* <- candidate for mount? Or is there only ever the one file: datastore_key.json
• /etc/mistral/mistral.conf
• /root/.st2/config
• /root/.st2/token-st2admin - Either save this, or make sure it gets regenerated on each container start

Things that I would guess don't need auto-bind/mount, because they'll just always be the same in the image (How often do people change them anyway?):

• /etc/st2/logging.*.conf
• /etc/st2/syslog.*.conf
• /etc/mistral/logging.conf
• /etc/mistral/policy.json
• /etc/mistral/wf_trace_logging.conf

Other things I wonder about (and don't know enough about to suggest whether they might need a bind mount to preserve data across docker-compose down/up:

• /opt/stackstorm/runners/ <- people don't install new runners very often, right? yet: Pluggable runners st2#2824
• /opt/stackstorm/exports/ <- maybe just a note that exports are't preserved is good enough?

[cognifloyd]

I'm adding this in a docker-compose.override.yml for now:

version: '3'

services:
  stackstorm:
    volumes:
      - ~/st2config/etc/st2:/etc/st2
      - ~/st2config/etc/mistral:/etc/mistral
      - ~/st2config/etc/ssl/st2:/etc/ssl/st2
      - ~/st2config/etc/nginx/conf.d:/etc/nginx/conf.d/
      - ~/st2config/opt/stackstorm/runners:/opt/stackstorm/runners
      - ~/st2config/opt/stackstorm/exports:/opt/stackstorm/exports
      - stackstorm-pack-volume:/opt/stackstorm/packs
      - stackstorm-venv-volume:/opt/stackstorm/virtualenvs

volumes:
  stackstorm-pack-volume:
  stackstorm-venv-volume:

I used a volume for packs because I won't manually change things there. Everything else I might want to make manual changes, and so I want easier access.

Note, adding to the earlier list, I added these (to change the ssl cert):

• /etc/ssl/st2
• /etc/nginx/conf.d

The big gotcha with doing this, is I have to let the stackstorm image populate all of those first, then copy them into my local folders (making sure permissions don't get messed up), and then add the override file before finally bringing the containers down and up to have the mounts take effect.

I really hope there can be a much more automatic way to allow saving all the stackstorm data between container rebuilds. That could be through a customizable entrypoint or whatever. Maybe on first boot, the extra volumes/mounts are mounted somewhere temporary, then data gets copied into them, then some signal file gets put somewhere (where?) so that the copying only happens the first time. Then, the container's mounts get updated to have them in the proper place. Can an entrypoint even do something like that? Is there such thing as an overlay mounted volume?

Hmm. Maybe something like this: moby/moby#10947 (comment)
That doesn't sound like something an entrypoint customization would be able to do (I'm very new to docker however, I'm not familiar with the limits of entrypoint, I'm guessing it is only run inside the container when it starts or gets created). Maybe this would be a new entry in make? make preserve-data?

[johnarnold]

+1 this. I use a data volume for Jenkins similarly with good results. Easy to rebuild the image and redeploy without nuking all state.