Add native binary support

[ltitanb]

Currently Commit Boost relies heavily on Docker to launch and organize the different modules. This has the advantage of better cross-platform support, sandboxing of modules, and easier integration for node operators which already use docker or k8s. However, this tights us closely to Docker and the Docker API, which not all operators may want to use.

We need to explore how to support a similar setup without using Docker, by running native binaries instead.

On linux there are number of internal utilities that can achieve similar sandboxing as docker (and which Docker uses under the hood):

• seccomp filters to fine grain syscalls permissions
• PID/NET/MNT namespacing to sandbox processes, network and file system
• control groups (cgroups) to limit memory and disk use
• user permissions

More research is needed on how to best use these and if there are alternative ways to do it, and how to achieve a similar setup on other platforms eg Windows

[orbmis]

Personally I can see that using docker-compose tightly couples the solution to docker, which is a bit of a drawback, but I don't see it as a massive drawback tbh. I have a background in software development, and I would have always preferred using docker/k8s over other solutions.

Docker is well understood by all devs and works across all platforms, and docker-compose is a neutral declarative format, which can be used to derive k8s manifests, Helm charts, CFN templates, terraform etc. if needed. I think that supporting docker-compose while also potentially supplying Helm charts could be really useful to a lot of operators.

Using binaries means you will have to make it really easy for devs to compile everything in case they need to, and/or you will also need to compile and ship binaries for well known platforms with every release, which is a bit of a headache for the team that maintains the software.

[aimxhaisse]

Related: #81 (comment)

[Akagi201]

I use systemd to manage all my services
I found that systemd support sandboxing. These may be useful
redis harden example: https://github.com/krathalan/systemd-sandboxing/blob/master/redis.service.d/hardening.conf
https://wiki.archlinux.org/title/User:NetSysFire/systemd_sandboxing
https://wiki.debian.org/ServiceSandboxing

[ltitanb]

Thanks all for your feedbacks! I think we can proceed as follows: the binaries can be published along the docker images (shouldn't be too much of an effort since we already have the workflow setup). These binaries will not have any additional security measure, ie. they will be the same as the images run in docker. The expectation is that node operators will have their own internal setup/restrictions to use.

Apart from publishing the images, what is left is tidying up and documenting the configs and env variables that are set by commit-boost init. Currently we have a hybrid approach in that each container uses both of the toml config and env variables. I think we can keep this and leave as env variables only those that are generated dynamically by commit boost and are not directly specified in the toml.