[enhancement]: more flexible timing for `write-files` module

[cdepillabout]

Enhancement

As a follow-up to #916, it feels like it would be nice to have more flexible timing for when the write-files module is run.

For instance, I'd like to use write-files to write a file to a filesystem that is mounted using the mount module, but before runcmd is called. However, by default, write-files happens before mount. So the new filesystem will be mounted over-top of the file I just wrote.

I would suggest moving the write-files module to happen after mount, but that would break backwards compatibility, and also not work for the use-case of a user writing files like /etc/fstab that get taken into consideration in the mount module.

It feels like it would be nice to have more flexibility for the end-user to choose exactly when given files get written. Right now, with #916, there is basically one write-files hook that happens in the beginning of the boot process, and one write-files that happens at the very end of the cloud-init process. It would be nice to have a baked-in way of writing files at other points in the cloud-init process.

[cdepillabout]

In order to solve my own problem of "I want the write-files module to be executed after the mount module`, I'm using a cloud-init userdata file that explicitly sets the module order:

my-userdata.yaml:

cloud_init_modules:
  - migrator
  - seed_random
  - bootcmd
  - growpart
  - resizefs
  - disk_setup
  - mounts
  - write-files
  - set_hostname
  - update_hostname
  - update_etc_hosts
  - ca-certs
  - rsyslog
  - users-groups
  - ssh

If you compare this to the default list of cloud_init_modules, you can see that in my modified version, write-files comes after mounts, so I'm able to use write_files to write files to mounted filesystems:

cloud-init/config/cloud.cfg.tmpl

Lines 79 to 106 in 06a1475

	# The modules that run in the 'init' stage
	cloud_init_modules:
	- migrator
	{% if variant not in ["netbsd"] %}
	- seed_random
	{% endif %}
	- bootcmd
	- write_files
	{% if variant not in ["netbsd", "openbsd"] %}
	- growpart
	- resizefs
	{% endif %}
	{% if not is_bsd %}
	- disk_setup
	- mounts
	{% endif %}
	- set_hostname
	- update_hostname
	- update_etc_hosts
	{% if variant in ["alpine", "photon"] %}
	- resolv_conf
	{% endif %}
	{% if not is_bsd or variant not in ["photon"] %}
	- ca-certs
	{% endif %}
	- rsyslog
	- users_groups
	- ssh

[TheRealFalcon]

For instance, I'd like to use write-files to write a file to a filesystem that is mounted using the mount module, but before runcmd is called.

You reference the new write_files_deferred module, but have you tried using it? Runcmd is a little weird in that it doesn't actually run the commands provided; it writes out a script to be run by the scripts_user module. That comes after write_files_deferred, so it may work for you as-is.

Beyond that, as you've shown here, you can already re-order modules in /etc/cloud/cloud.cfg. Is there additional functionality beyond that that you'd like to see?

[cdepillabout]

You reference the new write_files_deferred module, but have you tried using it? Runcmd is a little weird in that it doesn't actually run the commands provided; it writes out a script to be run by the scripts_user module. That comes after write_files_deferred, so it may work for you as-is.

Oh, I see. Thanks, I didn't realize this! Although now that I look in the docs for runcmd, it definitely does say this...

Beyond that, as you've shown here, you can already re-order modules in /etc/cloud/cloud.cfg. Is there additional functionality beyond that that you'd like to see?

I haven't actually tried it yet, but it looks like you're correct that write_files_deferred does work for my exact use-case.

I think one could still argue that it would be useful to have more flexible timing for write-files. For instance, I could imagine a user wanting a version of write_files that runs after mounts, but before users-groups. However, I don't personally have a use for this functionality (at least yet).

I can understand if you'd want to close this as wontfix.

[TheRealFalcon]

Since there are no plans to change this behavior, I'm going to close this.