Allow module to take-over entire directory

[erik-smit]

I sometimes run into problems with unmanaged resources, either by

• configfiles supplied by packages, but not used by me
• configfiles of previously managed resources.

So I want to try to make all my used modules purge everything unmanaged in the configdir.

In my head, providing a copy of the distro configs makes sense.

One thing I am not sure about:

If a feature like this is accepted by you, would it make sense to use the source_dir mechanics for this, or would it make sense to invent a new variable for this particular purpose. Like $manage_configdir, which could be implemented by all applicable modules.

For instance, I've done something similiar with Bacula: https://github.com/rharrison10/rharrison-bacula/pull/3/files

But there it is easier, because 100% of the configfiles are already managed by bacula.

[erik-smit]

Any comments?

[alvagante]

Of the whole commit, this is what I would take... the rest seems really an overwhelming effort in the long term

[alvagante]

Uhm I though to have already answered to this, but it was probably a similar topic... anyway I don't think it's manageable in the long term the (optional) management of the default configurations for the main distros.
If you need to have full control of the source dir, you can use the source_dir_purge parameter (yes, the force opzion should be added). and eventually place the default files in your custom source dir.

[erik-smit]

Are your concerns mainly with:

• Long-term maintainability of such an option?
  or
• Don't see the value of this option for (many) use-cases?
  or
• Why do this decision in the central module when it can be reproduced by the user at will?

[alvagante]

Actually for all the 3 points, mostly the first and the third one.

[erik-smit]

1. What long-term maintenance do you foresee with a feature like this?
   The only 'work' I see, is is changes/bugfixes in the default settings of a package during the lifetime of a distribution release. But this is very rare, as distributions often strive to keep these things stable during the release.
   Distributions that have not been implemented for this feature, are not affected.
   Other code in this module, is not affected.
   But I could be missing something
2. I need to have a 2, otherwise github changes the next 3 into a 2. :)
3. It's a non-trivial amount of work that ends up providing benefit to users, at the flick of a switch.
   I probably would not bother with implementing this, if it would only provide benefit for myself.
   But implementing this in a way that would allow others to benefit, hopefully encouraging them to return the favor, is something I see myself doing.

Also, providing a default configuration that follows the intentions of the software developer and/or distribution is something I see in the job-description of a puppet module.
One of the reasons I ended up implementing it here, is because I did not feel this belongs in my site-modules.
And creating a -default module for every service also doesn't look very 'correct' to me.

In my view:

service-module (puppet-apache) - provides a consistent baseline for the service.
role-module (zylon::role::mailscrubber) - implements a site/role/customer/application-specific policy.

For me, the benefits are:

• Enforces consistency between systems that use the module. Especially in my use-case, where I try to slowly migrate bits and pieces of existing environments into puppet. This function allows me to flick a switch, run it one-time with no-op/audit to see if I have not forgotten any customization and bam, the dirt is gone.
• Increases overall coverage of puppet on a particular system.
• Is a relatively easy way to make puppet fully leading with regards to the configuration of that service. If a resource is not in the catalog, it must not exist on the server and vice versa. Might be valuable for security, auditing, etc. This is not (yet) one of my use-cases, but I imagine it could be valuable for this.
• 100% idempotent. One issue that I feel is missing in puppet-world is idempotency. Many modules, even the example modules that Puppetlabs writes for things like deploying Razor, are horrible hacks that only work on clean environments, and preferably are only ran once from a masterless setup.
  I think this is very bad practice. I understand that best practice is not always possible, but I'm actually quite disappointed at the example Puppetlabs sets. :(

This ended up a bit more wordy than intended. I hope the above is understandable and not too erratic.

[alvagante]

I'm seriously thinking about it, but I'm still not sold... :-)

I agree that this is stuff for a main module and not a site specific one, but since it's so easy to reproduce the behaviour setting a source_dir to a custom place and rsyncing there whatever default content you want, the extra value is not huge.

Also if we consider all the cases that may drive to a default base directory different from the vanilla one (for example after the installation of packages that place files in /etc/apache2) , we can have a base set of files that have still to be maintained somehow and, if applied, could remove any eventual additional file provided by other packages.

I wonder if we can achieve the same result setting a parameter that triggers an unattended dpkg-autoconfigure, for example... it might do the same (on Debian derivatives) without the need to manage all the files in the module itself.

What do you think about this alternative? Could it work?

[erik-smit]

For reference: http://wiki.apache.org/httpd/DistrosDefaultLayout

I agree that this is stuff for a main module and not a site specific one, but since it's so easy to reproduce the
behaviour setting a source_dir to a custom place and rsyncing there whatever default content you want, the extra
value is not huge.

I disagree it's easy to reproduce.
You need to:

• Obtain a vanilla set of configfiles from your respective packagemanagement (Extract from running server? Hope it's not tainted. Extract from package? Get package, find out how to unpack, etc.)
• Trial & error your way into a working situation. As you can see from your commit-history
• Think of a location where to put it in your tree because it doesn't 'fit' in a site-module.

While typing this, maybe it would be acceptable to create a /etc/puppet/modules/baseconfig module?
With
baseconfig/files/$DISTRO/$VERSION/$MODULE, like:
baseconfig/files/debian/squeeze/apache
baseconfig/files/redhat/6/proftpd
?

Not an ideal solution, but I can't think of anything better atm.

Also if we consider all the cases that may drive to a default base directory different from the vanilla one (for
example after the installation of packages that place files in /etc/apache2) , we can have a base set of files that
have still to be maintained somehow and, if applied, could remove any eventual additional file provided by other
packages.

Like a RPM package for an apache module that, on Redhat, puts the LoadModule configuration in /etc/apache2/conf.d?

I agree this is a relevant issue.
In this patch, I kinda worked around it by not purging /etc/apache2/modules-(enabled|available).

Maybe we could say: When using $manage_baseconfig, everything inside /etc/apache2 is considered runtime configuration, and must be provided by puppet.

I wonder if we can achieve the same result setting a parameter that triggers an unattended dpkg-autoconfigure, for > example... it might do the same (on Debian derivatives) without the need to manage all the files in the module
itself.

What do you think about this alternative? Could it work?

I thought about this before making this patch, but for me this has the following drawbacks/effects:

• This would lose no-op/audit/filebucket, breaks this usecase:

  This function allows me to flick a switch, run it one-time with no-op/audit to see if I have not forgotten any customization and bam, the dirt is gone.

• Doesn't purge. Breaks removal of unconfigured resources.

• Distribution-specific. This would only work for Debian-derived systems. I maintain a mix of systems, depending on the particular requirements of a setup. Currently mostly Debian, Ubuntu and Redhat-derivatives.
  But uven if a solution work function for these particular distributions, I would prefer something more distribution-agnostic, as I might want to use OpenSolaris, Arch Linux or something else in the future.

[alvagante]

I definitively like the idea of a baseconfig module.
I think it's the saner compromise:

• It allows you (and who has the same need) to reset to defaults the configs
• It doesn't weight the application module with all those extra files
• It somehow allows quicker population / update from new distros/packages

If you want we can start the work together.

[erik-smit]

https://github.com/zylon-internet/puppet-defaultconfig

[alvagante]

Good, we "just" need to fill it up ...