From fa2a9e5a2147be0017b32b76b117ee6c77ef52d8 Mon Sep 17 00:00:00 2001 From: Michael Raskin <7c6f434c@mail.ru> Date: Thu, 19 Nov 2020 15:54:20 +0100 Subject: [PATCH 1/4] Adding RFC proposal 0078 --- rfcs/0078-configuration-file-generators.md | 147 +++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 rfcs/0078-configuration-file-generators.md diff --git a/rfcs/0078-configuration-file-generators.md b/rfcs/0078-configuration-file-generators.md new file mode 100644 index 000000000..0c36fdfe4 --- /dev/null +++ b/rfcs/0078-configuration-file-generators.md @@ -0,0 +1,147 @@ +--- +feature: configuration_file_generators +start-date: 2020-10-18 +author: Michael Raskin @7c6f434c +co-authors: (in-progress) +shepherd-team: (names, to be nominated and accepted by RFC steering committee) +shepherd-leader: (name to be appointed by RFC steering committee) +related-issues: (will contain links to implementation PRs) +--- + +# Summary +[summary]: #summary + +Provide the configuration file generation functionality separately from NixOS +as a whole, and using the module system without the global namespace. + +# Motivation +[motivation]: #motivation + +There is currently a lot of code duplication between +* NixOS +* home-manager +* nix-darwin + +and obviously growing code duplication with any effort running on below-Tier-2 +platforms. That leads both to effort duplication, and to confusion among users +using multiple Nix ecosystem tools for service generation, + +While fully resolving the problem is likely not fully compatible with the core +preferences of maintainers on each specific platform, we believe that config +file generation could be a shared resource just like Nixpkgs based on the +following properties: +* unlike, say, upstream `systemd` unit files we typically do not + reuse-and-override upstream configs, but generate our own (both in NixOS and + other service collections) +* the configs for a piece of software often have few dependencies on the + general environment +* it can be defined as a purely functional library, simplifying integration + +We aim to reduce duplication of effort and code while also having a high level +of inspection and tweaking possibilities without needing to patch module +source. + +# Detailed design +[design]: #detailed-design + +This RFC proposal builds upon RFCs#42 defining a configuration file abstraction +approach. + +A top-level directory is added to the Nixpkgs repository, and a library of +functions implementing program configuration file generation is created within +this directory. Each such generator takes as an input a NixOS module system +based attribute set «subtree», e.g. the attribute set typically bound to `cfg` +variable in the current NixOS service modules. + +The output of each such function is an attribute set. The keys are relative +file names of configuration files, and the values are attribute sets of RFCs#42 +settings abstraction and serialiser. + +The function is provided as a member of an attribute set, which also contains +the corresponding type specifications for input and output modules are defined. + +In some cases we only provide low-level overridable default configuration. In +this case the input may have the type that is always an empty attribute set, +`{}`. + +# Examples and Interactions +[examples-and-interactions]: #examples-and-interactions + +A minimalistic silly example is: + +Input: +``` + { + port = 1234; + nodeName = "localhost"; + content = "Hello"; + } +``` + +Output: +``` + { + "subdir/listen.conf" = { + serialiser = toJSON; + settings = { + listenPort = "1234"; + serverGreet = "I am localhost"; + }; + type = …; + }; + "subdir/content.conf" = { + serialiser = toINI; + settings = { + greeting = { + greeter = "localhost"; + }; + payload = { + value = "I say to you: ‘Hello’"; + }; + }; + type = …; + }; + } +``` + +# Drawbacks +[drawbacks]: #drawbacks + +The proposed approach increases the spreading of the related code, from package +and service module to package and configuration generator and module. + +The proposed refactoring incurs a cost in terms of implementation effort across +NixOS, and likely to create a medium-term situation of partial migration. + +# Alternatives +[alternatives]: #alternatives + +Do nothing, continuing with the current code duplication. + +Same, but put configuration generators closer to packages. This would mean +widespread use of the module system inside `pkgs/`. There is also no guarantee +that all the configuration files describing interaction of multiple software +packages will have a clear choice of reference package. + +There also have been many solutions proposed based on a significant rework of the module system. + +Abstract generation of configuration files with package-like flat arguments and +plain text file outputs. This approach will need less code as long as it we do +not want type checks, or ease of overriding values. + +Implement a complete service abstraction not tied to global system-wide +assumptions. + +# Unresolved questions +[unresolved]: #unresolved-questions + +Currently none. + +# Future work +[future]: #future-work + +Define generic service abstraction for non-`systemd` systems, possibly with a +convertor from `systemd` units. + +Consider some way of referring from package to related configuration file +generators via `meta` or `passthru`, in a way similar to `passthru.tests`. From 84b40a4ae1581bfcf01f52a4e6a7e5a671c78972 Mon Sep 17 00:00:00 2001 From: Michael Raskin <7c6f434c@mail.ru> Date: Fri, 18 Dec 2020 09:52:51 +0100 Subject: [PATCH 2/4] Additions based on review --- rfcs/0078-configuration-file-generators.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/rfcs/0078-configuration-file-generators.md b/rfcs/0078-configuration-file-generators.md index 0c36fdfe4..923d6fc87 100644 --- a/rfcs/0078-configuration-file-generators.md +++ b/rfcs/0078-configuration-file-generators.md @@ -39,7 +39,8 @@ following properties: We aim to reduce duplication of effort and code while also having a high level of inspection and tweaking possibilities without needing to patch module -source. +source. At the same time we aim to minimise the amount of code in the NixOS +subtree that cannot be tested on NixOS. # Detailed design [design]: #detailed-design @@ -59,6 +60,8 @@ settings abstraction and serialiser. The function is provided as a member of an attribute set, which also contains the corresponding type specifications for input and output modules are defined. +The specifications for the input should be suitable for importing as a part of +the NixOS service module options. In some cases we only provide low-level overridable default configuration. In this case the input may have the type that is always an empty attribute set, @@ -123,7 +126,15 @@ widespread use of the module system inside `pkgs/`. There is also no guarantee that all the configuration files describing interaction of multiple software packages will have a clear choice of reference package. -There also have been many solutions proposed based on a significant rework of the module system. +A complete or partial merge of the module collections of the major currently +existing module system based code bases, with internal options for platform +specific implementation details. Unfortunately, this would force more coupling +of the development cycle for the platform-specific parts. It also does not +create a clean inspection/override point, and mixes the code with different +platform requirements for testing. + +There also have been many solutions proposed based on a significant rework of +the module system. Abstract generation of configuration files with package-like flat arguments and plain text file outputs. This approach will need less code as long as it we do @@ -135,7 +146,7 @@ assumptions. # Unresolved questions [unresolved]: #unresolved-questions -Currently none. +Full proof-of-concept implementation for some module. # Future work [future]: #future-work From 1c4e05d334b2aecd65608fbedc33b02a6ff63fc0 Mon Sep 17 00:00:00 2001 From: Michael Raskin <7c6f434c@mail.ru> Date: Fri, 18 Dec 2020 10:20:22 +0100 Subject: [PATCH 3/4] Reveal more design goals, add one more alternative --- rfcs/0078-configuration-file-generators.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/rfcs/0078-configuration-file-generators.md b/rfcs/0078-configuration-file-generators.md index 923d6fc87..aec0f99c0 100644 --- a/rfcs/0078-configuration-file-generators.md +++ b/rfcs/0078-configuration-file-generators.md @@ -39,8 +39,10 @@ following properties: We aim to reduce duplication of effort and code while also having a high level of inspection and tweaking possibilities without needing to patch module -source. At the same time we aim to minimise the amount of code in the NixOS -subtree that cannot be tested on NixOS. +source. An additional goal is to move the more package-like configuration file +generation code closer to the Nixpkgs model with scope/visibility rules. At +the same time we aim to avoid or minimise code in the NixOS subtree that +cannot be tested on NixOS. # Detailed design [design]: #detailed-design @@ -126,6 +128,13 @@ widespread use of the module system inside `pkgs/`. There is also no guarantee that all the configuration files describing interaction of multiple software packages will have a clear choice of reference package. +Introduce a guideline on making configuration files available to the module +system after generation (i.e. avoidance of only putting them in a `let` to add +a path reference inside some string; maybe with best practices on naming the +corresponding options). Does not help with scoping, still needs a non-trivial +amount of refactoring. Might make other desirable refactorings such as +multiple instances of a module marginally harder. + A complete or partial merge of the module collections of the major currently existing module system based code bases, with internal options for platform specific implementation details. Unfortunately, this would force more coupling From d88a16211ed5dc90fe155be2f4f34e1d7b0ccbfd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rg=20Thalheim?= Date: Thu, 21 Jan 2021 14:13:05 +0000 Subject: [PATCH 4/4] add shepherd team --- rfcs/0078-configuration-file-generators.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/0078-configuration-file-generators.md b/rfcs/0078-configuration-file-generators.md index aec0f99c0..a22dfa908 100644 --- a/rfcs/0078-configuration-file-generators.md +++ b/rfcs/0078-configuration-file-generators.md @@ -3,8 +3,8 @@ feature: configuration_file_generators start-date: 2020-10-18 author: Michael Raskin @7c6f434c co-authors: (in-progress) -shepherd-team: (names, to be nominated and accepted by RFC steering committee) -shepherd-leader: (name to be appointed by RFC steering committee) +shepherd-team: @ehmry, @edolstra, @roberth +shepherd-leader: @edolstra related-issues: (will contain links to implementation PRs) ---