Documentation update for overrides

docs/source/packs.rst

@@ -210,6 +210,163 @@ have to comply with Health and Safety. The generated file will be placed in
 
 For more nice tricks on pack configuration, see :doc:`/reference/pack_configs`.
 
+Overriding Pack Defaults
+------------------------
+
+When installing a pack the state of the resources is taken from the metadata files in the pack. Occassionally, when installing community packs you might not want all resources enabled, e.g. you only want to use a subset of actions, or want to disable the sensors.
+
+Prior, to |st2| 3.7.0 then this state can be altered, by:
+
+  * using the |st2| APIs to disable the resource. However, this will be forgotten upon a ``st2ctl reload`` or pack reinstall.
+  * changing the metadata file manually. This would be lost upon an upgrade, and cannot be easily tracked.
+
+In |st2| 3.7.0 we have introduced the override feature, so that the metadata of a packs resources can be overridden by configuration files. This will always be read upon a reload or pack install. The ST2 APIs will still allow you to override this, but as before any changes made by the |st2| APIs to enable/disable resources is forgotten upon reload or re-install.
+
+The override facility is currently restricted to allowing the enabled property to be overridden only for resources of type: action, alias, rule and sensor. It is controlled by the ``/opt/stackstorm/overrides`` directory.
+
+Upon a pack install or reload resource state is managed as follows:
+
+* State is read from the pack's resource metadata files that reside in ``/opt/stackstorm/packs/<packname>``. These are downloaded from the relevant repository of the pack, e.g. GIT, StackStorm-Exchange.
+* If ``/opt/stackstorm/overrides/_global.yaml`` is present, then any global overrides are applied. The ``_global.yaml`` allows you to specify the default state of a particular resource types, e.g. disable all sensors.
+  The format of the _global.yaml is as follows (set to disable everything):
+
+.. code-block:: bash
+
+   ---
+   sensors:
+     defaults:
+       enabled: false
+   actions:
+     defaults:
+       enabled: false
+   aliases:
+     defaults:
+       enabled: false
+   rules:
+     defaults:
+       enabled: false
+
+* If ``/opt/stackstorm/overrides/<packname>.yaml`` is present, then any pack defaults, or resource specific overrides are applied. The <pack>.yaml allows you to alter the state of all the resources of a particular type, via the defaults property (e.g. all sensors within the pack), or by individual resource type, via the exceptions property (e.g. only alter state of individual actions). As the pack name is determined by the name of the overrides file, the name of the resource in the file should omit the pack name, e.g. ``action1`` in the ``pack1.yaml`` refers to the resource ``pack1.action1``.
+  The format of the ``<pack>.yaml`` is as follows (set to disable everything, except for individual resource of each type):
+
+.. code-block:: bash
+
+   ---
+   sensors:
+     defaults:
+       enabled: false
+     exceptions:
+       sensorname1:
+         enabled: true
+   actions:
+     defaults:
+       enabled: false
+     exceptions:
+       actionname1:
+         enabled: true
+   aliases:
+     defaults:
+       enabled: false
+     exceptions:
+       aliasname1:
+         enabled: true
+   rules:
+     defaults:
+       enabled: false
+     exceptions:
+       rulename1:
+         enabled: true
+
+If overrides are taking place, then the number of resources affected will be output on the ``st2ctl reload`` output, for example:
+
+.. code-block:: bash
+
+   $ st2ctl reload --register-all
+   Registering content...[flags = --config-file /etc/st2/st2.conf --register-all]
+   2022-02-07 12:56:43,694 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "None".
+   2022-02-07 12:56:43,704 INFO [-] Successfully connected to database "st2" @ "127.0.0.1:27017" as user "None".
+   2022-02-07 12:56:44,254 INFO [-] =========================================================
+   2022-02-07 12:56:44,254 INFO [-] ############## Registering triggers #####################
+   2022-02-07 12:56:44,254 INFO [-] =========================================================
+   2022-02-07 12:56:44,549 INFO [-] Registered 1 triggers.
+   2022-02-07 12:56:44,549 INFO [-] =========================================================
+   2022-02-07 12:56:44,549 INFO [-] ############## Registering sensors ######################
+   2022-02-07 12:56:44,549 INFO [-] =========================================================
+   2022-02-07 12:56:44,832 INFO [-] Registered 9 sensors.
+   2022-02-07 12:56:44,832 INFO [-] 7 sensors had their metadata overridden.
+   ...
+
+On a pack install, then the number of resources that have their metadata overridden will be reported in the output of the register_pack action by the "(overridden)" entries. If no resources of that type have their metadata overridden, then there will be no correspdoning overridden entry outputted:
+
+
+.. code-block:: bash
+
+   {
+     "runners": 14,
+     "rule_types": 2,
+     "policy_types": 3,
+     "triggers": 0,
+     "sensors": 2,
+     "sensors(overridden)": 2,
+     "actions": 1,
+     "rules": 1,
+     "aliases": 0,
+     "policies": 0,
+     "configs": 0
+    }
+
+.. rubric:: Example: Disabling all sensors in one pack
+
+For example, to disable all sensors in an individual pack then we would create a ``/opt/stackstorm/overrides/<packname>.yaml`` that contained:
+
+.. code-block:: bash
+
+   ---
+   sensors:
+     defaults:
+       enabled: false
+
+.. rubric:: Example: Disabling all sensors in all packs, except for one pack
+
+If, instead we wanted to override all sensors except for a single pack, then we would instead create a ``/opt/stackstorm/overrides/_global.yaml`` to disable all sensors:
+
+.. code-block:: bash
+
+    ---
+    sensors:
+      defaults:
+        enabled: false
+
+And an ``/opt/stackstorm/overrides/<packname>.yaml`` to enable the sensors for the one pack we needed:
+
+.. code-block:: bash
+
+    ---
+    sensors:
+      defaults:
+        enabled: true
+
+.. rubric:: Example: Overriding state of individual resources
+
+In this next example, we disable all actions in ``pack1``, except for the actions ``pack1.action1`` and ``pack1.action2``. We also change the state of just a single rule to disabled. To do this, we create a ``/opt/stackstorm/overrides/pack1.yaml`` with:
+
+.. code-block:: bash
+
+    ---
+    actions:
+      defaults:
+        enabled: false
+      exceptions:
+        action1:
+          enabled: true
+        action2:
+          enabled: true
+    rules:
+      exceptions:
+        rule1:
+          enabled: false
+
+
 Developing a Pack
 -----------------
 

docs/source/reference/packs.rst

@@ -169,6 +169,8 @@ available at :github_st2:`st2/contrib/hello_st2 <contrib/hello_st2>`.
      ``name`` value in ``pack.yaml`` must only contain lowercase letters, digits, and underscores,
      unless you set the ``ref`` value explicitly in ``pack.yaml``. Finally the email attribute in
      ``pack.yaml`` must contain a properly formatted email address.
+
+     From |st2| 3.7.0, the name ``_global`` is reserved, and cannot be used as a pack name.
 
      The ``python_versions`` field declares which Python versions this pack supports. As |st2| 
      only supports Python **3** from version 3.4.0 onwards, packs submitted to the StackStorm

docs/source/upgrade_notes.rst

@@ -3,6 +3,15 @@
 Upgrade Notes
 =============
 
+.. _ref-upgrade-notes-v3-7:
+
+|st2| v3.7
+----------
+
+* As part of introducing the override pack metadata functionality, the name ``_global`` is
+  reserved, and cannot be used for pack names or pack references, to avoid conflict between
+  the global override file and individual pack override files.
+
 .. _ref-upgrade-notes-v3-6:
 
 |st2| v3.6