Update process

.dotstop.dot

@@ -56,6 +56,7 @@ digraph G {
 "JLS-24" [sha=b16224d3ab676c00b313ae91760072d92aed9f20da99b363621effa3e033e012];
 "JLS-25" [sha="8bb517191450f370679dbafd85342e1bbcf797cc84f2a6f1fc119568b534d5e0"];
 "JLS-26" [sha=cf1b73b375697ee56d9788aab79ed01b2730b126a2cc4d7041c9525113e7ed7c];
+"JLS-27" [sha="efd4b438331c155eebaec96cd1eda337567794f8696b327562aaaed5fa8ded69"];
 "NJF-01" [sha="548dc86014e093974f68660942daa231271496a471885bbed092a375b3079bd8"];
 "NJF-02" [sha="6ea015646d696e3f014390ff41612eab66ac940f20cf27ce933cbadf8482d526"];
 "NJF-03" [sha="4bd1f8210b7bba9a248055a437f377d9da0b7576c5e3ed053606cf8b5b2febe3"];
@@ -356,6 +357,7 @@ digraph G {
 "TA-BEHAVIOURS" -> "AOU-23" [sha=""];
 "TA-BEHAVIOURS" -> "AOU-24" [sha=""];
 "TA-BEHAVIOURS" -> "AOU-25" [sha=""];
+"TA-BEHAVIOURS" -> "JLS-27" [sha="9c3e4b4a4677ca22f28e2bb0cbc39c31676efa4c17ddc1fe66599589b83ef643"];
 "TA-CONFIDENCE" -> "JLS-08" [sha="506164051180023c8533ea1f6dedf1bad894c3ee6020ff16b002e33b109c2791"];
 "TA-CONFIDENCE" -> "JLS-09" [sha="80bbde95fc14f89acf3dad10b3831bc751943fe4a1d79d5cbf4702416c27530f"];
 "TA-CONFIDENCE" -> "AOU-10_COMBINED" [sha="5e5d7dc606d53423fbb1f2d5755780c98839bdc2d108704af5ee1aed50403f5e"];

.dotstop_extensions/README.md

@@ -14,9 +14,9 @@ Custom references are defined in `references.py`. A (custom) reference is used b
 
 ## CPPTestReference
 
-The content of a `CPPTestReference` is given by the lines of code corresponding to a test-case or a section of a test-case in the unit-tests given in tests/src and TSF/tests.
+The content of a `CPPTestReference` is given by the lines of code corresponding to a test-case or a section of a test-case in a specified unit-test-file. The sections are identified in the value of "name", where the nested sections are separated by semicolons.
 
-For the `CPPTestReference` an example is:
+For the `CPPTestReference` the expected configuration is:
 ```
 ---
 ...
@@ -30,9 +30,12 @@ references:
 
 ## JSONTestsuiteReference
 
-The content of a `JSONTestsuiteReference` is given by the lines of code corresponding to a test-case or a section of a test-case in the unit tests, where a (list of) specified test-file(s) located on an external test-repository is utilized, and the content of these test-files.
+The `JSONTestsuiteReference` is a variant of the function reference, which is augmented by an external file containing test-data in the form of well- or ill-formed JSON candidate data. 
+A `JSONTestsuiteReference` is therefore given by the data of a `CPPTestReference` together with a list containing the paths to these external files.
+The external files are stored in a separate branch of the repository, and their text is loaded via call to github.
+The content of a `JSONTestsuiteReference` is given by the content of the underlying `CPPTestReference` together with the sum of the contents of the external test-suite files.
 
-For the `JSONTestsuiteReference` an example is:
+For the `JSONTestsuiteReference` the expected configuration is:
 ```
 ---
 ...
@@ -65,7 +68,8 @@ references:
 ---
 ```
 
-Since functions may be overloaded, a `FunctionReference` can be initialised with an optional overload-parameter; additionally, it is possible to give a description. The full example is:
+Since functions may be overloaded, a `FunctionReference` can be initialised with an optional overload-parameter. 
+The overload-parameter specifies which implementation of the function is referred to, i.e. if the overload-parameter for the function ``class::function()`` is set to _n_, then the _n_-th implementation when counting the occurrences from top to bottom of ``function()`` within the class ``class`` is used, if it exists; otherwise, an error is thrown. Additionally, it is possible, but not mandatory, to give a description. The full example is:
 ```
 ---
 ...
@@ -140,14 +144,6 @@ The content of a `TimeVaryingWebReference` is given by the content of a changelo
 
 An example of the complete configuration for `TimeVaryingWebReference` is
 
-in case of a custom description.
-
-## TimeVaryingWebReference
-
-The content of a `TimeVaryingWebReference` is given by the content of a changelog, whose default value is `ChangeLog.md`, which mirrors the changelog of nlohmann/json. This reference is intended for websites, whose content is constantly changing, so that a `WebContentReference` makes the item un-reviewable, but whose content at the time of an update influences the trustability. An example is `https://github.com/nlohmann/json/pulse/monthly`, which can be used to demonstrate that nlohmann/json is *up to the most recent version* under active development.
-
-An example of the complete configuration for `TimeVaryingWebReference` is
-
 ```
 ---
 ...
@@ -192,6 +188,26 @@ references:
 ---
 ```
 
+## workflow_failures
+
+This reference queries `https://github.com/{self._owner}/{self._repo}/actions?query=is%3Afailure+branch%3A{self._branch}` and collects the number of failed workflow runs as its content.
+Here, owner, repo and branch are the arguments given to the constructor of the reference.
+If no branch is specified, then all failures are collected, i.e. `https://github.com/{self._owner}/{self._repo}/actions?query=is%3Afailure` is queried.
+In case the website is un-reachable, or the github layout changes drastically so that the number of failed workflow runs does not exist at the expected location, an error is thrown.
+
+The expected configuration is 
+
+```
+---
+...
+references:
+- type: workflow_failures
+      owner: "eclipse-score"
+      repo: "inc_nlohmann_json"
+      branch: "json_version_3_12_0"
+---
+```
+
 ## ItemReference
 
 Some references support every (directly or indirectly) supporting item of an item. 
@@ -281,7 +297,7 @@ The test-files are called unit-FILE_NAME.cpp. In the configuration, FILE_NAME is
 
 For each test specified in test-files, the number of passed and failed test-cases is calculated, while the number of skipped test-cases is ignored. The score of each test is then the ratio of passed test-cases compared to all non-skipped test-cases; the total score is the mean of the individual scores.
 
-## issue_checker
+## check_issues
 
 The automatic validator `check_issues` is intended to evaluate the json-lists `raw_open_issues.json` and `raw_closed_issues.json` and compare with the list of known issues of nlohmann/json labelled as bug opened since the release of the version of nlohmann/json that is documented. The json lists are generated in the publish_documentation-Workflow, and not persistently stored.
 
@@ -304,6 +320,100 @@ From `raw_closed_issues.json`, all issue IDs are collected, which are labelled a
 If for any of these IDs, it is not explicitly indicated in the list of known misbehaviours that this issue does not apply to Eclipse S-CORE, then the score 0.0 is returned.
 Otherwise, the score 1.0 is assigned.
 
+## did_workflows_fail
+
+The automatic validator `did_workflows_fail` queries the web-site `https://github.com/{owner}/{repo}/actions?query=event%3A{event}+is%3Afailure+branch%3A{branch}` and looks on the number of workflow run results which is printed at the head of the table.
+In case that this number is not zero, a score of 0.0 is returned, and 1.0 otherwise.
+
+The expected configuration is given as follows:
+
+```
+evidence:
+    type: did_workflows_fail
+    configuration:
+        owner: "eclipse-score" # owner of the repository
+        repo: "inc_nlohmann_json" # name of the repository
+        branch: "json_version_3_12_0" # name of the branch
+        action: "push" # optional, default is push
+```
+
+It is of utmost importance that the arguments come with quotation marks. Otherwise, the update helper does not work as intended.
+
+## coveralls_reporter
+
+The automatic validator `coveralls_reporter` queries the [coveralls](https://coveralls.io/) api to get the line and branch coverages calculated by the service, which is running on the repository.
+Unless the version of `nlohmann/json` documented in this repository changes, it is expected that both coverage numbers remain constant.
+When initialising the reference, the current code coverage is given as a parameter, to which the fetched coverages are compared.
+If no branch is specified, then the most recently calculated coverage is fetched, so that it is generally recommended to specify a branch.
+Moreover, it is possible to specify the number of decimal digits, which is defaulted to three, when not specified.
+The validator returns a score of 1.0 if both fetched coverages rounded to the specified number of decimal digits coincide with the specified ones, and a score of 0.0 otherwise.
+
+The expected configuration is the following:
+
+```
+evidence:
+    type: coveralls_reporter
+    configuration:
+        owner: "score-json"
+        repo: "json"
+        branch: "main"
+        line_coverage: 99.186
+        branch_coverage: 93.865
+        digits: 3
+```
+
+## combinator
+
+The trudag tool does currently not support the use of multiple custom validators for one single TSF item. To work around this, the validator `combinator` is implemented as a meta-validator that executes multiple validators and combines their scores using a weighted average. This enables the validation of complex trustable items that require evidence from multiple sources or validation methods.
+
+The combinator accepts a list of validators, each with its own configuration and optional weight. Each validator is executed independently, and their scores are combined using the formula: `(score1 * weight1 + score2 * weight2 + ...) / (weight1 + weight2 + ...)`. If no weights are specified, all validators are treated with equal weight (weight = 1.0).
+
+The combinator supports the following validator types:
+- `check_artifact_exists`
+- `https_response_time` 
+- `check_test_results`
+- `file_exists`
+- `sha_checker`
+- `check_issues`
+- `did_workflows_fail`
+- `coveralls_reporter`
+
+The expected configuration is as follows:
+
+```
+evidence:
+    type: combinator
+    configuration:
+        validators:
+            - type: "check_test_results"
+              weight: 2.0  # optional, defaults to 1.0
+              configuration:
+                  tests:
+                      - class_lexer
+                      - unicode1
+            - type: "https_response_time"
+              weight: 1.0  # optional, defaults to 1.0  
+              configuration:
+                  target_seconds: 2
+                  urls:
+                      - "https://github.com/nlohmann/json/issues"
+            - type: "coveralls_reporter"
+              weight: 1.5  # optional, defaults to 1.0
+              configuration:
+                  owner: "score-json"
+                  repo: "json"
+                  branch: "main"
+                  line_coverage: 99.186
+                  branch_coverage: 93.865
+                  digits: 3
+            - type: "did_workflows_fail"
+              configuration:
+                  owner: "eclipse-score"
+                  repo: "inc_nlohmann_json" 
+                  branch: "json_version_3_12_0"
+```
+
+All weights must be non-negative. If the sum of all weights is zero, the combinator returns a score of 0.0. The combinator aggregates all exceptions and warnings from the individual validators and returns them alongside the combined score.
 
 # Data store interface