Conversation
Signed-off-by: aschemmel-tech <aschemmel_job@arcor.de>
aschemmel-tech
force-pushed
the
aschemmel-tech-patch-3
branch
from
c6fb5d0 to
e7253a8
Compare
|
|
||
| Evidence: | ||
|
|
||
| 1. list of reproducible SHAs |
There was a problem hiding this comment.
nlohmann/json library is just a header file (single_include/nlohmann/json.hpp), so this would be the only thing on the list.
Furthermore, this should already be covered by JLS-14
|
|
||
| 2. list of non-reproducible elements with: | ||
| - explanation and justification | ||
| - details of what is not reproducible |
There was a problem hiding this comment.
nothing is non reproducible
| - explanation and justification | ||
| - details of what is not reproducible | ||
|
|
||
| 3. evidence of configuration management for build instructions and infrastructure |
There was a problem hiding this comment.
TODO:
- Look into build manifest files
- Check if there is already some documentation on config management/build instructions in nlohmann/json
|
|
||
| 3. evidence of configuration management for build instructions and infrastructure | ||
|
|
||
| 4. evidence of repeatable builds |
|
|
||
| Evidence: | ||
|
|
||
| 1. list of tests |
There was a problem hiding this comment.
already provided (JLS-16)
|
|
||
| aschemmel-tech: we need a split in four statements/evidences (JLS) as defined in TA-TESTS. Here two are accumulated. | ||
|
|
||
| For the topic of test list: missing argument why we think tests are complete e.g. structtural coverage, linking to the RFC8259 standard? |
There was a problem hiding this comment.
Statement on coverage is provided in JLS-27 (recently added)
| aschemmel-tech: we need a split in four statements/evidences (JLS) as defined in TA-TESTS. Here two are accumulated. | ||
|
|
||
| For the topic of test list: missing argument why we think tests are complete e.g. structtural coverage, linking to the RFC8259 standard? | ||
| I also see that nlohman defines fuzz testing (e.g. https://github.com/score-json/json/blob/main/tests/src/fuzzer-parse_bjdata.cpp). We should refer to this but maybe we cannot reproduce? Where are the results for this? |
There was a problem hiding this comment.
Fuzz testing is already stated and referred to in JLS-02.
TO-DO: Check if we should/can add another link from TA-TESTS to JLS-02.
| I also see that nlohman defines fuzz testing (e.g. https://github.com/score-json/json/blob/main/tests/src/fuzzer-parse_bjdata.cpp). We should refer to this but maybe we cannot reproduce? Where are the results for this? | ||
|
|
||
| For the topic of test execution environment(s) : my understanding: only one environment was used: the github host environment together with Doctest (version ?) | ||
| but with a wide range of compilers and versions of these. No newline at end of file |
There was a problem hiding this comment.
- Check if we can create a list of compiler+compiler version used for each test.
- Add some reference to github actions page? (to show that we are using github host environment)
|
|
||
| All components, dependencies and tools are listed in a manifest. | ||
|
|
||
| aschemmel-tech: missing evidence, so impossible to check if the completeness of this statement fulfillment is given. |
There was a problem hiding this comment.
Already updated (in a separate branch. Will be merged to #115)
| A score is calculated based on the number of mirrored and unmirrored things. (TODO) No newline at end of file | ||
| A score is calculated based on the number of mirrored and unmirrored things. | ||
|
|
||
| aschemmel-lib: This statement should be about "evidence of repeatable builds". Mirroring is one aspect, but also the "internet access" topic, the repeatability on different server and how much the used tools are under our control (i.e. are these available during our product lifecycle). For example do we need CMake, Doctest mirrored? No newline at end of file |
There was a problem hiding this comment.
Revisit (discuss internally)
There was a problem hiding this comment.
The following evidences are already stated in JLS-10 and JLS-19:
- source code
- build instructions
- tests
- attestations
tests in JLS-10 will be split up into test code and test results summary to align with the evidence list provided by the trustable doc.
|
Topics taken into account in eclipse-score#9 |
4 participants
no impact on nlohman/json
modified assertions and evidences and added SME scores