Audit preparation: Verification #80
Description
"not started" topics:
- Testing of the embedded software: present that it is tailored out
Actions and Deviations:
- Deviation_13: The method selection for the unit tests seems to be too theoretical or wrong. How is a control flow and data flow analysis made on Unit level? Why is requirements based testing and interface based testing not selected? The table should be reworked in accordance to the mapping of the specified test levels to the ISO 26262 test levels.
- Action_43: The mapping of the test levels to the ISO 26262-6 test levels shall be shown e.g., in the PMP.
- Action_44: Provide an argumentation about which tests have been selected for a feature.
- unclear action, https://eclipse-score.github.io/process_description/main/process_areas/verification/verification_workproducts.html#wp__verification_feat_int_test describes from which input the feature tests are derived, need to clarify if we want to have a review checklist.
- Action_48: An argument shall be provided which covers the ISO 26262 requirement that all test levels are executed in a target near environment.
- Action_49: The methods of ISO 26262 shall be explained.
- Action_50: The Coverage target values as listed by table 37 need to described in more detail. E.g. what is meant by “Coverage of detailed/architectural design” (Sequence diagram coverage?)
- Action_51: It needs to be clarified where the reference system is located physically (affecting e.g. access rights).
- planned, see Improvement: Location of reference hardware score#608
- Not yet decided which HW to use see also: https://eclipse-score.github.io/score/main/platform_management_plan/software_verification.html#test-execution-environment-and-reference-hardware
- https://github.com/orgs/eclipse-score/discussions/472#discussioncomment-15122092 is the start to discuss about the reference hardware
- Action_53: The tool Bazel needs to be evaluated as it may not be able to see the dependencies of changes in requirements, code and tests and may skip some tests. Define the counter measurements.
- see Tool management: Audit preparation: Tool management #81
- Action_54: The test shall have an attribute which level the test case belongs to.
- unclear, linking to requirements is not solving this as IF tests are not linked to these.
- the test in the link, show the trace to component requirements and also include "CIT" as "component integration Test" as additional identifier in the name of the file. This is optional, but makes life simpler. See: https://github.com/eclipse-score/persistency/blob/main/tests/python_test_cases/tests/test_cit_supported_datatypes.py#L57
- Another example is feature integration test, where the folder is named accordingly. See: https://github.com/eclipse-score/reference_integration/blob/main/feature_integration_tests/python_test_cases/tests/basic/test_orchestartion_with_persistency.py#L23
- Second it‘s not clear which test belongs to which level; this shall be fixed.
- Third, the application of the test methods must be recognizable (e.g. boundary value analysis, equivalence class definition, interface-based testing, ...).
- unclear, linking to requirements is not solving this as IF tests are not linked to these.
- Deviation_19: The Verification Plan does not define the required review activities. (on Existing tests from the Open-Source Community)
- Action_55: The independence of the verification team is not yet described in the Verification Plan
- Action_55: It remains unclear how many Verification reports will be created (one per module?)
Metadata
Metadata
Assignees
Labels
No labels