Resolve gardener hashing / signing issue

[fabianburth]

Description
Support investigation and resolving failing verification of gardener component descriptors after ocm-cli upgrade from 0.18.0 to 0.22.1.

Timebox: 5 day(s)

Done Criteria

• Estimation of impact on existing code incl. tests
• Estimation of impact on Enduser Documentation updated (if applicable)
• Estimation of impact on Internal technical Documentation created/updated (if applicable)
• Created refinable tasks for the actual implementation

[fabianburth]

There were 2 main issues:

• ocm-cli version 0.18.0 had a similar bug as reported here with regards to the extraIdentity defaulting during normalization.
  • If the extraIdentity is an empty object (thus, extraIdentity: {}, the normalization does default the version into the
    extraIdentity IF there are 2 resources with the same name (or identity in general).
  • If the extraIdentity is null or non existent, is does not do that defaulting.
• ocm-cli hash command has an --update flag which can alter the component descriptors

[jakobmoellerdev]

In general, my post-mortem observations are:

• --update is never a good idea and should not be included in the CI.
• --normalization must be set to jsonNormalisation/v1 to generate the correct digests for you.
• When you switched to the new CLI, Gardener used ocm hash cv without a dedicated normalization algorithm. When they then wrote the signature (with your sign script), they wrote the digest (generated with jsonNormalisation/v3) into a field that specified jsonNormalisation/v1.
• When the verification failed, it was rolled back, but the --update already corrupted the version.
• The following debugging with the helper script also had --update and modified the descriptors. After we noticed this, we realized that staging was also broken.
• Even after a rollback to 0.18.0, digest verification failed.
  Then we noticed that the extraIdentity was skewed, which led to different hashes with ocm 0.18.0, even after the rollback.
  To fix this, we manually patched back descriptors and refreshed signatures.
  This allowed successful verifications with both the new and old CLI, both dev and stage.

[jakobmoellerdev]

As concrete findings we identified:

• The ocm hash command does not properly respect the recursive option and instead always recursively hashes.
• --update is very dangerous on hash and needs a warning
• The ocm hash/sign/verify implementations differ in how they interpret the normalisationalgorithms. It is important that they are always the same in matching commands (if signed with jsonNormalisation/v1 also verify with that). This technically isnt needed because the algo is inside the descriptor.