fix: update notation verify error messages

[ghost]

In this PR:

1. Updated notation verify error message to display failure reasons for each signature WITHOUT the need to enable -v or -d flags. This is to improve Notation CLI's usability during verification failures.
2. Improved trust store related error messages readability.

This PR should only be reviewed/merged after its corresponding notation-go PR is merged: notaryproject/notation-go#345.

Based on discussions from issues #699, #700, and #701, the error messages become:

# no certificate in the trust store
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, no x509 certificates were found in trust store "alpine" of type "ca". Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, no x509 certificates were found in trust store "alpine" of type "ca". Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

# the trust store is missing
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, the trust store "alpine" of type "ca" doesn't exist. Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, the trust store "alpine" of type "ca" doesn't exist. Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

# no permission to read the certificate from the trust store
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, failed to read the trusted certificate "/home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt": open /home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt: permission denied
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, failed to read the trusted certificate "/home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt": open /home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt: permission denied
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

[priteshbandi]

IMO showing error or warning for each signature in a successful verification scenario is too much information which might confuse user.

[ghost]

IMO showing error or warning for each signature in a successful verification scenario is too much information which might confuse user.

@priteshbandi Yes, I agree. For a successful verification, none of the above error will be printed out.
On success, the output message of Notation CLI would just be:

$ notation verify testnotation.azurecr.io/hello@sha256:abcdef
Successfully verified signature for testnotation.azurecr.io/hello@sha256:abcdef

/cc: @yizha1 @FeynmanZhou

[codecov-commenter]

Codecov Report

Merging #771 (aea48fe) into main (b7e5a6f) will increase coverage by 0.22%.
The diff coverage is 100.00%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main     #771      +/-   ##
==========================================
+ Coverage   63.98%   64.21%   +0.22%     
==========================================
  Files          40       40              
  Lines        2252     2266      +14     
==========================================
+ Hits         1441     1455      +14     
  Misses        689      689              
  Partials      122      122              

Files Changed	Coverage Δ
cmd/notation/verify.go	87.07% <100.00%> (+1.36%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[shizhMSFT]

This line need to be removed after merging notaryproject/notation-go#345

[shizhMSFT]

LGTM but waiting for notaryproject/notation-go#345

[priteshbandi]

# no certificate in the trust store
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, no x509 certificates were found in trust store "alpine" of type "ca". Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, no x509 certificates were found in trust store "alpine" of type "ca". Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

Ideal experience would be to not even retrieve signature if we know there are no certificates in trust store. Thus the error message becomes.

notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: signature verification failed because no x509 certificates were found in trust store "alpine" of type "ca".

---

# the trust store is missing
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, the trust store "alpine" of type "ca" doesn't exist. Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, the trust store "alpine" of type "ca" doesn't exist. Use command 'notation cert add' to create and add trusted certificates to the trust store.
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

Ditto, don't even retrieve signature if trust store is missing. Thus the error message becomes.

notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: signature verification failed because the trust store "alpine" of type "ca" doesnot exist.

---

# no permission to read the certificate from the trust store
$ notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: failed to verify signature with digest sha256:f1356cb804623506ded33ecb78fe413a01ea1ce9b643b13456bf4ac7d2ff9698, failed to read the trusted certificate "/home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt": open /home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt: permission denied
Error: failed to verify signature with digest sha256:f7b98d8402ad46668884c73444b10ae787cbec882670303d4c1e733379eeab51, failed to read the trusted certificate "/home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt": open /home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt: permission denied
Error: signature verification failed for all the signatures associated with testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0

Ditto, don't even retrieve signature if notation cannot read certificate. Thus the error message becomes.

notation verify testnotation.azurecr.io/alpine@sha256:93d5a28ff72d288d69b5997b8ba47396d2cbb62a72b5d87cd3351094b5d578a0
Error: signature verification failed because unable to read the trusted certificate "/home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt": open /home/patrickzheng/.config/notation/truststore/x509/ca/alpine/alpine.crt: permission denied

[ghost]

@priteshbandi Thanks for the suggestion, but according to the Notary Project spec, signatures are verified before the signing identity. You could find the spec here, see the last question of FAQ.
This PR is for updating error messages only. If we want to change the verification workflow itself, then that should be another PR in notation-go.

[priteshbandi]

@priteshbandi Thanks for the suggestion, but according to the Notary Project spec, signatures are verified before the signing identity. You could find the spec here, see the last question of FAQ. This PR is for updating error messages only. If we want to change the verification workflow itself, then that should be another PR in notation-go.

Opened issue to amend spec #790