JSON object is out printed when verification fails on `notation verify`

[shizhMSFT]

What is the areas you experience the issue in?

Notation CLI

What is not working as expected?

On verification failure,

$ notation verify <image_ref> --output json
signature verification failed for all the signatures associated with <image_ref>

The output is not in the JSON format and there is no failure details. Therefore, scripts cannot further process it.

What did you expect to happen?

The expected output would be

$ notation verify <image_ref> --output json
{
    "reference": "<image_ref>",
    "reason": "signature verification failed for all the signatures associated with <image_ref>",
    "details": [
        // detailed outcome
    ],
    "result": "Failed"
}

Related to #527 (comment)

[priteshbandi]

Moving conversation from PR to here
priteshbandi Feb 9, 2023 •

result can have two values, one is skip and other one is success. IMO for failure case displaying JSON for wont be useful because it wont contain any useful information which automation/script can use(non-zero exit code should suffice to show failure)

shizhMSFT Feb 9, 2023

The JSON output is usually consumed by scripts or programs. How can them obtain the structured error message and detailed verification outcomes?

priteshbandi Feb 9, 2023
To display error message/detailed verification output we need more than result field something like resultReason because result will only say failure. Also, in current state it wont be of much help since as there is only one genuine/expected failure use case i.e failed signature verification for all the signatures. Apart from genuine/expected error, all other errors should always be emitted as stdErr.

Are you suggest we should emit json for expected failure use cases or all failure use cases?

[shizhMSFT]

Are you suggest we should emit json for expected failure use cases or all failure use cases?

I am suggesting emitting json for expected failure use cases.

[byronchien]

is there a specific use case for consuming the messages programmatically? i feel like message strings like reason and details would be hard to write scripts around. the benefit i could see in keeping them would be that users can log the messages on skipped/failed result status, but notation should already be logging them stderr