Add spec for verify json output

[byronchien]

Adds spec for notation verify $IMAGE --output json.

Changes from previous iteration:

• adds a details field for additional information related to the result
• outputs a JSON object on failure and skipped results

Related issues and PRs:

• feat: add support for json output for notation verify #527
• JSON object is out printed when verification fails on notation verify #546
• SkippedByTrustPolicy from JSON output of notation verify is not documented #547

[FeynmanZhou]

@byronchien You may need to rebase your branch and sync with the upstream main as it has conflicts.

[codecov-commenter]

Codecov Report

Merging #559 (f71dc76) into main (0391202) will not change coverage.
The diff coverage is n/a.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@           Coverage Diff           @@
##             main     #559   +/-   ##
=======================================
  Coverage   63.52%   63.52%           
=======================================
  Files          40       40           
  Lines        2237     2237           
=======================================
  Hits         1421     1421           
  Misses        695      695           
  Partials      121      121           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[priteshbandi]

LGTM

[priteshbandi]

LGTM

[priteshbandi]

LGTM

[sajayantony]

+1. I think it's worth merging this. Typically it would be good to call out all possible values of result if there are more.

[FeynmanZhou]

LGTM

[shizhMSFT]

Should it be merged post v1?

[yizha1]

Should it be merged post v1?

+1 on merging it post v1.

[ghost]

Can you also add this new flag along with its description into the Outline session of this spec? (https://github.com/notaryproject/notation/blob/main/specs/commandline/verify.md#outline)

[ghost]

Just curious, what error message are we expecting from the following command:
notation verify --output <something_not_defined> localhost:5000/net-monitor@sha256:b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

[yizha1]

@byronchien I reviewed this PR again. Here are two comments:

• Unified solution for all Notation CLI commands: I couldn't recall any reason behind this PR that only verify command is addressed. The Json output should be considered for all other Notation CLI commands. So, it is suggested to create a unified solution for all Notation CLI commands.
• About the flag name -o --output: There is a new feature about signing and verifying arbitrary files. For this feature, we may need to output signature to a file during signing, there may be a conflict of using --output. It seems notation inspect command already uses the flag --output. We need to align the usage of flag --output.
  /cc: @priteshbandi @shizhMSFT @FeynmanZhou

[github-actions]

This PR is stale because it has been opened for 45 days with no activity. Remove stale label or comment. Otherwise, it will be closed in 30 days.

[github-actions]

PR closed due to no activity in the past 30 days.