Skip to content

Comments

docs: Improve Notation Diagnostic Experience#1260

Merged
FeynmanZhou merged 6 commits intonotaryproject:mainfrom
FeynmanZhou:debug-log
May 13, 2025
Merged

docs: Improve Notation Diagnostic Experience#1260
FeynmanZhou merged 6 commits intonotaryproject:mainfrom
FeynmanZhou:debug-log

Conversation

@FeynmanZhou
Copy link
Member

@FeynmanZhou FeynmanZhou commented Apr 18, 2025

Resolve #1247

This proposal document aims to:

  1. Identify the usability issues of the --verbose and --debug options.
  2. Clarify the concepts of different types of output and logs for diagnostic purposes.
  3. List the guiding principles to write comprehensive, clear, and conducive debug output and debug logs for effective diagnosis.
  4. Propose solutions to improve the diagnostic experience for notation CLI users and developers.

Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the diagnostic experience for the notation CLI by documenting clearer guidelines and conventions around log outputs. It outlines the existing issues with verbose and debug flags, proposes enhanced log formatting, and includes detailed examples for reference.

  • Added a new specification document detailing diagnostic experience improvements.
  • Outlined concrete changes such as deprecating the --verbose flag, adding separator lines in logs, and enhancing timestamp precision.
  • Provided example debug log outputs with proposed improvements.

Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
@codecov
Copy link

codecov bot commented Apr 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.96%. Comparing base (12c60b7) to head (a386ca7).
Report is 14 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1260   +/-   ##
=======================================
  Coverage   76.96%   76.96%           
=======================================
  Files          68       68           
  Lines        3847     3847           
=======================================
  Hits         2961     2961           
  Misses        682      682           
  Partials      204      204           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ghost ghost changed the title doc: Improve Notation Diagnostic Experience docs: Improve Notation Diagnostic Experience Apr 18, 2025
Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
@FeynmanZhou FeynmanZhou requested review from a user, JeyJeyGao and yizha1 April 23, 2025 04:04
JeyJeyGao
JeyJeyGao previously approved these changes Apr 23, 2025
Copy link
Contributor

@JeyJeyGao JeyJeyGao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ghost
ghost previously approved these changes Apr 23, 2025
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just waiting for others' opinions and idea.

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just waiting for other opinions and idea.

@ghost ghost self-requested a review April 23, 2025 08:13
@ghost ghost dismissed their stale review April 23, 2025 08:14

LGTM, just waiting for other opinions and idea.

Copy link
Contributor

@yizha1 yizha1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FeynmanZhou it is much clear now. Thanks.

Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
Copy link
Contributor

@JeyJeyGao JeyJeyGao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

JeyJeyGao added a commit that referenced this pull request May 13, 2025
Pending on #1260

Example of notation version command
```
Notation - a tool to sign and verify artifacts.

Version:     v2.0.0-alpha.1+unreleased
Go version:  go1.24.0
OS/Arch:     linux/amd64
Git commit:  c68425f
```

Resolve part of #1247

---------

Signed-off-by: Junjie Gao <junjiegao@microsoft.com>
JeyJeyGao added a commit that referenced this pull request May 13, 2025
Pending on #1260

This PR removed the `--verbose` flag support as proposed in
#1260.

Resolve part of #1247

---------

Signed-off-by: Junjie Gao <junjiegao@microsoft.com>
@FeynmanZhou FeynmanZhou merged commit e97caa9 into notaryproject:main May 13, 2025
7 checks passed
FeynmanZhou pushed a commit to FeynmanZhou/notation that referenced this pull request May 15, 2025
Pending on notaryproject#1260

Example of notation version command
```
Notation - a tool to sign and verify artifacts.

Version:     v2.0.0-alpha.1+unreleased
Go version:  go1.24.0
OS/Arch:     linux/amd64
Git commit:  c68425f
```

Resolve part of notaryproject#1247

---------

Signed-off-by: Junjie Gao <junjiegao@microsoft.com>
FeynmanZhou pushed a commit to FeynmanZhou/notation that referenced this pull request May 15, 2025
…ct#1276)

Pending on notaryproject#1260

This PR removed the `--verbose` flag support as proposed in
notaryproject#1260.

Resolve part of notaryproject#1247

---------

Signed-off-by: Junjie Gao <junjiegao@microsoft.com>
FeynmanZhou added a commit to FeynmanZhou/notation that referenced this pull request May 15, 2025
Resolve notaryproject#1247 

This proposal document aims to:

1. Identify the usability issues of the `--verbose` and `--debug`
options.
2. Clarify the concepts of different types of output and logs for
diagnostic purposes.
3. List the guiding principles to write comprehensive, clear, and
conducive debug output and debug logs for effective diagnosis.
4. Propose solutions to improve the diagnostic experience for `notation`
CLI users and developers.

---------

Signed-off-by: Feynman Zhou <feynmanzhou@microsoft.com>
JeyJeyGao added a commit that referenced this pull request May 15, 2025
Pending on #1260

Fix:
- updated timestamp to UTC with nanosecond
- added new line after request and response log

Previous example:
```
DEBU[2025-04-14T07:42:25Z] Request #0
> Request: "HEAD" "https://notationreg.azurecr.io/v2/hello/manifests/v1"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "User-Agent": "notation/v2.0.0-alpha.1+unreleased"
DEBU[2025-04-14T07:42:26Z] Response #0
< Response status: "401 Unauthorized"
< Response headers:
   "Content-Length": "199"
   "X-Content-Type-Options": "nosniff"
   "Date": "Mon, 14 Apr 2025 07:42:26 GMT"
   "Server": "AzureContainerRegistry"
   "Connection": "keep-alive"
   "Strict-Transport-Security": "max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains"
   "Content-Type": "application/json; charset=utf-8"
   "Access-Control-Expose-Headers": "Docker-Content-Digest, WWW-Authenticate, Link, X-Ms-Correlation-Request-Id"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Www-Authenticate": "Bearer realm=\"https://notationreg.azurecr.io/oauth2/token\",service=\"notationreg.azurecr.io\",scope=\"repository:hello:pull\""
   "X-Ms-Correlation-Request-Id": "df5ee6ff-0afb-47d6-ab98-6835c2e41f8a"
DEBU[2025-04-14T07:42:26Z] Request #1
> Request: "POST" "https://notationreg.azurecr.io/oauth2/token"
> Request headers:
   "Content-Type": "application/x-www-form-urlencoded"
   "User-Agent": "notation/v2.0.0-alpha.1+unreleased"
```

Current example:
```                  
notation sign notationreg.azurecr.io/hello:v1 -d
DEBU[2025-05-12 07:41:42.720229189Z] Request #0
> Request: "HEAD" "https://notationreg.azurecr.io/v2/hello/manifests/v1"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "User-Agent": "notation/v2.0.0-alpha.1+unreleased"


DEBU[2025-05-12 07:41:43.248214367Z] Response #0
< Response status: "401 Unauthorized"
< Response headers:
   "Date": "Mon, 12 May 2025 07:41:43 GMT"
   "Server": "AzureContainerRegistry"
   "Access-Control-Expose-Headers": "Docker-Content-Digest, WWW-Authenticate, Link, X-Ms-Correlation-Request-Id"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Strict-Transport-Security": "max-age=31536000; includeSubDomains, max-age=31536000; includeSubDomains"
   "X-Content-Type-Options": "nosniff"
   "X-Ms-Correlation-Request-Id": "0c11d32c-4ffe-480c-acb3-8d145a83a87e"
   "Content-Type": "application/json; charset=utf-8"
   "Content-Length": "199"
   "Connection": "keep-alive"
   "Www-Authenticate": "Bearer realm=\"https://notationreg.azurecr.io/oauth2/token\",service=\"notationreg.azurecr.io\",scope=\"repository:hello:pull\""


DEBU[2025-05-12 07:41:43.248394566Z] Request #1
> Request: "POST" "https://notationreg.azurecr.io/oauth2/token"
> Request headers:
   "Content-Type": "application/x-www-form-urlencoded"
   "User-Agent": "notation/v2.0.0-alpha.1+unreleased"


DEBU[2025-05-12 07:41:43.414852708Z] Response #1
< Response status: "200 OK"
< Response headers:
   "Connection": "keep-alive"
   "X-Ms-Correlation-Request-Id": "d4baeae0-e47e-4b6f-8767-a7002229a1d6"
   "X-Ms-Ratelimit-Remaining-Calls-Per-Second": "333.316667"
   "Strict-Transport-Security": "max-age=31536000; includeSubDomains"
   "Server": "AzureContainerRegistry"
   "Date": "Mon, 12 May 2025 07:41:43 GMT"
   "Content-Type": "application/json; charset=utf-8"
```

Color:

![image](https://github.com/user-attachments/assets/6a54c33e-2f84-4e4b-a88a-49f1877a2b29)

![image](https://github.com/user-attachments/assets/08173d5c-4327-4545-8ee1-7be94babde18)

![image](https://github.com/user-attachments/assets/c9bc1270-58bd-42f8-9fcf-819779a16687)

![image](https://github.com/user-attachments/assets/2c3a85b9-5118-4e19-9ae1-7457174a934a)

Resolve part of #1247

---------

Signed-off-by: Junjie Gao <junjiegao@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve the diagnostic experience by deprecating the flag "--verbose" and standardize the debug logs

4 participants