Skip to content

fix: safe-output-items.jsonl always empty and SafeItemsCount always 0#20899

Merged
pelikhan merged 5 commits intomainfrom
copilot/fix-empty-safe-output-items-jsonl
Mar 14, 2026
Merged

fix: safe-output-items.jsonl always empty and SafeItemsCount always 0#20899
pelikhan merged 5 commits intomainfrom
copilot/fix-empty-safe-output-items-jsonl

Conversation

Copy link
Contributor

Copilot AI commented Mar 14, 2026

safe-output-items.jsonl was always empty and SafeItemsCount always 0 for workflows executing modification-type safe outputs (e.g. add_labels, close_issue, update_issue). Only the 11 types in CREATE_ITEM_TYPES were ever logged; everything else was silently dropped.

Root cause

extractCreatedItemFromResult gated on CREATE_ITEM_TYPES membership and a non-empty URL. Modification handlers (e.g. add_labels) return { success, number, labelsAdded } — no URL, not in CREATE_ITEM_TYPES — so every call returned null. The Go side compounded this by skipping manifest entries with an empty url field.

Changes

  • safe_output_manifest.cjs — Replaced the 30+ type LOGGED_TYPES allowlist with a small NOT_LOGGED_TYPES exclusion set (noop, missing_tool, missing_data). extractCreatedItemFromResult now accepts any type not explicitly excluded — built-in handlers, custom safe job types, and future types are all logged automatically without needing to update any list. URL is treated as optional so modification types are logged as {type, number, timestamp} without a URL. createManifestLogger no longer drops URL-less entries.

  • safe_output_handler_manager.cjs — Extracted formatManifestLogMessage() helper to cleanly log both URL-bearing and URL-less manifest entries. When the handler manager routes a message to a custom safe output job, it now logs the dispatch to the manifest (using item_number/issue_number/pull_request_number from the message) so those operations are counted in SafeItemsCount.

  • audit_report.goextractCreatedItemsFromManifest now skips entries with an empty type (truly malformed) rather than empty url. CreatedItemReport.URL marked omitempty.

After this fix, an executed add_labels produces a manifest entry like:

{"type":"add_labels","number":20875,"timestamp":"2026-03-14T01:00:00.000Z"}

and SafeItemsCount correctly reflects all executed safe output operations, including those handled by custom safe jobs.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 2 commits March 14, 2026 01:53
Expand LOGGED_TYPES in safe_output_manifest.cjs to include all handler
types (add_labels, close_issue, update_issue, etc.), not just creation
types. Update extractCreatedItemFromResult to use LOGGED_TYPES and make
URL optional for modification types. Fix extractCreatedItemsFromManifest
in audit_report.go to skip entries without type instead of without URL.
Add formatManifestLogMessage helper. Update tests accordingly.

Fixes #(issue number)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot review code and be generic and also support safe jobs

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes safe output manifest logging so modification-type handlers (e.g., add_labels, close_issue, update_issue) are included in safe-output-items.jsonl, allowing SafeItemsCount to reflect all executed safe outputs rather than only creation types.

Changes:

  • Broaden manifest logging from CREATE_ITEM_TYPES to LOGGED_TYPES and make url optional in manifest entries.
  • Improve manifest log output formatting to avoid logging → undefined when no URL exists.
  • Update Go audit parsing and tests to accept manifest entries without url and to skip only truly malformed entries (missing type).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
actions/setup/js/safe_output_manifest.cjs Adds LOGGED_TYPES and updates manifest extraction/logging to allow URL-less modification entries.
actions/setup/js/safe_output_manifest.test.cjs Expands unit tests to cover LOGGED_TYPES behavior and URL-less entries.
actions/setup/js/safe_output_handler_manager.cjs Adds formatManifestLogMessage() to produce clean logs for entries with/without URL.
pkg/cli/audit_report.go Parses manifest entries even when url is absent; makes URL JSON field omitempty.
pkg/cli/audit_report_test.go Updates/extends tests to assert inclusion of modification entries and skipping entries missing type.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

pelikhan and others added 2 commits March 13, 2026 19:49
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…e jobs

Replace hardcoded LOGGED_TYPES (30+ types) with NOT_LOGGED_TYPES exclusion
set (noop, missing_tool, missing_data). Any type not in NOT_LOGGED_TYPES is
automatically logged — new handlers and custom safe job types included
without needing to update any list.

Log custom safe output job dispatches to manifest so SafeItemsCount
counts operations handled by user-defined safe jobs.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 14, 2026

@copilot review code and be generic and also support safe jobs

Done in 84aa8a2.

Generic: Replaced the 30+ hardcoded LOGGED_TYPES allowlist with a 3-type NOT_LOGGED_TYPES exclusion set (noop, missing_tool, missing_data). Any type not explicitly excluded is now logged automatically — new handler types and future types are picked up without updating any list.

Safe jobs: When the handler manager routes a message to a custom safe output job, it now logs the dispatch to the manifest (using item_number/issue_number/pull_request_number from the message) so those operations are counted in SafeItemsCount.

Copilot AI changed the title fix: safe-output-items.jsonl always empty and SafeItemsCount always 0 for modification types fix: safe-output-items.jsonl always empty and SafeItemsCount always 0 Mar 14, 2026
Copilot AI requested a review from pelikhan March 14, 2026 02:58
@pelikhan pelikhan merged commit ea378ee into main Mar 14, 2026
81 checks passed
@pelikhan pelikhan deleted the copilot/fix-empty-safe-output-items-jsonl branch March 14, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cli-tools-test] safe-output-items.jsonl always empty and SafeItemsCount always 0 in metrics despite safe outputs being executed

3 participants