Skip to content

Store secret verification result in step/job output for agent failure context#10408

Merged
pelikhan merged 7 commits intomainfrom
copilot/store-verification-result-output
Jan 17, 2026
Merged

Store secret verification result in step/job output for agent failure context#10408
pelikhan merged 7 commits intomainfrom
copilot/store-verification-result-output

Conversation

Copy link
Contributor

Copilot AI commented Jan 17, 2026 •
edited
Loading

Implementation Complete: Store Secret Verification Result in Step/Job Output

Summary

Successfully implemented a feature to capture secret verification results and pass them to the conclusion job's agent failure handler for better error context.

Changes Made

  1. Shell Script (validate_multi_secret.sh) - Added step output (verification_result)
  2. Go Code (agentic_engine.go) - Added id: validate-secret to validation step
  3. Agent Job (compiler_activation_jobs.go) - Added secret_verification_result output
  4. Conclusion Job (notify_comment.go) - Pass result to handle_agent_failure step
  5. Failure Handler (handle_agent_failure.cjs) - Display warning when verification failed
  6. Templates - Added conditional {secret_verification_context} placeholder
  7. Tests - Added comprehensive test coverage

Latest Changes

  • ✅ Ran make fmt - all code formatted successfully
  • ✅ Ran make lint-cjs - JavaScript linting passed
  • ✅ Ran make lint - Go linting passed
  • ✅ Recompiled workflows to reflect code changes

All formatting and linting checks pass successfully.

Original prompt

Store result of the verification of the secret in the step/job output

Use this value to create the agent failure issue/comment if it failed in the conclusion job

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Changeset

  • Type: patch
  • Description: Store secret verification result in step/job outputs so agent failure comments include secret verification context.

Ahoy! This treasure was crafted by 🏴‍☠️ Changeset Generator

Copilot AI and others added 3 commits January 17, 2026 04:16
@pelikhan
Add secret verification result output to agent job and conclusion job
e656a3d 
- Modified validate_multi_secret.sh to set step output with verification result
- Added id field to secret validation step in GenerateMultiSecretValidationStep
- Added secret_verification_result output to agent job
- Pass secret verification result to conclusion job handle_agent_failure step
- Updated handle_agent_failure.cjs to read and use secret verification result
- Added secret verification context to agent failure issue and comment templates
- Added tests to verify the new functionality

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Verify secret verification output implementation works correctly
2e3f64e 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Final validation - secret verification output feature complete
0940911 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Store result of secret verification in job output Store secret verification result in step/job output for agent failure context Jan 17, 2026
Copilot AI requested a review from pelikhan January 17, 2026 04:30
@pelikhan pelikhan added the smoke label Jan 17, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026 •
edited by agentic-workflows-dev bot
Loading

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026 •
edited
Loading

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026 •
edited
Loading

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026 •
edited
Loading

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026

chore: refine campaign generator success and failure messages
Fix invalid action SHAs in release workflow
GitHub MCP: ✅
Serena-go: ❌
Playwright: ✅
Tavily search: ✅
File write: ✅
Bash cat: ✅
Overall: FAIL

AI generated by Smoke Codex

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026

Smoke Test Results

Last 2 merged PRs:

  • chore: refine campaign generator success and failure messages
  • Fix invalid action SHAs in release workflow

✅ GitHub MCP - list/search PRs working
✅ Tavily Web Search - returned 3 results
✅ Playwright - navigated to github.com (title: "GitHub · Change is constant...")
✅ File Writing - created /tmp/gh-aw/agent/smoke-test-claude-21088621560.txt
✅ Bash Tool - verified file contents

Overall: PASS

AI generated by Smoke Claude

@github-actions
Copy link
Contributor

github-actions bot commented Jan 17, 2026

Smoke Test Results

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP: Successfully retrieved merged PRs
  • ❌ Serena Go: go command not found
  • ✅ Playwright: Page title verified "GitHub · Change is constant. GitHub keeps you ahead. · GitHub"
  • ✅ File Writing: Test file created successfully
  • ✅ Bash Tool: File read back successfully

Overall Status: FAIL (go tool unavailable)

cc @GH_AW_GITHUB_ACTOR

AI generated by Smoke Copilot

@pelikhan pelikhan marked this pull request as ready for review January 17, 2026 04:48
@pelikhan
Copy link
Contributor

pelikhan commented Jan 17, 2026

@copilot format, lint js, lint go

@pelikhan
Run formatting and linting as requested (format, lint js, lint go)
8d0d915 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 17, 2026

@copilot format, lint js, lint go

Done in commit 8d0d915. All formatting and linting checks passed:

  • ✅ make fmt - code formatted
  • ✅ make lint-cjs - JavaScript linting passed
  • ✅ make lint - Go linting passed

The workflow lock files were recompiled to reflect the code changes (secret verification output added to agent and conclusion jobs).

@pelikhan pelikhan merged commit feb9da8 into main Jan 17, 2026
46 of 47 checks passed
@pelikhan pelikhan deleted the copilot/store-verification-result-output branch January 17, 2026 05:35
@github-actions github-actions bot mentioned this pull request Jan 17, 2026
Closed
Copilot AI mentioned this pull request Jan 17, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pelikhan pelikhan Awaiting requested review from pelikhan

Assignees

@pelikhan pelikhan

Copilot code review Copilot

Labels

smoke smoke-claude

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pelikhan