Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/patch-add-reply-review-config.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

70 changes: 66 additions & 4 deletions .github/workflows/smoke-copilot.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion .github/workflows/smoke-copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ safe-outputs:
create-pull-request-review-comment:
max: 5
submit-pull-request-review:
reply-to-pull-request-review-comment:
max: 5
add-labels:
allowed: [smoke-copilot]
allowed-repos: ["github/gh-aw"]
Expand Down Expand Up @@ -136,7 +138,7 @@ strict: true
9. **Build gh-aw**: Run `GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod make build` to verify the agent can successfully build the gh-aw project (both caches must be set to /tmp because the default cache locations are not writable). If the command fails, mark this test as ❌ and report the failure.
10. **Discussion Creation Testing**: Use the `create_discussion` safe-output tool to create a discussion in the announcements category titled "copilot was here" with the label "ai-generated"
11. **Workflow Dispatch Testing**: Use the `dispatch_workflow` safe output tool to trigger the `haiku-printer` workflow with a haiku as the message input. Create an original, creative haiku about software testing or automation.
12. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`.
12. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`. After submitting the review, use `reply_to_pull_request_review_comment` to reply to one of the inline comments you just created (use the `comment_id` returned by `create_pull_request_review_comment`).

## Output

Expand Down
11 changes: 11 additions & 0 deletions pkg/workflow/safe_outputs_config_generation.go
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,17 @@ func generateSafeOutputsConfig(data *WorkflowData) string {
1, // default max
)
}
if data.SafeOutputs.ReplyToPullRequestReviewComment != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good addition! This correctly follows the same pattern used for other safe-output tools like ResolvePullRequestReviewThread. The nil-guard ensures the config key is only emitted when the tool is explicitly configured.

additionalFields := newHandlerConfigBuilder().
AddTemplatableBool("footer", data.SafeOutputs.ReplyToPullRequestReviewComment.Footer).
Build()
safeOutputsConfig["reply_to_pull_request_review_comment"] = generateTargetConfigWithRepos(
data.SafeOutputs.ReplyToPullRequestReviewComment.SafeOutputTargetConfig,
data.SafeOutputs.ReplyToPullRequestReviewComment.Max,
10, // default max
additionalFields,
)
}
if data.SafeOutputs.ResolvePullRequestReviewThread != nil {
safeOutputsConfig["resolve_pull_request_review_thread"] = generateMaxConfig(
data.SafeOutputs.ResolvePullRequestReviewThread.Max,
Expand Down
62 changes: 62 additions & 0 deletions pkg/workflow/safe_outputs_config_generation_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -546,3 +546,65 @@ func TestGenerateSafeOutputsConfigEmptyRepoMemory(t *testing.T) {
_, hasPushRepoMemory := parsed["push_repo_memory"]
assert.False(t, hasPushRepoMemory, "push_repo_memory should not be present when Memories slice is empty")
}

// TestGenerateSafeOutputsConfigReplyToPullRequestReviewComment verifies that
// reply_to_pull_request_review_comment appears in config.json when configured.
// Previously this key was missing from generateSafeOutputsConfig, causing the
// safe-outputs MCP server to skip the tool at runtime.
func TestGenerateSafeOutputsConfigReplyToPullRequestReviewComment(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great test coverage! The comment explains exactly why this test exists (the tool was previously missing from config.json), making it easy to understand the regression this prevents.

data := &WorkflowData{
SafeOutputs: &SafeOutputsConfig{
ReplyToPullRequestReviewComment: &ReplyToPullRequestReviewCommentConfig{
BaseSafeOutputConfig: BaseSafeOutputConfig{Max: strPtr("25")},
},
},
}

result := generateSafeOutputsConfig(data)
require.NotEmpty(t, result, "Expected non-empty config")

var parsed map[string]any
require.NoError(t, json.Unmarshal([]byte(result), &parsed), "Result must be valid JSON")

replyConfig, ok := parsed["reply_to_pull_request_review_comment"].(map[string]any)
require.True(t, ok, "Expected reply_to_pull_request_review_comment key in config.json")
assert.InDelta(t, float64(25), replyConfig["max"], 0.0001, "max should be 25")
}

// TestGenerateSafeOutputsConfigReplyToPullRequestReviewCommentWithTarget verifies that
// target, target-repo, allowed_repos, and footer are forwarded to config.json.
func TestGenerateSafeOutputsConfigReplyToPullRequestReviewCommentWithTarget(t *testing.T) {
footerTrue := "true"
data := &WorkflowData{
SafeOutputs: &SafeOutputsConfig{
ReplyToPullRequestReviewComment: &ReplyToPullRequestReviewCommentConfig{
BaseSafeOutputConfig: BaseSafeOutputConfig{Max: strPtr("10")},
SafeOutputTargetConfig: SafeOutputTargetConfig{
Target: "pull_request",
TargetRepoSlug: "org/other-repo",
AllowedRepos: []string{"org/other-repo"},
},
Footer: &footerTrue,
},
},
}

result := generateSafeOutputsConfig(data)
require.NotEmpty(t, result, "Expected non-empty config")

var parsed map[string]any
require.NoError(t, json.Unmarshal([]byte(result), &parsed), "Result must be valid JSON")

replyConfig, ok := parsed["reply_to_pull_request_review_comment"].(map[string]any)
require.True(t, ok, "Expected reply_to_pull_request_review_comment key in config.json")
assert.InDelta(t, float64(10), replyConfig["max"], 0.0001, "max should be 10")
assert.Equal(t, "pull_request", replyConfig["target"], "target should be set")
assert.Equal(t, "org/other-repo", replyConfig["target-repo"], "target-repo should be set")

allowedRepos, ok := replyConfig["allowed_repos"].([]any)
require.True(t, ok, "allowed_repos should be an array")
assert.Len(t, allowedRepos, 1, "Should have 1 allowed repo")
assert.Equal(t, "org/other-repo", allowedRepos[0], "allowed_repos entry should match")

assert.Equal(t, true, replyConfig["footer"], "footer should be true")
}
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ jobs:
8. **Build gh-aw**: Run `GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod make build` to verify the agent can successfully build the gh-aw project (both caches must be set to /tmp because the default cache locations are not writable). If the command fails, mark this test as ❌ and report the failure.
9. **Discussion Creation Testing**: Use the `create_discussion` safe-output tool to create a discussion in the announcements category titled "copilot was here" with the label "ai-generated"
10. **Workflow Dispatch Testing**: Use the `dispatch_workflow` safe output tool to trigger the `haiku-printer` workflow with a haiku as the message input. Create an original, creative haiku about software testing or automation.
11. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`.
11. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`. After submitting the review, use `reply_to_pull_request_review_comment` to reply to one of the inline comments you just created (use the `comment_id` returned by `create_pull_request_review_comment`).

## Output

Expand Down
4 changes: 3 additions & 1 deletion pkg/workflow/testdata/wasm_golden/fixtures/smoke-copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ safe-outputs:
create-pull-request-review-comment:
max: 5
submit-pull-request-review:
reply-to-pull-request-review-comment:
max: 5
add-labels:
allowed: [smoke-copilot]
allowed-repos: ["github/gh-aw"]
Expand Down Expand Up @@ -130,7 +132,7 @@ strict: true
8. **Build gh-aw**: Run `GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod make build` to verify the agent can successfully build the gh-aw project (both caches must be set to /tmp because the default cache locations are not writable). If the command fails, mark this test as ❌ and report the failure.
9. **Discussion Creation Testing**: Use the `create_discussion` safe-output tool to create a discussion in the announcements category titled "copilot was here" with the label "ai-generated"
10. **Workflow Dispatch Testing**: Use the `dispatch_workflow` safe output tool to trigger the `haiku-printer` workflow with a haiku as the message input. Create an original, creative haiku about software testing or automation.
11. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`.
11. **PR Review Testing**: Review the diff of the current pull request. Leave 1-2 inline `create_pull_request_review_comment` comments on specific lines, then call `submit_pull_request_review` with a brief body summarizing your review and event `COMMENT`. After submitting the review, use `reply_to_pull_request_review_comment` to reply to one of the inline comments you just created (use the `comment_id` returned by `create_pull_request_review_comment`).

## Output

Expand Down