Skip to content

docs: Add documentation for customizing code review via skills #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xingyaoww wants to merge 1 commit into from
Closed

docs: Add documentation for customizing code review via skills #306

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 93 additions & 84 deletions sdk/guides/github-workflows/pr-review.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,67 @@ The reference workflow triggers on either the "review-this" label or when the op
- **Fast Reviews** - Results posted on the PR in only 2 or 3 minutes
- **Comprehensive Analysis** - Analyzes the changes given the repository context. Covers code quality, security, best practices
- **GitHub Integration** - Posts comments directly to the PR
- **Customizable** - Add your own code review guidelines without forking

## Security

- Users with write access (maintainers) can trigger reviews by requesting `openhands-agent` as a reviewer or adding the `review-this` label.
- Maintainers need to read the PR to make sure it's safe to run.

## Customizing the Code Review

Instead of forking the `agent_script.py`, you can customize the code review behavior by adding a `.openhands/skills/code-review.md` file to your repository. This is the **recommended approach** for customization.

### How It Works

The PR review agent uses skills from the [OpenHands/skills](https://github.com/OpenHands/skills) repository by default. When you add a `.openhands/skills/code-review.md` file to your repository, it **overrides** the default skill with your custom guidelines.

### Example: Custom Code Review Skill

Create `.openhands/skills/code-review.md` in your repository:

```markdown
---
name: code-review
description: Custom code review guidelines for my project
triggers:
- /codereview
---

# My Project Code Review Guidelines

You are a code reviewer for this project. Follow these guidelines:

## Review Decisions

- **APPROVE** straightforward changes (config updates, typo fixes, documentation)
- **COMMENT** when you have feedback or concerns

## What to Check

- Code follows our project conventions
- Tests are included for new functionality
- No security vulnerabilities introduced
- Documentation is updated if needed

## Communication Style

- Be direct and constructive
- Use GitHub suggestion syntax for code fixes
- Approve quickly when code is good
```

### Benefits of Custom Skills

1. **No forking required**: Keep using the official SDK while customizing behavior
2. **Version controlled**: Your review guidelines live in your repository
3. **Easy updates**: SDK updates don't overwrite your customizations
4. **Team alignment**: Everyone uses the same review standards

<Note>
See the [software-agent-sdk's own code-review skill](https://github.com/OpenHands/software-agent-sdk/blob/main/.openhands/skills/code-review.md) for a complete example of a custom code review skill.
</Note>

## Reference Workflow

<Note>
Expand All @@ -51,13 +106,19 @@ This example is available on GitHub: [examples/03_github_workflows/02_pr_review/

```yaml icon="yaml" expandable examples/03_github_workflows/02_pr_review/workflow.yml
---
# OpenHands PR Review Workflow
#
# To set this up:
# 1. Copy this file to .github/workflows/pr-review.yml in your repository
# 2. Add your LLM_API_KEY to the repository secrets
# 3. Commit this file to your repository
# 4. Trigger the review by either:
# 2. Add LLM_API_KEY to repository secrets
# 3. Customize the inputs below as needed
# 4. Commit this file to your repository
# 5. Trigger the review by either:
# - Adding the "review-this" label to any PR, OR
# - Requesting openhands-agent as a reviewer
#
# For more information, see:
# https://github.com/OpenHands/software-agent-sdk/tree/main/examples/03_github_workflows/02_pr_review
name: PR Review by OpenHands

on:
Expand All @@ -77,97 +138,45 @@ jobs:
github.event.label.name == 'review-this' ||
github.event.requested_reviewer.login == 'openhands-agent'
runs-on: ubuntu-latest
env:
# Configuration (modify these values as needed)
LLM_MODEL: <YOUR_LLM_MODEL>
LLM_BASE_URL: <YOUR_LLM_BASE_URL>
# Review style: 'standard' for pragmatic review, 'roasted' for Linus-style
REVIEW_STYLE: standard
# PR context will be automatically provided by the agent script
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }}
PR_BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_HEAD_BRANCH: ${{ github.event.pull_request.head.ref }}
REPO_NAME: ${{ github.repository }}
steps:
- name: Checkout software-agent-sdk repository
- name: Checkout for composite action
uses: actions/checkout@v4
with:
repository: OpenHands/software-agent-sdk
path: software-agent-sdk
# Use a specific version tag or branch (e.g., 'v1.0.0' or 'main')
ref: main
sparse-checkout: .github/actions/pr-review

- name: Checkout PR repository
uses: actions/checkout@v4
with:
# Fetch the full history to get the diff
fetch-depth: 0
path: pr-repo
# Check out the feature branch so agent can inspect the PR changes
ref: ${{ github.event.pull_request.head.ref }}

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install uv
uses: astral-sh/setup-uv@v6
- name: Run PR Review
uses: ./.github/actions/pr-review
with:
enable-cache: true

- name: Install GitHub CLI
run: |
# Install GitHub CLI for posting review comments
sudo apt-get update
sudo apt-get install -y gh

- name: Install OpenHands dependencies
run: |
# Install OpenHands SDK and tools from local checkout
uv pip install --system ./software-agent-sdk/openhands-sdk ./software-agent-sdk/openhands-tools

- name: Check required configuration
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
run: |
if [ -z "$LLM_API_KEY" ]; then
echo "Error: LLM_API_KEY secret is not set."
exit 1
fi

echo "PR Number: $PR_NUMBER"
echo "PR Title: $PR_TITLE"
echo "Repository: $REPO_NAME"
echo "LLM model: $LLM_MODEL"
if [ -n "$LLM_BASE_URL" ]; then
echo "LLM base URL: $LLM_BASE_URL"
fi

- name: Run PR review
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Change to the PR repository directory so agent can analyze the code
cd pr-repo

# Run the PR review script from the software-agent-sdk checkout
uv run python ../software-agent-sdk/examples/03_github_workflows/02_pr_review/agent_script.py

- name: Upload logs as artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: openhands-pr-review-logs
path: |
*.log
output/
retention-days: 7
# LLM configuration
llm-model: anthropic/claude-sonnet-4-5-20250929
llm-base-url: ''
# Review style: roasted (other option: standard)
review-style: roasted
# SDK version to use (version tag or branch name)
sdk-version: main
# Secrets
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.GITHUB_TOKEN }}
```

### Action Inputs

| Input | Description | Required | Default |
|-------|-------------|----------|---------|
| `llm-model` | LLM model to use | No | `anthropic/claude-sonnet-4-5-20250929` |
| `llm-base-url` | LLM base URL (optional) | No | `''` |
| `review-style` | Review style: 'standard' or 'roasted' | No | `roasted` |
| `sdk-version` | Git ref for SDK (tag, branch, or commit SHA) | No | `main` |
| `sdk-repo` | SDK repository (owner/repo) | No | `OpenHands/software-agent-sdk` |
| `llm-api-key` | LLM API key | Yes | - |
| `github-token` | GitHub token for API access | Yes | - |

## Related Files

- [Agent Script](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/03_github_workflows/02_pr_review/agent_script.py)
- [Workflow File](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/03_github_workflows/02_pr_review/workflow.yml)
- [Prompt Template](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/03_github_workflows/02_pr_review/prompt.py)
- [Composite Action](https://github.com/OpenHands/software-agent-sdk/blob/main/.github/actions/pr-review/action.yml)