diff --git a/README.md b/README.md index 751de92..c31429a 100644 --- a/README.md +++ b/README.md @@ -228,6 +228,7 @@ The `code-simplifier` agent is vendored from [Anthropic's official plugins](http - [Agent Skills Specification](https://agentskills.io/specification) - [Sentry Engineering Practices](https://develop.sentry.dev/engineering-practices/) +- [Skill Synthesis Guide](docs/skill-synthesis.md) — using your AI agent to research and improve skills ## License diff --git a/docs/skill-synthesis.md b/docs/skill-synthesis.md new file mode 100644 index 0000000..1662541 --- /dev/null +++ b/docs/skill-synthesis.md @@ -0,0 +1,66 @@ +# Skill Synthesis Guide + +Skill synthesis is using your AI coding agent to research upstream implementations, specs, and best practices online before creating a skill. Instead of writing from scratch, you have the agent survey what already exists and synthesize the best ideas into your skill. + +This pairs with the [skill-creator](../plugins/sentry-skills/skills/skill-creator/SKILL.md) skill — synthesis handles the research phase, skill-creator handles the implementation. + +## Key Sources + +Point your agent at these when researching: + +| Source | What it covers | +|--------|---------------| +| [Agent Skills Specification](https://agentskills.io/specification) | The spec — frontmatter fields, structure, constraints | +| [Anthropic's official skills](https://github.com/anthropics/skills) | Anthropic's reference implementations | +| [OpenAI Codex's skills](https://github.com/openai/skills) | OpenAI's approach to the same problems | +| [Skills best practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices) | Claude's guidance on writing effective skills | +| [Claude Code skills docs](https://code.claude.com/docs/en/skills) | How skills integrate with Claude Code | + +Not every source is relevant to every skill. If you're creating a code review skill, look for upstream code review skills. If you're creating something novel, focus on the spec and best practices. + +## The Workflow + +### 1. Research — tell the agent what to look for + +Give the agent the concept and ask it to research before writing anything: + +``` +I want to create a skill for Django performance reviews. Before writing it, +research upstream implementations — check Anthropic's and OpenAI's skill repos +for anything similar, read the Agent Skills spec, and look at the best practices +docs. +``` + +The agent will fetch sources, cross-reference approaches, and report back what it found. + +### 2. Review — understand what the agent found + +The agent should show you: +- What patterns it found upstream worth adopting +- What's unique to our context that upstream skills don't cover +- How it recommends structuring the skill + +This is your checkpoint. Redirect before writing, not after. + +### 3. Create — hand off to skill-creator + +Once you agree on the approach, use the skill-creator to build the skill informed by the research: + +``` +Now use /skill-creator to create the skill, incorporating what you found. +``` + +The synthesis research stays in context, so the skill-creator produces a better result than it would from a cold start. + +## When To Do This + +- **Creating any non-trivial skill** — if upstream implementations exist, research them first +- **Porting a skill from another repo** — understand the original before adapting +- **Rebuilding a skill** — when a rewrite is needed, check the current state of the art + +## Tips + +- **Research before writing.** The whole point is that the agent creates a better skill because it looked at what already exists. +- **Review the research, not just the output.** Understand what the agent pulled from where — some upstream patterns won't fit your context. +- **Check attribution.** If the agent adopts patterns from a specific upstream skill, follow the [vendoring guidelines](../README.md#vendoring-skills). +- **One skill at a time.** Synthesizing multiple skills in one session gets messy. diff --git a/plugins/sentry-skills/skills/skill-creator/SKILL.md b/plugins/sentry-skills/skills/skill-creator/SKILL.md index a482574..20e9b09 100644 --- a/plugins/sentry-skills/skills/skill-creator/SKILL.md +++ b/plugins/sentry-skills/skills/skill-creator/SKILL.md @@ -1,12 +1,17 @@ --- name: skill-creator -description: Create new agent skills following the Agent Skills specification. Use when asked to "create a skill", "add a new skill", "write a skill", "make a skill", "build a skill", or scaffold a new skill with SKILL.md. Guides through requirements, writing, registration, and verification. +description: Create new agent skills following the Agent Skills specification. Use when asked to "create a skill", "add a new skill", "write a skill", "make a skill", "build a skill", or scaffold a new skill with SKILL.md. Guides through requirements, planning, writing, registration, and verification. --- # Create a New Skill @@ -19,13 +24,15 @@ Gather requirements before writing anything. **Ask the user:** 1. What should this skill do? (one sentence) -2. When should an agent use it? (trigger phrases) +2. When should an agent use it? (trigger phrases users would say) 3. What tools does the skill need? (Read, Grep, Glob, Bash, Task, WebFetch, etc.) 4. Where should the skill live? (which plugin or directory) **Determine the skill name:** -- Lowercase alphanumeric with hyphens, 1-64 characters +- Lowercase letters, digits, and hyphens only (`a-z`, `0-9`, `-`) +- 1-64 characters; must not start or end with `-`; no consecutive hyphens (`--`) - Descriptive and unique among existing skills +- Prefer action-oriented names: `processing-pdfs`, `fix-issue`, `code-review` - Check the target skills directory to avoid name collisions **Choose a complexity tier:** @@ -39,7 +46,21 @@ Gather requirements before writing anything. Read `${CLAUDE_SKILL_ROOT}/references/design-principles.md` for guidance on keeping skills focused and concise. -## Step 2: Study Existing Skills +## Step 2: Plan the Skill + +Analyze how each use case would be executed from scratch. Identify what reusable resources would help when executing these tasks repeatedly. + +For each concrete example, ask: +1. What code would be rewritten every time? → candidate for `scripts/` +2. What documentation is needed to inform decisions? → candidate for `references/` +3. What templates or assets are used in output? → candidate for `assets/` + +Example analysis: +- "Rotate a PDF" → rotating requires rewriting the same code → `scripts/rotate_pdf.py` +- "Query BigQuery metrics" → need table schemas each time → `references/schema.md` +- "Build a frontend app" → same boilerplate HTML/React → `assets/hello-world/` + +## Step 3: Study Existing Skills Before writing, study 1-2 existing skills that match the chosen tier. Look for skills in the target repository or plugin to understand local conventions. @@ -47,7 +68,7 @@ Read `${CLAUDE_SKILL_ROOT}/references/skill-patterns.md` for concrete examples o Also read `CLAUDE.md` (or `AGENTS.md`) at the repository root for repo-specific conventions that the skill should follow. -## Step 3: Write the SKILL.md +## Step 4: Write the SKILL.md Create `//SKILL.md`. @@ -64,11 +85,34 @@ description: . Use when . . **Required fields:** - `name` — must match the directory name exactly -- `description` — up to 1024 chars; include trigger keywords that help agents match user intent +- `description` — up to 1024 chars, no angle brackets (`<` or `>`); include trigger keywords that help agents match user intent **Optional fields:** - `allowed-tools` — comma-separated list (e.g., `Read, Grep, Glob, Bash`); omit to allow all tools - `license` — license name or path (add when vendoring external content) +- `metadata` — arbitrary key-value mapping for additional metadata +- `compatibility` — environment requirements (max 500 chars); most skills don't need this + +For Claude Code-specific fields (`argument-hint`, `disable-model-invocation`, `context`, etc.), read `${CLAUDE_SKILL_ROOT}/references/claude-code-extensions.md`. + +### Description Guidelines + +The description is the **primary trigger mechanism** — it determines when agents activate the skill. All "when to use" information belongs here, not in the body. + +**Write in third person:** +- Good: "Processes Excel files and generates reports. Use when..." +- Bad: "I can help you process Excel files" or "You can use this to..." + +**Include natural trigger phrases:** +```yaml +# Good — specific triggers users would actually say +description: Security code review for vulnerabilities. Use when asked to "security review", "find vulnerabilities", "check for security issues", "audit security". + +# Bad — too vague, no trigger phrases +description: A helpful skill for code quality. +``` + +**Pattern:** `. Use when . .` ### Body Guidelines @@ -87,11 +131,17 @@ Write the body in **imperative voice** — these are instructions, not documenta 4. Include concrete examples of expected output 5. End with validation criteria or exit conditions +For workflow and output patterns, read: +- `${CLAUDE_SKILL_ROOT}/references/workflow-patterns.md` — sequential workflows, feedback loops, plan-validate-execute +- `${CLAUDE_SKILL_ROOT}/references/output-patterns.md` — template, examples, and structured data patterns + **Size limits:** -- Keep SKILL.md under **500 lines** +- Keep SKILL.md under **500 lines** (< 5000 tokens recommended) - If approaching the limit, move reference material to `references/` files - Load reference files conditionally based on context (not all at once) +**Use consistent terminology** — pick one term for each concept and stick with it throughout. Don't alternate between "API endpoint", "URL", "route", and "path". + ### Attribution If the skill is based on or adapted from external sources, add an HTML comment **after** the frontmatter closing `---`: @@ -108,7 +158,18 @@ https://github.com/example/original-source --> ``` -## Step 4: Create Supporting Files +## Step 5: Create Supporting Files + +### What to Include + +Only include files that directly support the skill's function. + +### What NOT to Include + +Do not create extraneous documentation or auxiliary files: +- README.md, INSTALLATION_GUIDE.md, QUICK_REFERENCE.md, CHANGELOG.md + +A skill should contain only what an agent needs to do the job. Not setup procedures, not user-facing docs, not development history. ### References (`references/`) @@ -127,7 +188,12 @@ Reference from SKILL.md with: Read `${CLAUDE_SKILL_ROOT}/references/topic-a.md` for details on [topic]. ``` -Keep each reference file focused on one topic. Use markdown with tables and code blocks. +Guidelines: +- Keep each reference file focused on one topic +- Keep references **one level deep** from SKILL.md (no nested reference chains) +- For files over 100 lines, add a table of contents at the top +- For files over 10k words, include grep search patterns in SKILL.md +- Information should live in either SKILL.md or references, not both ### Scripts (`scripts/`) @@ -154,16 +220,32 @@ Use for workflow automation that benefits from structured Python. - Output structured JSON for agent consumption - Run from the **repository root**, not the skill directory - Document the script's interface in SKILL.md (arguments, output format) +- Handle errors explicitly — don't punt to the agent ### Assets (`assets/`) -Use for static files the skill references (templates, configs, etc.). +Use for static files used in the skill's output (templates, images, boilerplate code, fonts). These are not loaded into context — they're copied or used directly. ### LICENSE Include a LICENSE file in the skill directory when vendoring content with specific licensing requirements. -## Step 5: Register the Skill +## Step 6: Validate the Skill + +Run the validation script to catch issues early: + +```bash +uv run ${CLAUDE_SKILL_ROOT}/scripts/quick_validate.py +``` + +The script checks frontmatter format, required fields, naming rules, and common mistakes. Fix any errors and re-run until validation passes. + +Alternatively, use the upstream validation tool: +```bash +skills-ref validate +``` + +## Step 7: Register the Skill Registration steps vary by repository. Check the repository's `CLAUDE.md` or `README.md` for specific instructions. @@ -172,14 +254,16 @@ Registration steps vary by repository. Check the repository's `CLAUDE.md` or `RE 3. **Update permissions** — if the repo has `.claude/settings.json`, add `Skill(:)` to the `permissions.allow` array 4. **Check CLAUDE.md** — read the repository's `CLAUDE.md` for any additional registration steps specific to that project -## Step 6: Verify +## Step 8: Verify Run through this checklist before finishing: ### Frontmatter - [ ] `name` matches directory name -- [ ] `description` is under 1024 characters -- [ ] `description` includes trigger keywords +- [ ] `name` uses only lowercase letters, digits, hyphens (no leading/trailing/consecutive hyphens) +- [ ] `description` is under 1024 characters, no angle brackets +- [ ] `description` is in third person and includes trigger keywords +- [ ] All "when to use" info is in description, not in body - [ ] No content before the opening `---` ### Content @@ -187,7 +271,9 @@ Run through this checklist before finishing: - [ ] Written in imperative voice - [ ] Steps are numbered and clear - [ ] Examples of expected output included +- [ ] Consistent terminology throughout - [ ] Reference files loaded conditionally (not unconditionally) +- [ ] No extraneous files (README.md, CHANGELOG.md, etc.) ### Registration - [ ] Directory name matches frontmatter `name` @@ -199,6 +285,11 @@ Run through this checklist before finishing: - [ ] Uses `uv run ${CLAUDE_SKILL_ROOT}/scripts/...` - [ ] Has PEP 723 inline metadata - [ ] Outputs structured JSON +- [ ] Handles errors explicitly - [ ] Documented in SKILL.md +### Validation +- [ ] `uv run ${CLAUDE_SKILL_ROOT}/scripts/quick_validate.py` passes +- [ ] Tested with a real usage scenario + Report any issues found and fix them before completing. diff --git a/plugins/sentry-skills/skills/skill-creator/references/claude-code-extensions.md b/plugins/sentry-skills/skills/skill-creator/references/claude-code-extensions.md new file mode 100644 index 0000000..24075e5 --- /dev/null +++ b/plugins/sentry-skills/skills/skill-creator/references/claude-code-extensions.md @@ -0,0 +1,111 @@ +# Claude Code Extensions + +Claude Code extends the [Agent Skills specification](https://agentskills.io/specification) with additional frontmatter fields and features. These are optional — skills that use only the base spec remain portable across all compatible tools. + +## Extended Frontmatter Fields + +| Field | Description | +|-------|-------------| +| `argument-hint` | Hint shown during autocomplete (e.g., `[issue-number]`, `[filename] [format]`) | +| `disable-model-invocation` | Set `true` to prevent Claude from auto-loading; manual `/name` only | +| `user-invocable` | Set `false` to hide from `/` menu; background knowledge only | +| `model` | Override the model when this skill is active | +| `context` | Set to `fork` to run in an isolated subagent | +| `agent` | Which subagent type to use with `context: fork` (e.g., `Explore`, `Plan`) | +| `hooks` | Hooks scoped to the skill's lifecycle | + +### Invocation Control + +By default, both users and Claude can invoke any skill. Two fields restrict this: + +```yaml +# Only the user can invoke (manual trigger for side-effects like deploy) +disable-model-invocation: true + +# Only Claude can invoke (background knowledge, not a meaningful user action) +user-invocable: false +``` + +| Setting | User can invoke | Claude can invoke | +|---------|----------------|-------------------| +| (default) | Yes | Yes | +| `disable-model-invocation: true` | Yes | No | +| `user-invocable: false` | No | Yes | + +### Subagent Execution + +Set `context: fork` to run a skill in an isolated subagent. The skill content becomes the prompt — the subagent won't have access to conversation history. + +```yaml +--- +name: deep-research +description: Research a topic thoroughly +context: fork +agent: Explore +--- + +Research $ARGUMENTS thoroughly: +1. Find relevant files using Glob and Grep +2. Read and analyze the code +3. Summarize findings with specific file references +``` + +Only use `context: fork` with skills that contain explicit tasks. Skills that provide guidelines without a task ("use these API conventions") return nothing useful from a subagent. + +## String Substitutions + +Skills support dynamic values in content: + +| Variable | Description | +|----------|-------------| +| `$ARGUMENTS` | All arguments passed when invoking the skill | +| `$ARGUMENTS[N]` | Specific argument by 0-based index | +| `$N` | Shorthand for `$ARGUMENTS[N]` (e.g., `$0`, `$1`) | +| `${CLAUDE_SESSION_ID}` | Current session ID | + +```yaml +--- +name: fix-issue +description: Fix a GitHub issue +disable-model-invocation: true +--- + +Fix GitHub issue $ARGUMENTS following our coding standards. +``` + +If `$ARGUMENTS` is not present in the content, arguments are appended as `ARGUMENTS: `. + +## Dynamic Context Injection + +The `` !`command` `` syntax runs shell commands before the skill content reaches Claude. Output replaces the placeholder. + +```yaml +--- +name: pr-summary +description: Summarize changes in a pull request +context: fork +agent: Explore +--- + +## Pull request context +- PR diff: !`gh pr diff` +- PR comments: !`gh pr view --comments` + +## Your task +Summarize this pull request. +``` + +Commands execute immediately as preprocessing — Claude only sees the output. + +## Skill Locations + +| Level | Path | Scope | +|-------|------|-------| +| Enterprise | Managed settings | All org users | +| Personal | `~/.claude/skills//SKILL.md` | All your projects | +| Project | `.claude/skills//SKILL.md` | This project | +| Plugin | `/skills//SKILL.md` | Where plugin is enabled | + +Higher-priority locations win when names collide (enterprise > personal > project). Plugin skills use `plugin-name:skill-name` namespacing. + +In monorepos, Claude Code auto-discovers skills from nested `.claude/skills/` directories relative to the files being edited. diff --git a/plugins/sentry-skills/skills/skill-creator/references/design-principles.md b/plugins/sentry-skills/skills/skill-creator/references/design-principles.md index 824e6fc..cec6963 100644 --- a/plugins/sentry-skills/skills/skill-creator/references/design-principles.md +++ b/plugins/sentry-skills/skills/skill-creator/references/design-principles.md @@ -57,6 +57,20 @@ This keeps the base context small while making deep knowledge available when nee The `description` field determines when agents activate the skill. It must contain the phrases users actually say. +**Write in third person** — the description is injected into the system prompt, and inconsistent point-of-view causes discovery problems: +```yaml +# Good — third person +description: Processes Excel files and generates reports. Use when working with spreadsheets. + +# Bad — first person +description: I can help you process Excel files. + +# Bad — second person +description: You can use this to process Excel files. +``` + +**Include all "when to use" information in the description**, not in the body. The body is only loaded after triggering, so "When to Use This Skill" sections in the body are not helpful. + **Effective descriptions:** ```yaml # Good — includes natural trigger phrases @@ -92,3 +106,60 @@ Skills are instructions to an agent, not documentation for humans. Write in impe | Skip test files unless explicitly requested | Test files are generally skipped unless explicitly requested | The agent interprets imperative instructions as direct commands. Descriptive language introduces ambiguity about whether an action is required or optional. + +## Consistent Terminology + +Pick one term for each concept and use it throughout the skill. Inconsistent terminology confuses agents and leads to inconsistent behavior. + +| Do (pick one) | Don't (mix these) | +|---------------|-------------------| +| "API endpoint" everywhere | "API endpoint", "URL", "API route", "path" | +| "field" everywhere | "field", "box", "element", "control" | +| "extract" everywhere | "extract", "pull", "get", "retrieve" | + +## Avoid Duplication + +Information should live in either SKILL.md or reference files, not both. Prefer reference files for detailed content and SKILL.md for the core procedural workflow. + +Similarly, don't repeat conventions already in `CLAUDE.md` or `AGENTS.md`. Reference them instead: "Follow the commit conventions in CLAUDE.md" rather than copying the entire format spec. + +## Avoid Time-Sensitive Information + +Don't include information that will become outdated: + +```markdown +# Bad — will become wrong +If you're doing this before August 2025, use the old API. + +# Good — use "old patterns" section +## Current method +Use the v2 API endpoint. + +## Legacy patterns (deprecated) +The v1 API is no longer supported. +``` + +## Long Reference Files + +For reference files longer than 100 lines, include a table of contents at the top so agents can see the full scope when previewing: + +```markdown +# API Reference + +## Contents +- Authentication and setup +- Core methods (create, read, update, delete) +- Advanced features (batch operations, webhooks) +- Error handling patterns + +## Authentication and setup +... +``` + +For very large reference files (>10k words), include grep search patterns in SKILL.md so agents can find relevant sections: + +```markdown +Find specific metrics using grep: +- Revenue data: `grep -i "revenue" ${CLAUDE_SKILL_ROOT}/references/finance.md` +- Pipeline data: `grep -i "pipeline" ${CLAUDE_SKILL_ROOT}/references/sales.md` +``` diff --git a/plugins/sentry-skills/skills/skill-creator/references/output-patterns.md b/plugins/sentry-skills/skills/skill-creator/references/output-patterns.md new file mode 100644 index 0000000..4033fc0 --- /dev/null +++ b/plugins/sentry-skills/skills/skill-creator/references/output-patterns.md @@ -0,0 +1,118 @@ +# Output Patterns + +Patterns for producing consistent, high-quality output from skills. + +## Template Pattern + +Provide templates when the skill must produce a specific format. Match strictness to requirements. + +**Strict (for API responses, reports, data formats):** + +```markdown +## Report structure + +ALWAYS use this exact template: + +# [Analysis Title] + +## Executive summary +[One-paragraph overview of key findings] + +## Key findings +- Finding 1 with supporting data +- Finding 2 with supporting data + +## Recommendations +1. Specific actionable recommendation +2. Specific actionable recommendation +``` + +**Flexible (when adaptation is useful):** + +```markdown +## Report structure + +Use this as a sensible default, but adapt based on context: + +# [Analysis Title] + +## Executive summary +[Overview] + +## Key findings +[Adapt sections based on what you discover] + +## Recommendations +[Tailor to the specific context] +``` + +## Examples Pattern + +When output quality depends on style or format, provide input/output pairs: + +````markdown +## Commit message format + +Generate commit messages following these examples: + +**Example 1:** +Input: Added user authentication with JWT tokens +Output: +``` +feat(auth): implement JWT-based authentication + +Add login endpoint and token validation middleware +``` + +**Example 2:** +Input: Fixed bug where dates displayed incorrectly +Output: +``` +fix(reports): correct date formatting in timezone conversion + +Use UTC timestamps consistently across report generation +``` + +Follow this style: type(scope): brief description, then detailed explanation. +```` + +Examples help agents understand desired style and detail level more clearly than descriptions alone. + +## Decision Table Pattern + +Use tables when the output format depends on input characteristics: + +```markdown +## Output format selection + +| Input Type | Output Format | Example | +|-----------|--------------|---------| +| Single file | Inline summary | "Found 3 issues in auth.py: ..." | +| Multiple files | Grouped report | Markdown report with per-file sections | +| Full repository | Executive summary + details | Summary table + expandable sections | +``` + +## Structured Data Pattern + +When scripts or downstream tools consume the output, specify the exact schema: + +````markdown +## Output format + +Return results as JSON: + +```json +{ + "status": "success" | "failure", + "findings": [ + { + "severity": "HIGH" | "MEDIUM" | "LOW", + "file": "path/to/file.py", + "line": 42, + "message": "Description of the finding" + } + ], + "summary": "One-line summary of results" +} +``` +```` diff --git a/plugins/sentry-skills/skills/skill-creator/references/skill-patterns.md b/plugins/sentry-skills/skills/skill-creator/references/skill-patterns.md index d982c0c..faeac41 100644 --- a/plugins/sentry-skills/skills/skill-creator/references/skill-patterns.md +++ b/plugins/sentry-skills/skills/skill-creator/references/skill-patterns.md @@ -49,6 +49,7 @@ iterate-pr/ - Invoked with `uv run ${CLAUDE_SKILL_ROOT}/scripts/script_name.py` - Scripts run from the **repository root**, not the skill directory - Scripts output structured JSON for agent consumption +- Scripts handle errors explicitly — don't punt to the agent - SKILL.md includes a fallback section for when scripts fail **When to use:** The workflow benefits from structured data extraction, API calls, or processing that would be fragile as inline bash commands. @@ -88,10 +89,42 @@ security-review/ ``` - Each reference file is self-contained and focused on one topic - SKILL.md includes a file index so the agent knows what's available +- References are one level deep from SKILL.md (no nested chains) +- Files over 100 lines include a table of contents at the top - LICENSE included because content is adapted from external sources **When to use:** The domain is too large for one file, but the agent only needs a subset for any given task. Progressive disclosure keeps context small. +## Argument-Accepting Skills + +Use when the skill takes user input as parameters. + +**Structure:** +```yaml +--- +name: fix-issue +description: Fix a GitHub issue by number. Use when asked to fix, resolve, or address a GitHub issue. +disable-model-invocation: true +argument-hint: "[issue-number]" +--- + +Fix GitHub issue $ARGUMENTS following our coding standards. + +1. Read the issue description +2. Implement the fix +3. Write tests +4. Create a commit +``` + +**Pattern highlights:** +- `$ARGUMENTS` is replaced with whatever follows `/fix-issue` (e.g., `/fix-issue 123`) +- `$ARGUMENTS[N]` or `$N` accesses individual arguments by position +- `argument-hint` provides autocomplete guidance +- `disable-model-invocation: true` prevents Claude from triggering it automatically (appropriate for side-effect-heavy workflows) +- If `$ARGUMENTS` is absent from the content, arguments are appended as `ARGUMENTS: ` + +**Note:** These features are Claude Code extensions. See `${CLAUDE_SKILL_ROOT}/references/claude-code-extensions.md`. + ## Anti-Patterns ### Over-long SKILL.md @@ -106,6 +139,12 @@ security-review/ **Fix:** Include the actual phrases users say: `Use when asked to "review code", "find bugs", "check for issues"`. +### Trigger Info in Body Instead of Description + +**Problem:** The body includes a "When to Use This Skill" section, but the description is vague. The body is only loaded *after* triggering, so this information never helps with skill selection. + +**Fix:** Move all "when to use" information into the `description` field. The body should contain *how* to execute, not *when* to activate. + ### Duplicating CLAUDE.md **Problem:** SKILL.md repeats repo conventions already in CLAUDE.md (commit format, PR process, etc.). @@ -124,6 +163,18 @@ security-review/ | JavaScript | `references/javascript.md` | ``` +### Large References Without Navigation + +**Problem:** A reference file is 500+ lines with no table of contents. Agents preview with partial reads and miss important sections. + +**Fix:** Add a table of contents at the top of files over 100 lines. For very large files (>10k words), include grep patterns in SKILL.md. + +### Extraneous Files + +**Problem:** The skill directory includes README.md, CHANGELOG.md, INSTALLATION_GUIDE.md, or other documentation files. + +**Fix:** A skill should only contain files an agent needs to do the job: SKILL.md, references, scripts, assets, and LICENSE. Remove user-facing docs, development history, and setup guides. + ### Scripts Without Documentation **Problem:** SKILL.md says `uv run ${CLAUDE_SKILL_ROOT}/scripts/tool.py` but doesn't document what arguments it takes or what it outputs. @@ -146,3 +197,15 @@ Returns JSON: **Problem:** SKILL.md references a hardcoded path like `plugins/my-plugin/skills/my-skill/scripts/tool.py`. **Fix:** Always use `${CLAUDE_SKILL_ROOT}/scripts/tool.py`. The variable resolves to the skill's directory regardless of where the agent runs from. + +### First/Second Person Descriptions + +**Problem:** Description says "I can help you process files" or "You can use this to process files." Inconsistent point-of-view causes discovery problems. + +**Fix:** Write in third person: "Processes files and generates reports. Use when working with data files." + +### Time-Sensitive Information + +**Problem:** SKILL.md includes "If before August 2025, use the old API" which will become wrong. + +**Fix:** Use a "Legacy patterns" section with the deprecated date noted, or remove time-sensitive content entirely. diff --git a/plugins/sentry-skills/skills/skill-creator/references/workflow-patterns.md b/plugins/sentry-skills/skills/skill-creator/references/workflow-patterns.md new file mode 100644 index 0000000..9b3614d --- /dev/null +++ b/plugins/sentry-skills/skills/skill-creator/references/workflow-patterns.md @@ -0,0 +1,102 @@ +# Workflow Patterns + +Patterns for structuring multi-step workflows and decision logic in skills. + +## Sequential Workflows + +Break complex tasks into numbered steps. Give an overview early in SKILL.md so the agent knows the full process before starting. + +```markdown +Filling a PDF form involves these steps: + +1. Analyze the form (run analyze_form.py) +2. Create field mapping (edit fields.json) +3. Validate mapping (run validate_fields.py) +4. Fill the form (run fill_form.py) +5. Verify output (run verify_output.py) +``` + +For particularly complex workflows, provide a checklist the agent can track: + +```markdown +Copy this checklist and track progress: + +- [ ] Step 1: Analyze the form +- [ ] Step 2: Create field mapping +- [ ] Step 3: Validate mapping +- [ ] Step 4: Fill the form +- [ ] Step 5: Verify output +``` + +## Conditional Workflows + +Guide agents through decision points with clear branching: + +```markdown +1. Determine the modification type: + + **Creating new content?** → Follow "Creation workflow" below + **Editing existing content?** → Follow "Editing workflow" below + +2. Creation workflow: + - Use docx-js library + - Build document from scratch + - Export to .docx format + +3. Editing workflow: + - Unpack existing document + - Modify XML directly + - Validate after each change + - Repack when complete +``` + +When branches get large, push them into separate reference files: + +```markdown +| Task Type | Read This Reference | +|-----------|-------------------| +| Creating documents | `${CLAUDE_SKILL_ROOT}/references/creation.md` | +| Editing documents | `${CLAUDE_SKILL_ROOT}/references/editing.md` | +``` + +## Feedback Loops + +Use a validate-fix-repeat pattern for tasks where output quality matters: + +```markdown +## Validation loop + +1. Make edits to the document +2. Validate immediately: `uv run ${CLAUDE_SKILL_ROOT}/scripts/validate.py` +3. If validation fails: + - Review the error message + - Fix the issues + - Run validation again +4. Only proceed when validation passes +``` + +This pattern works for: +- Code generation (lint → fix → re-lint) +- Document editing (validate XML → fix → re-validate) +- Data processing (check schema → fix → re-check) +- Form filling (validate fields → fix → re-validate) + +## Plan-Validate-Execute + +For complex, high-stakes tasks, have the agent create a plan file before executing: + +```markdown +1. Analyze the input and generate `changes.json` with planned modifications +2. Validate the plan: `uv run ${CLAUDE_SKILL_ROOT}/scripts/validate_plan.py changes.json` +3. If validation fails, revise the plan and re-validate +4. Execute the plan: `uv run ${CLAUDE_SKILL_ROOT}/scripts/apply_changes.py changes.json` +5. Verify the result +``` + +Benefits: +- Catches errors before changes are applied +- Machine-verifiable intermediate output +- Agent can iterate on the plan without touching originals +- Clear debugging — error messages point to specific plan entries + +Use this pattern for: batch operations, destructive changes, complex data transformations. diff --git a/plugins/sentry-skills/skills/skill-creator/scripts/quick_validate.py b/plugins/sentry-skills/skills/skill-creator/scripts/quick_validate.py new file mode 100644 index 0000000..5a3065f --- /dev/null +++ b/plugins/sentry-skills/skills/skill-creator/scripts/quick_validate.py @@ -0,0 +1,185 @@ +# /// script +# requires-python = ">=3.12" +# dependencies = ["pyyaml"] +# /// +""" +Quick validation script for Agent Skills. + +Validates SKILL.md frontmatter, naming conventions, and directory structure. + +Usage: + uv run quick_validate.py + +Returns exit code 0 on success, 1 on failure. Outputs JSON with validation results. +""" + +import json +import re +import sys +from pathlib import Path + +import yaml + +MAX_NAME_LENGTH = 64 +MAX_DESCRIPTION_LENGTH = 1024 +MAX_SKILL_LINES = 500 + + +def validate_skill(skill_path: Path) -> tuple[bool, list[str], list[str]]: + """Validate a skill directory. Returns (valid, errors, warnings).""" + errors: list[str] = [] + warnings: list[str] = [] + + # Check SKILL.md exists + skill_md = skill_path / "SKILL.md" + if not skill_md.exists(): + return False, ["SKILL.md not found"], [] + + content = skill_md.read_text() + lines = content.splitlines() + + # Check frontmatter exists and is first + if not content.startswith("---"): + errors.append("No YAML frontmatter found (file must start with ---)") + return False, errors, warnings + + match = re.match(r"^---\n(.*?)\n---", content, re.DOTALL) + if not match: + errors.append("Invalid frontmatter format (missing closing ---)") + return False, errors, warnings + + # Parse frontmatter + frontmatter_text = match.group(1) + try: + frontmatter = yaml.safe_load(frontmatter_text) + if not isinstance(frontmatter, dict): + errors.append("Frontmatter must be a YAML mapping") + return False, errors, warnings + except yaml.YAMLError as e: + errors.append(f"Invalid YAML in frontmatter: {e}") + return False, errors, warnings + + # Validate allowed fields + allowed_fields = { + "name", "description", "license", "compatibility", + "metadata", "allowed-tools", + # Claude Code extensions + "argument-hint", "disable-model-invocation", "user-invocable", + "model", "context", "agent", "hooks", + } + unexpected = set(frontmatter.keys()) - allowed_fields + if unexpected: + warnings.append( + f"Unexpected frontmatter field(s): {', '.join(sorted(unexpected))}. " + f"These may be ignored by some tools." + ) + + # Validate name + if "name" not in frontmatter: + errors.append("Missing required field: name") + else: + name = frontmatter["name"] + if not isinstance(name, str): + errors.append(f"name must be a string, got {type(name).__name__}") + else: + name = name.strip() + if not name: + errors.append("name must not be empty") + elif len(name) > MAX_NAME_LENGTH: + errors.append( + f"name is too long ({len(name)} chars, max {MAX_NAME_LENGTH})" + ) + elif not re.match(r"^[a-z0-9-]+$", name): + errors.append( + f"name '{name}' must contain only lowercase letters, digits, and hyphens" + ) + elif name.startswith("-") or name.endswith("-"): + errors.append(f"name '{name}' must not start or end with a hyphen") + elif "--" in name: + errors.append(f"name '{name}' must not contain consecutive hyphens") + elif name != skill_path.name: + errors.append( + f"name '{name}' does not match directory name '{skill_path.name}'" + ) + + # Validate description + if "description" not in frontmatter: + errors.append("Missing required field: description") + else: + desc = frontmatter["description"] + if not isinstance(desc, str): + errors.append(f"description must be a string, got {type(desc).__name__}") + else: + desc = desc.strip() + if not desc: + errors.append("description must not be empty") + elif len(desc) > MAX_DESCRIPTION_LENGTH: + errors.append( + f"description is too long ({len(desc)} chars, max {MAX_DESCRIPTION_LENGTH})" + ) + if "<" in desc or ">" in desc: + errors.append("description must not contain angle brackets (< or >)") + + # Quality checks + lower_desc = desc.lower() + if not any( + kw in lower_desc + for kw in ["use when", "use for", "use to", "trigger", "invoke"] + ): + warnings.append( + "description should include trigger phrases " + '(e.g., \'Use when asked to "review code"\')' + ) + if lower_desc.startswith(("i ", "i can", "you ")): + warnings.append( + "description should be in third person " + '("Processes files..." not "I can process files...")' + ) + + # Check line count + body_start = content.index("---", 3) + 3 + body_lines = content[body_start:].strip().splitlines() + if len(body_lines) > MAX_SKILL_LINES: + warnings.append( + f"SKILL.md body is {len(body_lines)} lines (recommended max {MAX_SKILL_LINES}). " + "Consider moving content to references/." + ) + + # Check for common issues + if "references/" in content or "scripts/" in content: + refs_dir = skill_path / "references" + scripts_dir = skill_path / "scripts" + if "references/" in content and not refs_dir.exists(): + errors.append("SKILL.md references 'references/' but directory does not exist") + if "scripts/" in content and not scripts_dir.exists(): + errors.append("SKILL.md references 'scripts/' but directory does not exist") + + # Check for hardcoded paths (should use ${CLAUDE_SKILL_ROOT}) + if re.search(r"(?:plugins|skills)/[a-z-]+/(?:scripts|references|assets)/", content): + warnings.append( + "SKILL.md may contain hardcoded paths. " + "Use ${CLAUDE_SKILL_ROOT}/scripts/... instead." + ) + + return len(errors) == 0, errors, warnings + + +def main() -> None: + if len(sys.argv) != 2: + print("Usage: uv run quick_validate.py ", file=sys.stderr) + sys.exit(1) + + skill_path = Path(sys.argv[1]).resolve() + if not skill_path.is_dir(): + print(json.dumps({"valid": False, "errors": [f"Not a directory: {skill_path}"]})) + sys.exit(1) + + valid, errors, warnings = validate_skill(skill_path) + + result = {"valid": valid, "errors": errors, "warnings": warnings} + print(json.dumps(result, indent=2)) + sys.exit(0 if valid else 1) + + +if __name__ == "__main__": + main()