opencode-skill-creator

Create, test, and optimize OpenCode skills — from first draft to production-grade.

A skill + plugin for OpenCode that brings eval-driven development to AI agent skills — based on Anthropic's official skill-creator for Claude Code, ported to TypeScript and adapted for OpenCode's plugin architecture.

Install · What it does · Plugin tools · Usage · Architecture

---

Why this exists

Creating AI agent skills is guesswork. You write a skill, test it manually, maybe tweak the description, and hope it triggers correctly. There's no systematic way to measure whether a skill works or to track improvements across iterations.

opencode-skill-creator fixes this with eval-driven development for skills:

• Test — Auto-generate eval test sets and measure trigger accuracy
• Optimize — Iteratively improve skill descriptions with a train/test split
• Benchmark — Quantitative comparison across iterations with variance analysis
• Review — Built-in visual eval viewer for human-in-the-loop feedback
• Install — Deploy validated skills to project or global config

Based on Anthropic's proven methodology. Free for everyone. Works with any model OpenCode supports.

Install

Package: https://www.npmjs.com/package/opencode-skill-creator

Pick your option

If you are...	Use this
New to OpenCode / non-developer	Option A (Recommended)
Already using other plugins	Option B
Setting up all projects on your computer	Option C (Global config)
Setting up only one project	Option D (Project config)
Cannot use npm / offline environment	Option E (Manual install)

Option A (Recommended): easiest setup for most users

Run one command (global install, recommended):

npx opencode-skill-creator install --global

Optional checks:

npx opencode-skill-creator --version
npx opencode-skill-creator --help
npx opencode-skill-creator --about

What this command does:

1. Creates/updates ~/.config/opencode/opencode.json
2. Adds "opencode-skill-creator" to the plugin array
3. Leaves your existing plugins untouched

Then:

4. Restart OpenCode
5. Ask OpenCode: Create a skill that helps with Docker compose files

That's it.

Manual equivalent for the same result:

1. Open (or create) ~/.config/opencode/opencode.json
2. Paste this:

{
  "plugin": ["opencode-skill-creator"]
}

3. Restart OpenCode.

If you want project-only install instead, use:

npx opencode-skill-creator install --project

Option B: you already have plugins

If your file already has plugins, append this package to the list:

{
  "plugin": [
    "your-existing-plugin",
    "opencode-skill-creator"
  ]
}

Do not remove your existing plugins.

Option C: global config (works in all projects)

Use global config when you want this plugin available everywhere.

Command version:

npx opencode-skill-creator install --global

1. Open (or create) ~/.config/opencode/opencode.json
2. Add:

{
  "plugin": ["opencode-skill-creator"]
}

3. Restart OpenCode.

Option D: project config (only one project)

Use project config when you want this plugin only for one repo.

Command version:

npx opencode-skill-creator install --project

1. Open (or create) opencode.json in that project root
2. Add:

{
  "plugin": ["opencode-skill-creator"]
}

3. Restart OpenCode in that project.

Option E: manual install (no npm)

git clone https://github.com/antongulin/opencode-skill-creator.git
cd opencode-skill-creator

# Install the skill (global)
cp -r skill-creator/ ~/.config/opencode/skills/skill-creator/

# Install the plugin (global)
cp -r plugin/ ~/.config/opencode/plugins/skill-creator/

Then create ~/.config/opencode/package.json if needed:

{
  "dependencies": {
    "@opencode-ai/plugin": ">=1.0.0"
  }
}

What happens after install

After you add opencode-skill-creator and restart OpenCode:

1. OpenCode installs the plugin from npm automatically.
2. On first plugin startup, it auto-copies skill files to:
   • ~/.config/opencode/skills/skill-creator/
3. The skill becomes available automatically in your sessions.

Verify install

Check that the skill file exists:

ls ~/.config/opencode/skills/skill-creator/SKILL.md

Then ask OpenCode:

Create a skill that helps with API documentation.

You should see it use the skill-creator workflow/tools.

Troubleshooting

• I don't have opencode.json: create one in project root (or use global config path).
• Nothing changed after edit: fully restart OpenCode.
• I already had plugins: keep them; just add opencode-skill-creator to the same array.
• I want a clean reinstall: delete ~/.config/opencode/skills/skill-creator/ and restart OpenCode.
• npx command failed: run npx opencode-skill-creator --help and then use install or install --global.

For LLMs / automation (compact)

{ "plugin": ["opencode-skill-creator"] }

What it does

When loaded, this skill guides OpenCode through the full skill development lifecycle:

1. Analyze the user's request and determine what kind of skill to build
2. Create a well-structured skill with proper frontmatter, SKILL.md, and supporting files
3. Generate an eval set of test queries (should-trigger and should-not-trigger)
4. Evaluate the skill's description by testing whether it triggers correctly
5. Optimize the description through iterative improvement loops
6. Benchmark skill performance with variance analysis
7. Install the skill to the project or global OpenCode skills directory

Plugin tools

The plugin registers these custom tools that OpenCode can call:

Tool	Purpose
skill_validate	Validate SKILL.md structure and frontmatter
skill_parse	Parse SKILL.md and extract name/description
skill_eval	Test trigger accuracy for eval queries
skill_improve_description	LLM-powered description improvement
skill_optimize_loop	Full eval→improve optimization loop
skill_aggregate_benchmark	Aggregate grading results into statistics
skill_generate_report	Generate HTML optimization report
skill_serve_review	Start the eval review viewer (HTTP server)
skill_stop_review	Stop a running review server
skill_export_static_review	Generate standalone HTML review file

Description optimization loop

The most impactful feature for skill quality. It treats skill descriptions as a search problem:

1. Generates 20 test queries (should-trigger + should-not-trigger)
2. Splits into 60% train / 40% test
3. Runs each query 3 times for statistical reliability
4. Analyzes failure patterns
5. LLM proposes improved descriptions
6. Re-evaluates on both train AND test sets
7. Selects the best description by test score (prevents overfitting)
8. Repeats up to 5 iterations

Review workflow guard (strict by default)

The review launch tools enforce paired comparison data by default:

• skill_serve_review and skill_export_static_review require each eval-* directory to include:
  • with_skill
  • baseline (without_skill or old_skill)
• If pairs are missing, the tools fail fast with a clear list of missing items.
• Override only when intentionally reviewing partial data by passing allowPartial: true.
• If benchmarkPath is omitted, the tools auto-generate benchmark.json and benchmark.md in the workspace.

Skill draft staging (recommended)

When creating new skills, use a staging path in the system temp directory outside your current repository:

• Unix/macOS draft skill path: /tmp/opencode-skills/<skill-name>/ (or $TMPDIR/opencode-skills/<skill-name>/)
• Unix/macOS eval workspace path: /tmp/opencode-skills/<skill-name>-workspace/
• Windows draft/eval paths: %TEMP%\opencode-skills\<skill-name>\ and %TEMP%\opencode-skills\<skill-name>-workspace\
• Install only the final validated skill to:
  • project: .opencode/skills/<skill-name>/
  • global: ~/.config/opencode/skills/<skill-name>/

This keeps plugin/source repositories clean while preserving the full eval loop.

Usage

Once installed, OpenCode will automatically detect the skill when you ask it to create or improve a skill. For example:

• "Create a skill that helps with Docker compose files"
• "Build me a skill for generating API documentation"
• "Help me make a skill that assists with database migrations"
• "Optimize the description of my existing skill"

OpenCode will load the skill-creator instructions and use the plugin tools to walk through the full workflow.

Architecture

This project has two components:

Component	What it is
Skill	Markdown instructions (SKILL.md + agents + templates) that tell the agent how to create, evaluate, and improve skills
Plugin	TypeScript module that registers custom tools for validation, eval, benchmarking, and review

The skill provides the workflow knowledge; the plugin provides the executable tools the agent calls during that workflow.

On first startup, the plugin automatically copies the bundled skill files to ~/.config/opencode/skills/skill-creator/. If you need to reinstall the skill (e.g., after an update), delete that directory and restart OpenCode.

Project structure

opencode-skill-creator/
├── README.md
├── LICENSE                            # Apache 2.0
├── skill-creator/                     # The SKILL
│   ├── SKILL.md                       # Main skill instructions
│   ├── agents/
│   │   ├── grader.md                  # Assertion evaluation
│   │   ├── analyzer.md                # Benchmark analysis
│   │   └── comparator.md              # Blind A/B comparison
│   ├── references/
│   │   └── schemas.md                 # JSON schema definitions
│   └── templates/
│       └── eval-review.html           # Eval set review/edit UI
└── plugin/                            # The PLUGIN (npm: opencode-skill-creator)
    ├── package.json                   # npm package metadata
    ├── skill-creator.ts               # Entry point — registers all tools
    ├── skill/                         # Bundled copy of skill (auto-installed)
    ├── lib/
    │   ├── utils.ts                   # SKILL.md frontmatter parsing
    │   ├── validate.ts                # Skill structure validation
    │   ├── run-eval.ts                # Trigger evaluation via opencode run
    │   ├── improve-description.ts     # LLM-powered description improvement
    │   ├── run-loop.ts                # Eval→improve optimization loop
    │   ├── aggregate.ts               # Benchmark aggregation
    │   ├── report.ts                  # HTML report generation
    │   └── review-server.ts           # Eval review HTTP server
    └── templates/
        └── viewer.html                # Eval review viewer UI

Differences from the Anthropic original

Area	Anthropic (Claude Code)	This repo (OpenCode)
CLI invocation	claude -p "prompt"	opencode run "prompt"
Skill location	.claude/commands/	.opencode/skills/
Automation scripts	Python (scripts/*.py)	TypeScript plugin (plugin/lib/*.ts)
Script execution	python -m scripts.run_loop	skill_optimize_loop tool call
Eval viewer	python generate_review.py	skill_serve_review tool call
Benchmarking	python aggregate_benchmark.py	skill_aggregate_benchmark tool call
Dependencies	Python 3.11+, pyyaml	Bun (via OpenCode), @opencode-ai/plugin
Packaging	.skill zip files	npm package + skill directory
Subagents	Built-in subagent concept	Task tool with general/explore types

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

License

Apache License 2.0 — see LICENSE for details.

Based on anthropics/skills by Anthropic.