Skip to content

feat: serve skill reference files via discovery RFC, expand icp-cli evals#81

Merged
raymondk merged 2 commits intomainfrom
feat/skill-discovery-evals-improvements
Mar 9, 2026
Merged

feat: serve skill reference files via discovery RFC, expand icp-cli evals#81
raymondk merged 2 commits intomainfrom
feat/skill-discovery-evals-improvements

Conversation

@marc0olo
Copy link
Copy Markdown
Member

@marc0olo marc0olo commented Mar 9, 2026

Summary

  • Serve skill reference files (references/*.md) at /.well-known/skills/{name}/{path} via a new Astro catch-all route, completing the Agent Skills Discovery RFC implementation
  • Include files array in /.well-known/skills/index.json for each skill, enabling agents to discover and fetch all bundled files
  • Expand icp-cli evaluations from 3 to 12 output evals, covering all 12 documented pitfalls (including adversarial prompts for env file patterns and dfx commands)
  • Fix eval harness shell expansion bug — $ICP_WASM_OUTPUT_PATH and $(mops ...) in skill content were being expanded by the shell. Replaced execSync with execFileSync + input option
  • Add --list flag and numeric --eval <N> index to eval harness for easier single-eval runs
  • Correct icp-cli pitfall #3: unpinned recipes are rejected by icp-cli (not just risky)
  • Document reference file endpoint on Access page and in README programmatic access table
  • Add pitfall coverage guidance to CONTRIBUTING.md evaluation section

…vals

- Serve reference files (references/*.md) at /.well-known/skills/{name}/{path}
  via a new Astro catch-all route, completing the Skills Discovery RFC chain
- Include files array in /.well-known/skills/index.json for each skill
- Expand icp-cli evaluations from 3 to 12 output evals covering all pitfalls
- Fix eval harness shell expansion bug (execFileSync instead of execSync)
- Add --list flag and numeric --eval index to eval harness
- Correct icp-cli pitfall #3: unpinned recipes are rejected, not just risky
- Document reference file endpoint on Access page and in README
@raymondk raymondk merged commit 06ec51d into main Mar 9, 2026
6 checks passed
@raymondk raymondk deleted the feat/skill-discovery-evals-improvements branch March 9, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants