Skip to content

Commit dadff60

Browse files
Move genesis-evals out of .apm/ to avoid scanner leak
APM's local-content scanner treats .apm/skills/ as a publishable source root regardless of devDep marker. A maintainer-only skill colocated under .apm/ would leak into apm pack --format plugin even when declared as a devDependency. Cure: live under dev/skills/ instead. - git mv .apm/skills/genesis-evals dev/skills/genesis-evals - update path refs in SKILL.md, scripts, composition-substrate asset - declare local-path devDependency in apm.yml - commit apm.lock.yaml (single local entry, is_dev: true) This dogfoods the BUNDLE LEAKAGE / DISPATCH CONTAMINATION discipline that genesis itself teaches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent efb3621 commit dadff60

19 files changed

Lines changed: 51 additions & 21 deletions

apm.lock.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
lockfile_version: '1'
2+
generated_at: '2026-04-26T11:12:53.114123+00:00'
3+
apm_version: 0.10.0
4+
dependencies:
5+
- repo_url: _local/genesis-evals
6+
package_type: claude_skill
7+
deployed_files:
8+
- .github/skills/genesis-evals
9+
source: local
10+
local_path: ./dev/skills/genesis-evals
11+
is_dev: true

apm.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,9 @@ description: A design discipline for agentic primitives. Substrate, patterns, co
44
author: Daniel Meppiel <danielmeppiel@users.noreply.github.com>
55
license: Apache-2.0
66
homepage: https://github.com/danielmeppiel/genesis
7+
8+
# Maintainer-only dev infra. Lives outside .apm/ so APM's local-content
9+
# scanner does not auto-publish it. Install with: apm install --dev
10+
devDependencies:
11+
apm:
12+
- path: ./dev/skills/genesis-evals
Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,10 @@ description: >-
99
or "regenerate the eval matrix". This skill orchestrates parallel cold
1010
sub-agent spawns via the harness's task tool, scores deterministically,
1111
and converges P>=0.8 / N>=0.8 / R==1.0 within max 3 iteration loops.
12-
This skill is contributor-only -- it lives under .apm/skills/ and is
13-
NOT shipped inside the user-facing skills/genesis/ bundle (BUNDLE
14-
LEAKAGE discipline).
12+
This skill is contributor-only -- it lives under dev/skills/ (OUTSIDE
13+
.apm/) and is NOT shipped inside the user-facing skills/genesis/
14+
bundle (BUNDLE LEAKAGE discipline). See "Why this lives outside
15+
.apm/" below.
1516
---
1617

1718
# genesis-evals: maintainer-side eval runner
@@ -20,14 +21,22 @@ Run the genesis self-eval suite. Steers the parent LLM session to
2021
orchestrate cold sub-agent spawns, capture responses, score
2122
deterministically, and report convergence.
2223

23-
## Why this lives under .apm/
24+
## Why this lives outside `.apm/`
2425

2526
Genesis ships to USERS via npx / `apm install`. Eval scenarios LOOK
2627
LIKE real user requests (that is the point). Colocating them under
2728
`skills/genesis/evals/` would risk DISPATCH CONTAMINATION (an
2829
over-eager harness loader pulling scenario prompts into the active
2930
context) and PAYLOAD BLOAT for users who never run evals.
3031

32+
We also keep this OUTSIDE `.apm/` because APM treats `.apm/` as the
33+
publishable source root: its local-content scanner picks up anything
34+
under `.apm/skills/` regardless of dev-marker, so `apm pack
35+
--format plugin` would leak this maintainer-only skill into the
36+
shipped artifact. Living under `dev/skills/` keeps it scanner-invisible
37+
while still letting `apm install --dev` deploy it via the local-path
38+
devDependency in the root `apm.yml`.
39+
3140
This is the inverse of PHANTOM DEPENDENCY (referenced-but-not-bundled):
3241
BUNDLE LEAKAGE (bundled-but-not-consumed-at-runtime). See
3342
`skills/genesis/assets/composition-substrate.md` "Anti-patterns
@@ -88,9 +97,9 @@ flagged at this step".
8897
## Step 1 -- validate
8998

9099
```
91-
python .apm/skills/genesis-evals/scripts/validate_scenarios.py \
92-
--scenarios-dir .apm/skills/genesis-evals/scenarios \
93-
--schema .apm/skills/genesis-evals/schema/scenario.schema.json
100+
python dev/skills/genesis-evals/scripts/validate_scenarios.py \
101+
--scenarios-dir dev/skills/genesis-evals/scenarios \
102+
--schema dev/skills/genesis-evals/schema/scenario.schema.json
94103
```
95104

96105
If exit non-zero, STOP. Fix the schema/yaml violations before any
@@ -100,7 +109,7 @@ spawn. No partial runs.
100109

101110
```
102111
RUN_ID=$(date -u +%Y%m%dT%H%M%SZ)
103-
RUNS_DIR=.apm/skills/genesis-evals/runs
112+
RUNS_DIR=dev/skills/genesis-evals/runs
104113
mkdir -p "$RUNS_DIR/$RUN_ID"
105114
```
106115

@@ -118,8 +127,8 @@ For EACH scenario YAML in `scenarios/`:
118127

119128
1. **Record the spawn intent** (deterministic, pre-spawn):
120129
```
121-
python .apm/skills/genesis-evals/scripts/spawn_record.py \
122-
--scenario .apm/skills/genesis-evals/scenarios/<id>.yml \
130+
python dev/skills/genesis-evals/scripts/spawn_record.py \
131+
--scenario dev/skills/genesis-evals/scenarios/<id>.yml \
123132
--half <with|without|single> \
124133
--run-id "$RUN_ID" \
125134
--runs-dir "$RUNS_DIR"
@@ -156,10 +165,10 @@ parallel as the harness permits. Do NOT serialize unless forced.
156165
## Step 4 -- score
157166

158167
```
159-
python .apm/skills/genesis-evals/scripts/score_run.py \
168+
python dev/skills/genesis-evals/scripts/score_run.py \
160169
--run-id "$RUN_ID" \
161170
--runs-dir "$RUNS_DIR" \
162-
--scenarios-dir .apm/skills/genesis-evals/scenarios
171+
--scenarios-dir dev/skills/genesis-evals/scenarios
163172
```
164173

165174
Writes `<runs-dir>/<run-id>/summary.md`. Exit code: 0 if converged,
@@ -242,7 +251,7 @@ the file (provenance / audit trail).
242251
## Setup (one-time)
243252

244253
```
245-
pip install -r .apm/skills/genesis-evals/requirements.txt
254+
pip install -r dev/skills/genesis-evals/requirements.txt
246255
```
247256

248257
(deps: pyyaml, jsonschema. Maintainer-side only. Users never see
File renamed without changes.

.apm/skills/genesis-evals/scenarios/n-001-debug-bug.yml renamed to dev/skills/genesis-evals/scenarios/n-001-debug-bug.yml

File renamed without changes.

.apm/skills/genesis-evals/scenarios/n-002-write-doc.yml renamed to dev/skills/genesis-evals/scenarios/n-002-write-doc.yml

File renamed without changes.

.apm/skills/genesis-evals/scenarios/n-003-explain-concept.yml renamed to dev/skills/genesis-evals/scenarios/n-003-explain-concept.yml

File renamed without changes.

.apm/skills/genesis-evals/scenarios/p-001-a10-recognition.yml renamed to dev/skills/genesis-evals/scenarios/p-001-a10-recognition.yml

File renamed without changes.

0 commit comments

Comments
 (0)