Summary
The current examples.json fixture workflow has several pain points that should be addressed in a future refactor.
Current problems
-
Graph serialization is unreadable — instances use petgraph's internal serde format ({"inner": {"edge_property": "undirected", "edges": [[0,1,null],...], "node_holes": [], "nodes": [null,...]}}) which is hard to write, review, and edit by hand.
-
Example data is hardcoded in Rust builders — canonical examples are constructed in src/example_db/model_builders.rs and rule_builders.rs as Rust code. The JSON file is a generated cache, not a source of truth. This makes the verification tests circular: code → solver → JSON → test checks JSON matches code.
-
Unit tests don't load input from the JSON fixture — per-model tests in src/unit_tests/models/ hardcode their own instances rather than validating against the canonical examples in examples.json.
Proposed changes
-
Use a simple graph JSON format — define a human-friendly JSON schema for graph instances (similar to the CLI's --graph 0-1,1-2,2-3 input syntax) with a deserialization layer that converts to the internal representation. This makes examples.json reviewable and hand-editable.
-
Encode issue data into JSON — when adding a new model via the add-model workflow, write the instance data from the GitHub issue (mathematical definition, example instances) directly into examples.json as the ground truth, rather than hardcoding in Rust builders.
-
Example unit tests load input from JSON rather than hardcode — per-model unit tests should load canonical instances from examples.json and verify the model implementation against them. This breaks the circularity: the JSON becomes an independently authored specification (grounded in the issue's math), and tests verify the code satisfies it.
Benefits
examples.json becomes the single source of truth, grounded in mathematical definitions from issues
- Tests become meaningful (verifying correctness against a spec) rather than circular (verifying code matches its own output)
- The JSON file becomes human-reviewable in PRs
- The
add-model workflow becomes: write JSON spec → implement model → tests verify against spec (TDD-like)
- Reduces merge conflict pain (human-friendly format is easier to resolve)
Migration notes
- The CLI already parses human-friendly graph input — reuse that parsing layer
- Existing builders can be kept temporarily for backwards compatibility during migration
- The
regenerate_fixtures binary would eventually be removed or repurposed
Summary
The current
examples.jsonfixture workflow has several pain points that should be addressed in a future refactor.Current problems
Graph serialization is unreadable — instances use petgraph's internal serde format (
{"inner": {"edge_property": "undirected", "edges": [[0,1,null],...], "node_holes": [], "nodes": [null,...]}}) which is hard to write, review, and edit by hand.Example data is hardcoded in Rust builders — canonical examples are constructed in
src/example_db/model_builders.rsandrule_builders.rsas Rust code. The JSON file is a generated cache, not a source of truth. This makes the verification tests circular: code → solver → JSON → test checks JSON matches code.Unit tests don't load input from the JSON fixture — per-model tests in
src/unit_tests/models/hardcode their own instances rather than validating against the canonical examples inexamples.json.Proposed changes
Use a simple graph JSON format — define a human-friendly JSON schema for graph instances (similar to the CLI's
--graph 0-1,1-2,2-3input syntax) with a deserialization layer that converts to the internal representation. This makesexamples.jsonreviewable and hand-editable.Encode issue data into JSON — when adding a new model via the
add-modelworkflow, write the instance data from the GitHub issue (mathematical definition, example instances) directly intoexamples.jsonas the ground truth, rather than hardcoding in Rust builders.Example unit tests load input from JSON rather than hardcode — per-model unit tests should load canonical instances from
examples.jsonand verify the model implementation against them. This breaks the circularity: the JSON becomes an independently authored specification (grounded in the issue's math), and tests verify the code satisfies it.Benefits
examples.jsonbecomes the single source of truth, grounded in mathematical definitions from issuesadd-modelworkflow becomes: write JSON spec → implement model → tests verify against spec (TDD-like)Migration notes
regenerate_fixturesbinary would eventually be removed or repurposed