Skip to content

Refactor example_db: human-friendly JSON format and test-from-fixture workflow #680

@GiggleLiu

Description

@GiggleLiu

Summary

The current examples.json fixture workflow has several pain points that should be addressed in a future refactor.

Current problems

  1. Graph serialization is unreadable — instances use petgraph's internal serde format ({"inner": {"edge_property": "undirected", "edges": [[0,1,null],...], "node_holes": [], "nodes": [null,...]}}) which is hard to write, review, and edit by hand.

  2. Example data is hardcoded in Rust builders — canonical examples are constructed in src/example_db/model_builders.rs and rule_builders.rs as Rust code. The JSON file is a generated cache, not a source of truth. This makes the verification tests circular: code → solver → JSON → test checks JSON matches code.

  3. Unit tests don't load input from the JSON fixture — per-model tests in src/unit_tests/models/ hardcode their own instances rather than validating against the canonical examples in examples.json.

Proposed changes

  1. Use a simple graph JSON format — define a human-friendly JSON schema for graph instances (similar to the CLI's --graph 0-1,1-2,2-3 input syntax) with a deserialization layer that converts to the internal representation. This makes examples.json reviewable and hand-editable.

  2. Encode issue data into JSON — when adding a new model via the add-model workflow, write the instance data from the GitHub issue (mathematical definition, example instances) directly into examples.json as the ground truth, rather than hardcoding in Rust builders.

  3. Example unit tests load input from JSON rather than hardcode — per-model unit tests should load canonical instances from examples.json and verify the model implementation against them. This breaks the circularity: the JSON becomes an independently authored specification (grounded in the issue's math), and tests verify the code satisfies it.

Benefits

  • examples.json becomes the single source of truth, grounded in mathematical definitions from issues
  • Tests become meaningful (verifying correctness against a spec) rather than circular (verifying code matches its own output)
  • The JSON file becomes human-reviewable in PRs
  • The add-model workflow becomes: write JSON spec → implement model → tests verify against spec (TDD-like)
  • Reduces merge conflict pain (human-friendly format is easier to resolve)

Migration notes

  • The CLI already parses human-friendly graph input — reuse that parsing layer
  • Existing builders can be kept temporarily for backwards compatibility during migration
  • The regenerate_fixtures binary would eventually be removed or repurposed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions