Morpheon Subnet

Bittensor subnet for world model quality evaluation.

What We Measure

World model quality as closed-loop simulation environments for training generalized robotics policies.

Pattern

Local Model Evaluation (Validator-Hosted)

Miners submit world models as safetensors files via S3-compatible storage (Cloudflare R2). Validators download models, load locally, and evaluate against standardized scenarios measuring prediction quality.

Scoring Criteria

Dimension	Weight	Description
Visual Fidelity	20%	Match to reference frames
Temporal Consistency	15%	Smoothness, no flickering
Physical Plausibility	15%	Physics sanity tests
Action Responsiveness	10%	Counterfactual divergence
Inference Speed	15%	FPS performance
Policy Transfer	20%	Train in WM, test in sim
Diversity Coverage	5%	Scenario variety

Installation

# Install uv (if not already installed)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install dependencies
uv sync --extra dev

Usage

# Show help
uv run morpheon --help

# Run validator
uv run morpheon --network finney --netuid 1 run --coldkey default --hotkey default

# Run with dry-run mode (no weights set on chain)
uv run morpheon --network local --netuid 1 run --dry-run

# Show subnet info
uv run morpheon --network finney --netuid 1 info

# List registered models
uv run morpheon --network finney --netuid 1 models

# Show validator status
uv run morpheon status

# Display configuration
uv run morpheon config
uv run morpheon config --format json

CLI Commands

Command	Description
run	Run the validator main loop
info	Show subnet and validator information
models	List registered models from chain
status	Show current validator status
config	Display configuration

Model Submission Format

Miners submit models in the following format:

model_submission/
├── model.safetensors      # Model weights (safe, no code execution)
├── config.json            # Architecture configuration
└── metadata.json          # Submission metadata

Configuration

Environment variables:

• NETWORK - Network to connect to (finney, test, local)
• NETUID - Subnet netuid
• WALLET_NAME - Wallet name
• HOTKEY_NAME - Hotkey name
• LOG_LEVEL - Logging level (DEBUG, INFO, WARNING, ERROR)
• S3_ENDPOINT_URL - S3 endpoint URL (e.g., https://<account_id>.r2.cloudflarestorage.com)
• S3_ACCESS_KEY_ID - S3 access key ID
• S3_SECRET_ACCESS_KEY - S3 secret access key
• S3_BUCKET_NAME - S3 bucket name for storing models
• S3_REGION - S3 region (default: auto for Cloudflare R2)

Development

# Run tests
uv run pytest

# Run linter
uv run ruff check validator.py

# Format code
uv run ruff format validator.py

Implementation Status

Feature	Status	Notes
Scenario Ground Truth Loading	Done	Loads reference frames from numpy files
Visual Fidelity (FVD)	Done	Fréchet Video Distance with MAE fallback
Collision Detection	Done	Velocity/frame change analysis
Object Permanence	Done	Divergence detection
Policy Transfer Scoring	Done	Composite proxy (see below)
ViT Model Builder	Done	Architecture-specific loader
S3 Timestamp Verification	Disabled	Anti-gaming (uncomment to enable)

Policy Transfer Scoring

Uses composite proxy metric (full simulator integration optional):

Component	Weight	Measures
Temporal Coherence	30%	Smooth predictions over time
Action Responsiveness	25%	Different actions → different outcomes
Prediction Stability	25%	Long-horizon predictions don't diverge
Uncertainty Calibration	20%	Well-calibrated confidence estimates

TODO (Optional Enhancements)

• Full policy transfer with Isaac Sim/MuJoCo integration
• Additional architecture builders (transformer, diffusion, autoencoder)
• Enable S3 timestamp verification for anti-gaming
• Generate scenario datasets with real ground truth frames

License

MIT