[Proposal] support simplestories models

[mivanit]

Proposal

Support the SimpleStories family of models.

Motivation

The TinyStories models already included in TransformerLens are incredibly useful both as objects of study in their own right, as well as for debugging research code in low-resource environments before investigating larger models. The SimpleStories models are an improvement on the TinyStories models, built on a much more diverse dataset.

Links

models on HF: https://huggingface.co/SimpleStories
training repo: https://github.com/danbraunai/simple_stories_train
dataset paper: https://arxiv.org/pdf/2504.09184

Checklist

• I have checked that there is no similar issue in the repo (required)

[jlarson4]

@mivanit Hello! There are several SimpleStories fine-tunes available in the new TransformerBridge system, see the model table here.. Migration guide for moving from HookedTransformer to TransformerBridge is here.

The base SimpleStories does not appear to be in our supported_models list at the moment, but verification for the root model itself should be quite easy. I'll see if I can tackle that before the next release.