Pretraining and inference code for a large-scale depth-recurrent language model
-
Updated
Dec 29, 2025 - Python
#
recurrent-depth
Here are 3 public repositories matching this topic...
Research and training stack for AVA — a tool-using, memory-aware virtual assistant targeting 4 GB VRAM. Spans custom transformers, verifier-RL, external memory, multi-domain benchmarks, and Gemma 4 inference optimization.
ai reproducible-research benchmarks personal-assistant quantization external-memory gemma virtual-assistant distillation inference-optimization tool-use edge-inference int4 llm long-context small-language-models kv-cache-compression recurrent-depth 4gb-vram verifier-rl
-
Updated
Apr 7, 2026 - Python
Recurrent-depth transformer, fixed. Fork of kyegomez/OpenMythos with scatter-based MoE (2.94x faster), proper ACT halting, DeepSeekMoE load balancing, SDPA kernels, and a working training loop.
pytorch transformer moe claude mla adaptive-computation-time mixture-of-experts deepseek recurrent-depth
-
Updated
Apr 20, 2026 - Python
Improve this page
Add a description, image, and links to the recurrent-depth topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the recurrent-depth topic, visit your repo's landing page and select "manage topics."