lexical_lab

TODO:

• FSDP Experiments
• RoPE Embeddings
• MoE Architecture
• MLA Latent Attention
• Grouped Query Attention
• Pre-Training
• LLM Evaluation
• SFT Training.
• RLHF Data + Training
• RL based on rule-based reward