Must-Read-LLM-Papers

RL Optimization Algorithms

Model	Paper Title	arXiv Link
PPO	Proximal Policy Optimization Algorithms	1707.06347
DPO	Direct Preference Optimization: Your Language Model is Secretly a Reward Model	2305.18290
GRPO	Generalized Reinforce Policy Optimization	2501.12948
DAPO	Divergence-Augmented Preference Optimization	2503.14476
Dr.GRPO	Distributionally Robust Generalized Reinforce Policy Optimization	2503.20783
StarPO	Stabilized Reinforcement Alignment via Preference Optimization	2504.20073
ProRL	Proximal Reinforcement Learning with Preference Feedback	2505.24864
GSPO	Generalized Soft Preference Optimization	2507.18071

Deep Research

Model	Paper Title	arXiv Link
R1-Searcher	R1-Searcher: Training Reasoners by Searching with Language Models	2503.05592
Search-R1	Search-R1: Scaling Reinforcement Learning with Tree Search for Reasoning	2503.09516
ReSearch	ReSearch: Reinforcement Learning with Search for Reasoning	2503.19470
StepSearch	StepSearch: Stepwise Search for Long-Horizon Reasoning	2505.15107
R1-Searcher++	R1-Searcher++: Scaling Search-Based Reinforcement Learning with Language Models	2505.17005
ZeroSearch	ZeroSearch: Training Zero-Shot Reasoners via Search	2505.04588
WebDancer	WebDancer: Reinforcement Learning with Web-Scale Exploration	2505.22648
WebSailor	WebSailor: Autonomous Web Navigation with Reinforcement Learning	2507.02592
ASearcher	ASearcher: Adaptive Search for Reasoning with Language Models	2508.07976
SSRL	SSRL: Self-Supervised Reinforcement Learning via Search	2508.10874

Foundation Models

Model	Paper Title	Link
GPT-3	Language Models are Few-Shot Learners	2005.14165
LLaMA	LLaMA: Open and Efficient Foundation LLMs	2302.13971
GPT-4	GPT-4 Technical Report	2303.08774
LLaMA 2	LLaMA 2: Open Foundation and Fine-Tuned Chat Models	2307.09288
Qwen	Qwen Technical Report	2309.16609
Baichuan 2	Baichuan 2: Open Large-Scale Language Models	Tech Report PDF
Pythia	Pythia: A Suite for Analyzing LLMs Across Training and Scaling	GitHub
Qwen2	Qwen2 Technical Report	2407.10671
Qwen2.5	Qwen2.5 Technical Report	2412.15115
DeepSeek-V3	DeepSeek-V3 Technical Report	2412.19437
DeepSeek-R1	DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL	2501.12948
Qwen2.5-VL	Qwen2.5-VL Technical Report	2502.13923
Kimi K2	Kimi K2: Open Agentic Intelligence	2507.20534
Gemma 3	Gemma 3 Technical Report	2503.19786
Qwen3	Qwen3 Technical Report	2505.09388
LongCat-Flash	LongCat-Flash Technical Report	2509.01322

LLM Interpretability

Paper Title	Author(s)	Link
INTERPRETABILITY IN THE WILD: A Circuit for Indirect Object Identification in GPT-2 Small	Anthropic	Distill
Language Models Implement Simple Word2Vec-style Vector Arithmetic	Ellie Pavlick	Notion
Break It Down: Structural Compositionality in Neural Networks	Ellie Pavlick	Notion
Does Circuit Analysis Interpretability Scale?	Neel Nanda (DeepMind)	Notion
Towards Automated Circuit Discovery	Neel Nanda	Notion
TransformerLens (Toolkit)	Neel Nanda	GitHub
Explaining Grokking via Circuit Efficiency	DeepMind	2309.02390

LLM Scaling Law & Training Dynamics

Paper Title	Author(s)	Link
Scaling Laws for Neural Language Models	OpenAI	2001.08361
Chinchilla: Training Compute-Optimal LLMs	DeepMind	2203.15556
Deep Double Descent	OpenAI	OpenAI Blog
Emergent Abilities of LLMs	Jason Wei	2206.07682
Grokking: Generalization Beyond Overfitting	OpenAI	2201.02177
Tensor Programs (series)	Greg Yang	Tensor Programs

LLM Preference Alignment

Paper Title	Author(s)	Link
Pretraining Language Models with Human Preferences	Sam Bowman	Notion
Alignment of Language Agents	DeepMind	Notion
Evaluating the Ripple Effects of Knowledge Editing	Mor Geva	Notion
Mass-Editing Memory in a Transformer	David Bau	Notion
RAIN: Self-Alignment without Fine-Tuning	—	Notion

Understanding LLMs

Paper Title	Author(s)	Link
Are Emergent Abilities in LLMs just In-Context Learning?	Google	2206.07682
Language Models (Mostly) Know What They Know	Anthropic	2304.05336
Sparks of Artificial General Intelligence: Early Experiments with GPT-4	Microsoft	2303.12712
Unveiling Theory of Mind in LLMs: A Parallel to Neurons in the Human Brain	—	Notion
Delays, Detours, and Forks in the Road: Latent State Models of Training Dynamics	Kyunghyun Cho	Notion
Generative Models as Complex Systems Science	Luke Zettlemoyer	Notion
What Learning Algorithm is In-Context Learning?	Tengyu Ma	2301.00234
Why Can GPT Learn In-Context? Gradient Descent as Meta-Optimization	Li Dong	Notion
Measuring Inductive Biases of In-Context Learning	He He	Notion
Faith and Fate: Limits of Transformers on Compositionality	Yejin Choi	Notion
Intrinsic Dimensionality Explains Effectiveness of LM Fine-Tuning	Luke Zettlemoyer	Notion

Reward Models & Benchmarks

Method / Benchmark	Full Title	Link
Checklists	Checklists Are Better Than Reward Models For Aligning Language Models	2507.18624
RewardAnything	RewardAnything: Generalizable Principle-Following Reward Models	2506.03637
RubricRM	Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains	2507.17746
RM-R1	RM-R1: Reward Modeling as Reasoning	2505.02387
GenRM	Generative Reward Models	2410.12832
RM-Bench	RM-Bench: Benchmarking Reward Models for Language Model Alignment	2406.11695
RewardBench	RewardBench: A Benchmark for Evaluating Reward Models in LLM Alignment	RewardBench
Skywork-Reward-V2	Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy	2507.01352
Deepseek-GRM	Inference-Time Scaling for Generalist Reward Modeling	2504.02495
Agent-as-a-Judge	Agent-as-a-Judge: Evaluate Agents with Agents	2410.10934

Agent Benchmark

Benchmark	Full Title	Link
Tau-Bench	Tau-Bench: A Benchmark for Tool-Augmented Language Models	Tau-Bench
Tau2-Bench	Tau2-Bench: Benchmarking General-Purpose Language Agents	Tau2-Bench
MCP-Bench	MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers	2508.20453