lmarena-draws

Code for the paper Drawing Conclusions from Draws: Rethinking Draw Semantics in Arena-Style LLM Evaluation.

Getting Started

Setup

1. Install the uv package manager.

2. Clone the repository: git clone ssh://git@github.com/daemon/lmarena-draws && cd lmarena-draws

3. Create and activate a virtual environment: uv venv --python 3.13 && source .venv/bin/activate

4. Install the package: uv pip install -e .

5. You're ready to go!

Reproducing Main Results

Run everything in the paper notebook.

Citation

@article{tang2025drawing,
  title={Drawing Conclusions from Draws: Rethinking Preference Semantics in Arena-Style {LLM} Evaluation}, 
  author={Raphael Tang and Crystina Zhang and Wenyan Li and Carmen Lai and Pontus Stenetorp and Yao Lu},
  journal={arXiv:2510.02306},
  year={2025}
}