LMSYS

An open-source reproduction of frontier-class MoE serving

Brought a cache layer for workloads with overlapping prompts, such as multi-turn and agents, down to the runtime level

Became the basis for automated evaluation and synthetic rewards, flowing directly into post-training pipelines

Shifted the focus from benchmarks with correct answers to evaluation based on human preference

Established at once the practice of training on synthetic and leaked conversations and the culture of LLM-as-judge evaluation