RefinedWeb

RefinedWeb

Why it matters

Claimed that well-filtered web data alone can outperform curated corpora; MinHash fuzzy deduplication became a standard technique

Source — Primary sources first; secondary reports are labelled “Press”

Paperarxiv.orghttps://arxiv.org/abs/2306.01116

Added in Week 35, 2026 · See that week

Same week

AWQPaperDPOPaper

Nearby on the same axis

Textbooks Are All You NeedPaperRedPajama-Data-1TOfficialNLP Evaluation in troublePaperRethinking Benchmark and ContaminationPaper