Given two answers, which one would a person score higher? — Humans scoring answers
Nobody told it the right answer, people only told it which one was better.
AIMade by AIThis lesson was written by AI (Claude) and published after automated checks. The owner sets the rules and doesn't read every lesson.How it's made
1 · Think first
You have the AI write two answers to the same question, then a person picks the better one, tens of thousands of times over. Those picks become the material for the next round of training.
It's not grading right or wrong. What gets collected is people's taste. First you build a small scoring model that copies that taste, then you polish the AI by following its scores.
It worked really well. In research released in 2022, people liked the answers of a small model polished this way more than the answers of a model over 130 times bigger.
2 · Experiment · Pick one
Try it — If you were the human scorer, which one would get your point?
In six situations you get two answers each. Give the point to the one that looks better. Once you've picked them all, you'll see the standards people used.
The 'right' pick isn't the more correct answer, it's the one more people chose. Pick like you're the scorer.
Figure
How people's picks turn into scores
People don't write the answers. They just pick one of two.
3 · Quiz
Quick check
Three questions. Pick an answer and it's marked right away, with the reason why.
Today in one line
Today in one line
Nobody told it the right answer,
people only told it which one was better.
Learn more
- The 2022 paper that showed this method at scale — a small model beat one more than 130 times bigger — Training language models to follow instructions with human feedback (2022)
- The official post that explains the same research in plain words — Aligning language models to follow instructions (OpenAI, 2022)
- There's another path too: scoring with written-down principles instead of people — Constitutional AI: Harmlessness from AI Feedback (2022)