If it can finish a job on its own, how much can you hand it? — Handing over a whole task
If you can undo it, hand it off; if you can't, look at it yourself.
1 · Think first
There's really only one rule for what you can hand off and what you can't — can you undo it if it goes wrong?
An AI that you give just a goal, and that then searches, opens files and tries again when it fails, all on its own, is called an agent. That's where it parts ways with a chatbot that answers once and is done.
But agents often miss things people find easy. So even when you hand a job off, the trick is to leave a last step where a person looks it over.
2 · Experiment · Pick one
Try it — Can you hand this whole job off?
Here are six situations. Think about just one thing — if it goes wrong, can you undo it or not?
Pick one of the two. As soon as you pick, I'll tell you why.
Figure
Easy for people, hard for AI — the first tests in 2023
These are the numbers from the first time anyone properly measured how well AI can "do it all on its own."
3 · Quiz
Quick check
Three questions. Pick an answer and it's marked right away, with the reason why.
Today in one line
Today in one line
If you can undo it, hand it off;
if you can't, look at it yourself.
Learn more
- The basic shape of an agent: it takes turns thinking and acting — ReAct: Synergizing Reasoning and Acting in Language Models (2022)
- A test that measured AI on everyday errand questions people find easy — people 92%, AI 15% — GAIA: a benchmark for General AI Assistants (2023)
- A test run on real, working websites — people 78.24%, AI 14.41% — WebArena: A Realistic Web Environment for Building Autonomous Agents (2023)