NoteGraph Engineering

Five criteria for deciding whether to split up agents

On the same task, depending on whether you use one coding agent or split the work up, performance can rise by more than 80% or drop by 70%. Anthropic's Research feature improved by more than 90% with parallel splitting and an academic experiment in financial analysis by 80.8%, but on planning tasks that have to proceed in order, split configurations fell by as much as 70%. The rule that splitting only hurts on tasks a single agent already solves about 45% of the time correctly predicted 94% of coding benchmark results, and when Berkeley researchers classified 1,600 multi-agent execution traces, 44% of failures came from system design, 32% from communication between agents and 23% from problems with checking results. In structures where agents don't check one another, errors spread up to 17-fold, while a centrally checked structure cuts that to 4.4-fold, at the cost of tasks handled per 1,000 tokens falling from 67.7 to 21.5. In the end, the graph should be designed not by how many agents to bolt on but by five criteria — whether work can run concurrently, the single-agent success rate, whether there are intermediate checks, who owns the state, and the stop condition — and the cited figures need to be checked against the original papers.

Key points

  • Parallelizable tasks gain 80–90% from splitting; sequential planning tasks drop by up to 70%
  • The rule 'if single-agent success is 45% or higher, splitting hurts' matched 94% of benchmark results
  • Causes of failure: system design 44%, communication 32%, checking 23%
  • Central checking cuts error propagation from 17-fold to 4.4-fold, but throughput per token falls to a third

This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.

Same week's notes

Three concepts the term 'graph engineering' lumps togetherHermes, a personal multi-agent system split up by roleSutton's diagnosis: an LLM whose weights never change is not a mindKorea's independent AI model project: second-round results and an agent for every citizenFrom teleoperation to onboard vision, the evolution of robots that see the ball