Is it still holding the first rule at the tenth file?

This is a comparison in which someone who uses two tools eight hours a day gives item-by-item grounds for what made them switch their main tool. The evaluation method is boiled down to one test: have it fix ten files by the same rule and see whether it loses the original rule around the eighth or ninth, which is said to be where the two part ways. As the more important criterion, what is put forward is not performance but how little hands-on attention a tool needs. Three points follow: the round trip of a person receiving the work, checking it and sending it back disappeared once the model began running things itself, fixing whatever broke and only then handing over a result; in screen control, success or failure hinges on whether it knows to wait for the screen to change; and limits should be treated as part of performance, because once you start rationing you stop experimenting. A change of habit also comes up, stripping out all the accumulated rule files and skills and running it stock, on the grounds that outdated instructions actually hold it back. However, the evidence is all one person's impressions, and it leaves out that in organizations rule files serve as audit and standardization mechanisms.

Key points

  • Judges by applying one rule to ten files and checking whether it holds in the later ones
  • Measures the effect by the number of human round trips, not output quality
  • In screen control, knowing to wait for transitions decides the success rate
  • Strips out all accumulated rule files and skills and switches to running stock

This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.

Same week's notes

Agents meant to be isolated exchanged signals with each otherA design that doesn't hand regulatory review wholesale to one modelWhat held back the switch was accumulated setup, not performanceA flood of malicious packages in a short span, and the bar for isolationThe review bottleneck weighs more than the calls to slow down