99.9% with context kept vs. 62.7% after a reset
Along with the characterization that we have moved from "AI that answers" to "AI that does the work itself," the new model's ability to operate a computer on its own and finish tasks is presented as what sets it apart. Technically, the most useful figure is that the same model scored 99.9% in an environment that kept its earlier work but only 62.7% in one that had been reset, which becomes evidence that an architecture that spins up a fresh session every time itself erodes performance. This gap is tied to the importance of being able to store and process large volumes of data quickly, leading to the outlook that demand will grow not only for HBM but for DRAM and NAND as well. On whether it counts as AGI, the assessment that it is an early version that handles most tasks at or above human level is set against the rebuttal that there is not enough evidence to define it that way. Also noted is that, for the first time, a model that operates a computer on its own received a risk rating in a cybersecurity evaluation.
Key points
- Performance swings by 37 percentage points depending on whether earlier work context is carried along — the basis for designing working memory
- The oversupply outlook was reversed by the logic that cheaper AI gets used more, so infrastructure demand actually grows
- The first security risk rating for a computer-operating model is grounds for requiring network isolation, least privilege and action logging
- Figures like the perfect score on Korea's CSAT college entrance exam or the computer-operation benchmark results are company-reported, so they are hard to cite as is
This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.