The debate over a model that returns only probabilities instead of sentences

The back-and-forth over a model that, instead of answering a question in long sentences, returns only a number for how likely each predefined option is to be correct is laid out with the arguments of both sides. The criticism is that a small classification model could do the same job and that the speed comparison was set up unfairly; the rebuttal rests on hardware — the sequential dependency of generating tokens one at a time remains even if you shrink the model or add more compute units, so throughput and response latency are separate problems. Hence the proposal to reset the yardstick from whether it works at all to which side waits less and spends less at the same accuracy, followed by a discussion of probability calibration: only if a judgment stated as 90% is actually right that often can you decide how far to automate. An earlier case that solved the same waiting problem with a dedicated chip is set alongside. However, it never actually presents the comparison figures by the very yardstick it set up, so the conclusion ends as a request rather than an argument.

This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.