Post-training that sorts wrong answers into three conditions and treats each

In a company presentation on sovereign transformation in healthcare, sovereignty is broken into three layers, data, model and operations, and its essence is defined as the freedom of choice to swap out whatever you currently rely on at any time. Half the presentation goes to post-training methodology: starting from a failure in which pouring in data actually made scores go down, it diagnoses wrong answers by splitting them into three conditions. The distinction is that not knowing at all, having the knowledge but not recognizing that this is where to use it, and being strongly convinced of something wrong each call for a different prescription. A diagnostic method is presented as a cheap tool: raise the temperature and ask twenty times to see whether the wrong answer is consistent, and switch the question format between fill-in-the-blank and four-option multiple choice to separate not knowing from failing to connect. The most immediately transferable piece is a gate that sets in advance the metrics that must be protected and throws out that round of training altogether if those scores drop by more than a set amount; the resource savings rate and the edge over competing models are company-reported claims with no measurement conditions given.

This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.