Playing them to the end exposed the flaws in what was generated
This is a record of having two commercial models build games in three genres from the same one-line instruction, then playing the finished products all the way through by hand to compare them. What is worth taking away is not who won but two things, one being the observation that quality jumped not when the model was changed but when an asset-generation service and a 3D production tool were connected via MCP. The other is that all three genres looked fine at first, but playing through to the end revealed defects: broken character images, failed rigging, and a final boss whose health reset when it was almost beaten. This means that for automatically generated output, whether it runs and whether its rules stay consistent are separate questions, leaving the implication that the test standard should be a full end-to-end run rather than a single pass. A figure is also given that costs concentrated not in code tokens but in asset-generation credits, with one 3D task costing about $100; but the judgments are all one person's impressions and conditions differed from round to round, so it cannot be used as grounds for choosing a model.
Key points
- Quality jumped on connecting external tools, not on switching models
- Broken images, failed rigging and health resets surface only when played to the end
- Costs concentrate in asset-generation credits, not code tokens
- Judgments are one person's impressions and round conditions are uneven, so it can't be reproduced
This is a note: my own write-up after reading several sources. It has no source link; events checked against a source are kept as records.