Loop & Harness Engineering
44 records · 9 notes · records tagged with this as a related axis are included.
How do you run a system that isn't done after a single call?
When you actually run an agent, the hard part is less the model than everything around it. How far to keep trying and when to stop, what to redo after a failure, what to remember and what to drop on a long task — this shell wrapped around the model is called a harness. This axis gathers records on designing repeat loops, managing in-between state, and the points where a person steps in. The same model can give very different results depending on its harness, so it's an axis you should always read alongside performance claims.
A hands-on run that follows all the way through the work of nailing down, as files, a structure that removes the need to ask for a fix every time. The structure has six parts: a standards document setting down tone and d
The authors reported that on OSWorld-v2 the average partial score of the 8 models evaluated rose by 19.6 points and that steps per task fell by 53.0% on average compared with running screen control alone. They released t
Company-reported, a single Sol agent at 52.1% rose to 61.8% once it worked as a team, and a team of a Sol lead with 16 Luna planners reached 61.8% for $155, about 30% of the cost of an all-Sol team that delegated every s
Under explicit assumptions it gives an expected generalization bound, a retention bound conditional on historical regression budgets, and a high-probability rule for admitting revisions using fresh independent data. It c
The authors reported that a single step raised GLM-5 and Gemini 3.5 Flash over their base harnesses by up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points on LAB. Research on recursive
The optimization target of harness engineering has widened from "what to fix" to "what to test". Abstracting failures into patterns to compose the curriculum automatically takes over the place where people used to pick s
The key is that speed has been split off into a separately billed tier. When latency becomes the bottleneck of agent loops, tokens per second become a product separate from quality, and investment in inference infrastruc
An approach that retrieves and adapts skills from repositories instead of learning them anew for each harness, turning skills from outputs tied to a model and harness into transferable assets. Read together with ClawScan
The first major commercial product to sell an agent not as "a session that runs when asked" but as "a persona resident in the account". The billing unit shifts from tokens and seats to the number of agents, and always-on
The operational foundation for agents moving from "running when a person calls them" to "waking up on their own" has become a GA cloud product. It matters that the option of giving agents their own identity came out at t
It recasts why agents break down on long tasks — not "because they cannot read the environment" but "because stale state descriptions they wrote themselves linger". It means what a harness has to manage is not tool calls
The key is that the target of self-improvement is the harness configuration, not the model weights — a measurement showing there is still room to cut costs by a third with the same model.
Backs "the stronger the model, the thinner the harness should be" with measurements across 176 configurations. It marks the point where design guidance for axis ⑤, loop and harness engineering, moves from rules of thumb
Context-compaction summaries are a channel a model leaves for itself — a frontier lab has disclosed for the first time, with figures, that the attacker in a prompt injection can be the model itself rather than an outside
Publicly correcting the conclusion given four months earlier, of using different tools for different purposes, the write-up now says all plugins that enforce procedures have been removed and only one setting, thinking ef
A head-on attack, through data separation, on the chronic problem of harness self-improvement: "fitting only the benchmark". Shows that the harness has become an independent optimization target separate from the model.
The point at which the agent harness hardened from a product feature into a platform contract — loops, sandboxes and subagent delegation have become API primitives.
The point made is that agent billing attaches not to the length of the request sentence but to the length of the actual execution path. Even a one-sentence request to add a single calendar event turns, behind the scenes,
Shows that the battlefield for coding agents has moved from new development to modernizing legacy code, a core corporate asset, and that the combination of checking harnesses and human gates is hardening into a working p
The first misconception to clear away from the news that 13 million lines of proof code were produced in 11 days is that a long-standing open problem was solved. The proof itself was completed by humans in 1994; what was
This is a comparison in which someone who uses two tools eight hours a day gives item-by-item grounds for what made them switch their main tool. The evaluation method is boiled down to one test: have it fix ten files by
This is a record of someone who never learned a professional 3D production tool using it for a day with nothing but an integration interface attached. It pins down the reason they had put it off not as a lack of skill bu
Herdr is a remote terminal server far lighter than Orca; it installs easily on anything with a terminal, including a Linux VM, and sessions stay alive even when Ghostty is closed. Put it on Grok Bot's virtual machine and
The 12-layer stack, said to be distilled from experience with more than 400 production agents, is easiest to grasp by toggling each layer on and off as a boolean flag in a hypothetical emergency-room triage agent and rep
A strand that does not take LLM-extracted graph edges at face value but recalibrates edge weights after the fact through counterfactual intervention — tackling, before the retrieval stage, the edge-reliability problem lo
The Hermes-based personal multi-agent system is split by role into Coder, which builds software autonomously; LifeBot, a productivity and health coach on Telegram; and Taco Bot, which acts as the team's COO. Coder is a w
With the Codex CLI, app-server, SDK and MCP open-sourced, the harness is established as a platform layer
The inner loop vs. outer loop framing is elevated to the industry's standard narrative
Establishes the popular definition of loop engineering
The representative critique that loops work for checkable, short-lived tasks but are fragile for code that has to be maintained
Takes an enterprise view, counting observability and governance as part of the harness
Problematized the term's many meanings academically, proposing as the condition a control mechanism that does not depend on the model's compliance
Established the term loop engineering; a shift in role from prompt writer to loop designer
The containment position: the harness is the delivery vehicle of context engineering
Harness-induced variance was 7.80× model-induced variance, and rankings flipped in 6 of 9 model pairs
The key chronicler of vibe coding's spread declared that the very distinction between the two axes of 2026 coding discourse is breaking down
The standard reference for the user-side definition of the harness as everything in an AI agent except the model
3 agents — Planner, Generator, Evaluator; a single agent ran 20 minutes for $9 versus 6 hours for $200 with the full harness
Defined harness engineering as a subset of context engineering, with the concepts of a context firewall and back-pressure
At least 59.4% of the audited problems had flawed tests; the vendor acknowledges that the benchmark harness's signal is contaminated
The first presentation of the guides/sensors × computational/inferential frame
An internal product of about 1 million lines written entirely by Codex; over 5 months, 3–7 engineers merged about 1,500 PRs
The starting point for the term's spread; step 5 of the 6 steps of AI adoption is Engineer the Harness
A case of the issuing organization itself showing that a change in methodology alone moves the numbers. The top model's 50% time horizon of about 5.3 hours is based on this revision and cannot be compared directly with t
The primary source for the line of definitions that treats the harness as an operating system
Harness interfaces moved from objects of vendor competition to shared infrastructure; AGENTS.md adopted by 60,000+ projects
A vendor making a community-born loop pattern official
Named the Claude Agent SDK explicitly as a general-purpose agent harness; the turning point at which a frontier lab adopted harness as a formal term
Names four patterns: compaction, structured note-taking, sub-agents and just-in-time retrieval
A contemporaneous record that pins down the generational handover from prompt engineering
The prototype of loop engineering that discards the context on every iteration and keeps state in git and files
Proposed a new coordinate system: time of autonomous work rather than benchmark scores
The full text confirms that the term was used in the evaluation community first, before it went mainstream