Researchers train T1, a 122B MoE terminal agent, with RL in cloud sandboxes

Trained T1, an MoE model with 122B total parameters, with reinforcement learning as it operated real shells in cloud sandboxes for more than 300 turns per task, rewarded by each task's own checkers. Warm-starts actor-critic training with a dense process reward that scores the absolute number of checkers passed, and stabilizes optimization with a TITO setup — training on the sampled token IDs as they are and correcting drift at turn boundaries — and rollout routing replay (R3), which records and replays the sampler's per-MoE-layer expert choices. The authors claim it cut the training-inference log-probability gap from 0.021 to 0.013 and brought token drift in the loss region to zero; that, although trained only on seed and synthetic tasks kept separate from Terminal-Bench 2.1, it raised the Terminal-Bench 2.1 resolution rate from the base model's 43.8% to 64.0%; and that it scored 27.9% on Long-Horizon Terminal Bench, ahead of GPT-5.4 and GLM-5.1.