Echoverse: Deep, evolving environments for computer-use agents

Scaling fidelity over sheer count, targeting the capabilities agents actually lack, and evolving with the models they train. At a glance We built twelve training worlds for computer-use agents: ten deep domain worlds and two capability worlds, each drilling a single control rendered in many forms (date pickers and nested filters). Depth is what makes them worth training on: these worlds reproduce an application’s real behavior, come seeded with realistic data, and keep state coherent across screens and users. Trained on all twelve, a 9B model nearly doubles its base score (36.5% to 67.1%), coming within fourteen points of GPT-5.4. The experiment taught us several lessons: High simulation fidelity is a must-have; shallow worlds hurt the agent. Trained on shallow and deep builds of the same sites, the model regressed on the shallow ones but improved on the deep ones. Agents often struggle with the same challenging UI elements, like date pickers and nested filters. Drilling those controls in varied forms taught the model to operate them in domains it never saw in training. Co-evolving the model, the world, and the verifier improves all of them. As the world grows more correct and its tasks grow harder, the model climbs with it. Reinforcement learning against the worlds pushes the agent past imitation. Using the grounded verifier as the reward, RL lifts held-out performance and teaches the agent to reach the goal in fewer steps. We’re releasing four of the worlds with their code, data, and grounded graders, to support research on high-fidelity computer-use worlds. Github: microsoft/Echoverse: Deep, Evolving Environments for Computer-Use Agents (opens in new tab) Hugging Face: microsoft/Echoverse · Datasets at Hugging Face (opens in new tab)Technical Report: https://www.microsoft.com/en-us/research/publication/echoverse-deep-evolving-environments-for-training-computer-use-agents-at-scale/ A computer-use agent learns the results of what its actions do only where they have real consequences. A click changes saved state, a message reaches a real person, or a page that refuses to move tells the agent its last move did nothing. A screenshot can show what an interface looks like, but only a working world shows what an action caused. The consequences worth learning from are stateful, and most of them sit behind a login. The work people want automated lives in closed systems: email and chat, banking, health records, the internal consoles for cloud and ML. You cannot train an agent against the live versions of these. Every attempt writes to a real account, there is no reset between tries, and the true state stays hidden behind the screen. So you rebuild the system as a synthetic world where the database is yours: the state is real and changes for real, but it is safe to break, quick to reset, and graded from the data rather than a screenshot. By a world we mean three things bound together: an environment (the application, its state, and the actions that change it), the tasks that set goals in it, and a verifier that grades the outcome against ground truth. The community is now good at making them: pipelines stand up an application, seed it, generate tasks, and attach verifiers, yielding hundreds of environments and thousands of checkable tasks. This work builds on that progress. However, once worlds are plentiful and its internal structure becomes the bottleneck: regardless of whether state stays coherent across users and screens, workflows keep their dependencies, a weak skill recurs in enough forms to generalize, and success is judged by outcome or by appearance. Our bet, the one Echoverse tests, is that the real leverage comes less from adding worlds than from a loop that keeps improving the ones you already have. It treats building the environment and training the model as one process, not two stages: run a model in a world, find where it fails, make the world, its tasks