AI Agents Learn to Operate Physical Robots Autonomously
A fleet of eight robotic arms at Nvidia’s GEAR lab has spent the past few weeks independently mastering tasks such as inserting pins, seating graphics cards, and cutting zip ties. The only human involvement was in the subsequent documentation of the process.
This capability stems from ENPIRE, a framework detailed in a paper published Tuesday by researchers from Nvidia, Carnegie Mellon University, and UC Berkeley. ENPIRE delegates the entire robot training process to AI coding agents—the same software that already writes and tests its own code—allowing them to execute this training directly on physical hardware.
For the past year, coding agents like OpenAI’s Codex, Anthropic’s Claude Code, and Moonshot’s Kimi Code have been engaged in what researchers term “autoresearch,” a cycle of writing, testing, and rewriting code without human intervention. This loop has largely remained confined to digital environments, where resetting a failed experiment incurs no cost. ENPIRE, however, extends this process into the physical realm, where resetting an experiment necessitates the manipulation of actual robotic arms.
The system operates in two distinct stages. Initially, a human guides the agent in developing two essential tools: a reset routine to return the workspace to its initial state, and a reward function that analyzes camera footage to evaluate success. This setup, akin to a tireless, ever-vigilant referee, is established once and then reused for all subsequent attempts.
Once these foundational tools are in place, the AI agent assumes complete control. It scours published research for relevant ideas, selects training methodologies such as imitation learning, reinforcement learning, or predefined rules, and then modifies its own code before testing the outcome on the robot. This entire loop requires no human oversight, a development that could be seen as either liberating or slightly disquieting, depending on one’s comfort level with unsupervised robots wielding tools.
Nvidia conducted its experiment using eight bimanual robot stations, each equipped with its own hardware, computer, and coding agent. Progress across these stations is shared via Git, a collaborative coding tool, enabling successful strategies to be disseminated across the entire fleet within minutes.
The effectiveness of the system was measured using two tasks: “Push-T,” where a robot uses only pushes to slide a T-shaped block into a designated zone, and pin insertion, which involves threading pins into 4-millimeter holes. Scaling the operation from one robot to eight reduced the time required to master Push-T from approximately five hours to two, and pin insertion from over 90 minutes to about 40 minutes.
Across the four real-world tasks evaluated, the AI agents achieved a 99% success rate with their learned policies, according to the paper. For pin insertion, the agents reached near-perfect reliability faster than a comparable human-in-the-loop method, which typically requires daily human supervision.
Jim Fan, co-lead of the GEAR Lab and director of Nvidia’s AI research, described the project as the first effort to enable autoresearch in the physical world. Fan stated that the team provided the agents with a fleet of robots, GPU resources, and a token budget, then allowed the robots to autonomously complete the tasks.
Bridging the Simulation-Reality Gap
The disparity between simulated environments and the physical world became apparent almost immediately. While all three coding agents successfully completed the Push-T task within a simulator, two of them failed when the same task was transferred to a physical robot, the paper noted. This highlights the challenges posed by real-world factors such as friction, which are not accurately replicated in simulations.
Nvidia also tested ENPIRE within RoboCasa, a simulated kitchen environment designed to assess robots on chores like opening cabinets and turning off stoves. In this benchmark, ENPIRE surpassed both Nvidia’s own end-to-end model, GR00T, and CaP-X, a tool-using agent that bypasses the autoresearch loop entirely.
ENPIRE builds upon an idea Nvidia first introduced with Eureka in 2023, a system that utilized a language model to generate reward functions for robots within a simulator, eliminating the need for manual engineering. ENPIRE takes this self-improvement loop a step further by moving it from simulation to actual hardware, with the AI agent not only designing its own rewards but also its own testing procedures.
This development coincides with Alibaba’s recent unveiling of its own embodied AI initiative, the Qwen-Robot Suite, a collection of foundation models for robot navigation, manipulation, and physics simulation. While Alibaba is developing software for robots it does not manufacture, Nvidia is exploring the potential for AI agents to manage the entire research and development loop on its own hardware. Both approaches signal a growing trend: physical robots are emerging as the next frontier for AI coding agents to compete and innovate.
Leave a Reply