At CoRL in November 2026, NVIDIA introduced GR00T 2, which addresses a central challenge in robot learning. Robots must acquire physical skills from human experience and adapt them to bodies that differ from our own. Built on NVIDIA Cosmos, GR00T 2 jointly predicts future video and robot actions, linking how a scene evolves to the movements that cause those changes.
A key advance is cross-adaptation, which transfers skills demonstrated by humans to a robot without teleoperated demonstrations of those tasks. MANUS gloves capture the finger articulation behind human manipulation. Combined with wrist tracking and synchronized video, this hand motion data becomes structured robot training data for dexterous manipulation.
NVIDIA’s human-to-robot capture system combines MANUS data gloves to measure finger joint angles, wrist-mounted PICO trackers to capture wrist position and orientation, and head-mounted and wrist-mounted cameras to record synchronized multi-view video. These streams provide visual and motion data for training GR00T 2 to learn to predict future observations and robot actions.
Demonstrators perform tasks directly with their own hands, without a robot following during recording. This separates demonstration capture from robot availability and preserves natural movement, including fast, targeted actions that the researchers’ teleoperation setup could not reproduce.
Human and robotic hands differ in size, proportions, and joint structure. Transferring a demonstration requires preserving the geometry of the interaction while translating movement into the robot’s available degrees of freedom.
NVIDIA first aligns the capture setup with the robot. Cameras are positioned at robot-matched viewpoints, and recorded wrist trajectories are expressed in the gravity-aligned robot base frame. A task-specific spatial offset places the trajectories within the robot’s workspace.
MANUS finger joint measurements, combined with wrist tracking, reconstruct the demonstrator’s 3D hand pose. NVIDIA’s geometric retargeting solver then establishes a shared reference frame anchored at the index fingertip. It positions the robot’s thumb relative to that fingertip to preserve the thumb–index pinch geometry, while the remaining fingers follow the human finger-bone directions.
Using fingertip targets, the solver computes joint-angle targets for the 22-degree-of-freedom Sharpa hand, then solves for the seven-degree-of-freedom arm configuration needed to position the hand.
For an interaction such as grasping a pipette, the objective is to preserve the fingertip relationship that enables the grasp. The robot’s joint angles can differ from the demonstrator’s while retaining the relevant contact geometry.
During post-training, NVIDIA combines human demonstration videos and their retargeted action labels with general interaction recordings from the target robot at a 1:1 sampling ratio. These robot recordings, termed “task-agnostic play data,” exclude the target tasks and provide examples of the robot’s appearance, movement, and interaction with its environment.
Building on cross-embodiment pretraining, GR00T 2 learns from both sources through joint video-and-action prediction. Human demonstrations provide the task strategy, while robot recordings ground it in the target body. Both sources train future-video prediction and share the robot action head.
This approach enabled GR00T 2 to perform dexterous manipulation tasks without task-specific teleoperated demonstrations.
Simulation expands GR00T 2’s training and evaluation capabilities. NVIDIA’s human-to-simulation-to-real workflow reconstructs objects and hand–object motion from human video, then trains a reinforcement-learning expert in simulation. Randomized conditions and execution disturbances generate diverse demonstrations and recovery behaviors, rendered in NVIDIA Isaac Lab and used to fine-tune GR00T 2.
Together, human demonstrations, robot experience, and simulation support skill acquisition with less task-specific teleoperation. MANUS provides the finger-motion measurements that help translate human dexterity into robot training data, supporting more adaptable robotic manipulation.
Explore the full GR00T2 Whitepaper, including the human demonstration pipeline in Section 4.4.
