Dexterous manipulation research relies on demonstration data collected through teleoperation. When an operator controls a multi-fingered hand to grasp an object, in Isaac Sim or on physical hardware, visual feedback alone is not sufficient to capture the physical interaction between the hand and the object. Grip force, slip, and contact timing are not directly observable through vision, which limits the operator's ability to react to these cues during teleoperation and, in turn, affects the precision of the resulting trajectories. This gap is where haptic feedback becomes relevant, giving the operator a physical sense of contact as it happens.
The MANUS Metagloves Pro Haptic capture the operator's hand motion and pass it to MANUS Core, which processes the raw signal into three data types: raw sensor data, joint position data, and joint angle data (ergo data). Developers can then choose the level of abstraction for their retargeting pipeline, working directly with sensor values or with resolved joint angles depending on what the target robot hand requires.
Inside Isaac Sim, the retargeter, running under Isaac Teleop, takes the data from MANUS Core and converts it into commands appropriate for the target robot hand's kinematics. The robot hand hardware supplies its URDF to Isaac Sim, keeping the simulated model consistent with the physical structure it represents, so retargeting stays accurate whether the target is a simulated hand or a physical one connected to the same environment.
Contact events generated by the robot hand, whether simulated in Isaac Sim or measured on physical hardware, can be returned to the operator as haptic feedback through the glove. Instead of inferring grip success from a video feed, the operator feels vibration when the robot hand's fingers meet an object. This closes the control loop between human intent and robot hand response, which improves the operator's ability to modulate grip force and adjust in-hand positioning during teleoperation.
Isaac Lab uses the resulting teleoperation sessions as training data for manipulation policies. Because the demonstrations were collected with haptic feedback, the trajectories reflect more deliberate contact and grasp adjustments than motion capture alone would produce. This matters for sim-to-real transfer, since policies trained on demonstrations that already account for contact forces are working with data closer to what the physical robot will encounter outside simulation.
