At NVIDIA GTC this year, NVIDIA brought Isaac Teleop to General Availability. Isaac Teleop is a unified framework for teleoperation and data collection across simulation and real-world systems, standardizing how human input is translated into robot actions.
This release reflects a broader shift in robotics. As the field moves toward embodied AI and generalist robot policies, the bottleneck is no longer just model architecture, but the availability of high-quality human demonstration data. NVIDIA’s Isaac ecosystem has been evolving to address this, providing an end-to-end stack spanning simulation (Isaac Sim), learning (Isaac Lab), and teleoperation (Isaac Teleop).
MANUS integrates into Isaac Teleop as a high-fidelity hand-tracking input device, capturing precise hand motion in high-dimensional form for real-time use across robotics and simulation environments.
By preserving subtle interaction dynamics such as finger curling, pinch modulation, and micro-adjustments, MANUS gloves enable dexterous manipulation in tasks like tool use, grasp adaptation, and bimanual coordination. This allows teleoperation workflows to incorporate detailed hand behavior that is critical for complex manipulation.
From a system perspective, MANUS integrates into Isaac Teleop as an input device within the teleoperation stack via a dedicated plugin. Hand and finger motion is captured in real time and processed through the retargeting pipeline to generate robot actions.
Isaac Teleop standardizes this pipeline across simulation and real-world systems, enabling consistent control and data collection in both environments. Human motion is translated into robot commands while enabling the collection of demonstration data.
With MANUS, the pipeline captures fine-grained hand and finger motion, improving teleoperation precision and data quality for learning. MANUS integrates into Isaac Teleop through the MANUS Gloves Plugin, which leverage the MANUS SDK* to stream tracking data into the Teleop framework.
For MANUS users, integration with Isaac Teleop simplifies how teleoperation workflows are built and scaled. Instead of building custom integration pipelines for each system, MANUS gloves can be connected through a plugin and streamed into Isaac Teleop. This reduces integration effort and enables reuse across different robots and environments.
It also improves consistency between simulation and real-world systems. The same MANUS setup can be reused across both environments, reducing discrepancies in data collection and supporting more efficient sim-to-real workflows.
Finally, it enables more effective data generation. Demonstrations captured with MANUS can be used in learning pipelines, with high-fidelity finger tracking preserving the detail required for dexterous manipulation and improving the quality of training data.
Within the Isaac ecosystem, Isaac Teleop acts as part of a broader data pipeline that connects simulation, data collection, and learning in Isaac Lab. MANUS enhances this pipeline by providing high‑fidelity hand‑driven input in both simulation and real‑world environments.
Demonstrations collected through Isaac Teleop can be integrated into learning pipelines, preserving fine‑grained motion for dexterous tasks. Through integration with Isaac Lab, developers can drive virtual robot hands using real human motion, enabling rapid prototyping, validation, and data generation in simulation before deploying policies to physical robots.
Together, Isaac Teleop and Isaac Lab provide a tightly integrated workflow from human demonstration to policy training and deployment on real hardware. Recent NVIDIA-led research EgoScale highlights the importance of large-scale human data in training dexterous robot policies. In these approaches, accurate hand and finger motion becomes a critical input signal for learning. Solutions like MANUS enable the capture of high‑fidelity human motion, supporting both small‑scale teleoperation data collection and emerging large‑scale human‑to‑robot learning workflows.
* For further information, see the MANUS Core SDK license.
