Training generalist robot manipulation policies requires large amounts of high-quality expert demonstration data. Existing teleoperation systems fall short for dexterous, bimanual tasks:
To address these challenges, researchers at Carnegie Mellon developed BiDex, a bimanual dexterous teleoperation system built around MANUS data gloves.
BiDex is a portable, low-latency bimanual teleoperation system that combines high-fidelity hand tracking with joint-level arm control to enable natural human-to-robot motion transfer. At its core, MANUS data gloves provide accurate fingertip tracking using EMF tracking, capturing the position and rotation of each fingertip and computing joint angles for each finger without requiring external cameras or tracking infrastructure.
This hand data is retargeted in real time through inverse kinematics to map operator hand motion into robot joint space, preserving key dexterous manipulation cues. A GELLO-inspired teacher arm simultaneously tracks wrist pose and arm kinematics, enabling synchronized control of both robot hands and arms.
BiDex outperformed both Vision Pro and SteamVR across teleoperation benchmarks, achieving a 95% handover completion rate compared to 60% with Vision Pro, while completing the task over three times faster (6.5s vs. 21.6s). It was also the only system suitable for mobile operation, as SteamVR requires fixed external lighthouse tracking.
Policies trained on BiDex demonstrations generally performed better than those trained on Vision Pro data across 20 to 100 demonstrations. This may be partly due to BiDex operating in joint space, which produces smoother actions, whereas Vision Pro controls in end-effector space where small prediction errors can lead to larger joint deviations. In a novice user study, participants also tended to find BiDex more accurate, responsive, and easier to use, though results varied across individuals.
BiDex highlights that a key constraint in dexterous robot learning is often the quality of demonstration data rather than the learning algorithm or robotic hardware alone. Within this setup, MANUS gloves support a wider range of portable bimanual manipulation tasks, from chopstick picking and hammering to dish clearing and drilling, by providing wearable, high-frequency finger sensing that can be mapped directly into robot joint space. This enables the collection of smooth, coordinated demonstrations for tasks that require fine hand articulation and two-handed interaction.
