Training general-purpose robot manipulation policies requires large amounts of high-quality human demonstration data. However, many existing teleoperation systems struggle to capture dexterous, bimanual manipulation reliably.
Vision-based VR systems such as Apple Vision Pro can suffer from wrist-tracking jitter, occlusions, and inaccurate finger estimation under varying lighting conditions. SteamVR tracking offers better accuracy but requires external lighthouse trackers placed around the environment, limiting portability. Marker-based motion capture systems such as Vicon provide high precision but are expensive and complex to deploy.
To address these challenges, researchers at Carnegie Mellon University developed BiDex, a portable bimanual teleoperation system designed to collect high-quality demonstration data for robot learning.
BiDex is a portable, low-latency bimanual teleoperation system that combines high-fidelity hand tracking with joint-level arm control to enable natural human-to-robot motion transfer. Importantly, MANUS data gloves served as the shared hand-tracking solution across all three systems evaluated in the study, including Vision Pro, SteamVR, and BiDex. This design choice allowed the researchers to isolate the effect of arm tracking method on overall teleoperation quality, rather than conflating arm and hand tracking differences.
The MANUS gloves use EMF sensors to track fingertip positions and estimate joint angles per finger without requiring external cameras or tracking infrastructure, and this data is retargeted in real time through inverse kinematics to map operator hand motion into robot joint space. For arm tracking, BiDex uses GELLO-inspired teacher arms that mirror the operator’s arm motion and provide accurate wrist pose and joint measurements. This combination enables synchronized control of both robot arms and dexterous multi-fingered hands.
Because hand tracking was held constant via MANUS gloves, the performance differences observed across systems can be primarily attributed to how each system handles arm tracking.
In experiments, BiDex outperformed both Vision Pro and SteamVR across multiple manipulation tasks.
In the Pringles handover task, BiDex achieved a 95% completion rate, compared to 80% with SteamVR and 60% with Vision Pro, while completing the task significantly faster (6.5 s vs. 17.5 s and 21.6 s). BiDex also achieved the highest success rate in bottle pouring (85%), compared to 70% with Vision Pro and 60% with SteamVR.
BiDex demonstrates that a key constraint in dexterous robot learning is often the quality of demonstration data, and specifically the fidelity of arm tracking, rather than the learning algorithm or robotic hardware alone. By holding hand tracking constant through MANUS gloves and varying only the arm tracking method, the study provides a cleaner comparison than prior work. The resulting system supports a wide range of portable bimanual manipulation tasks, from chopstick picking and hammering to dish clearing and drilling, enabling the collection of smooth, coordinated demonstrations for tasks that require fine hand articulation and two-handed interaction.
