This use case is based on HRDexDB: A Large-Scale Dataset of Dexterous Human and Robotic Hand Grasps. All results and metrics are reported by the paper’s authors. For complete technical details, please refer to the original research here.
Truly paired human and robot manipulation data, collected on the same objects under the same conditions within a unified pipeline, remains rare. Without it, studying cross-embodiment imitation learning or manipulation transfer means stitching together incompatible sources.
HRDexDB, developed at Seoul National University and RLWRLD, was built to close that gap. The dataset covers 1,400 grasping episodes across 100 objects and four embodiments, with synchronized 3D hand trajectories, object 6D poses, tactile sensing, egocentric RGBD streams, and grasp outcome annotations.
The HRDexDB capture platform combines 21 calibrated RGB cameras, egocentric stereo views, object tracking pipelines, and tactile sensing to reconstruct hand-object interactions across severe occlusions.
The robotic platform uses a 6-DOF xArm6 manipulator with interchangeable end-effectors — the Allegro Hand and two versions of the Inspire Hand (RH56DFTP and RH56F1). The system is operated through teleoperation, where MANUS gloves and an Xsens motion capture suit map the operator's wrist and finger motions directly to the robot. The consistency of that mapping directly affects the quality and reproducibility of the resulting manipulation data.
In the HRDexDB teleoperation workflow, MANUS gloves capture finger articulation throughout each grasping sequence, providing temporally stable finger tracking throughout each grasping sequence.
HRDexDB is built around paired demonstrations. After recording a human grasping sequence, a teleoperator reproduces the same approach using a robotic embodiment under identical conditions. The result is semantically aligned data across human and robotic hands while preserving embodiment-specific behavior. The dataset also includes unsuccessful grasp attempts alongside successful ones, which provide essential negative samples for learning robust manipulation policies.
For a dataset intended to be reusable across research groups and downstream applications, every component of the capture pipeline has to meet a consistent standard. Teleoperation is one of those components, and hand tracking quality within teleoperation determines the integrity of the resulting robotic trajectories.
In HRDexDB, MANUS gloves are a core part of how robotic manipulation sequences are captured and aligned with their human counterparts. As dexterous manipulation datasets grow in scale and complexity, the systems that support structured and synchronized data collection become part of the research infrastructure itself.
