Video2Sim2Real: Full-Stack Autonomous Dexterous Skill Acquisition from a Single Human Video

We present Video2Sim2Real, a full-stack framework for learning dexterous robot skills from a single human manipulation video. The system reconstructs a simulator-ready digital twin, extracts object-centric keyframes and motion priors, optimizes robot configurations in simulation, and bridges the sim-to-real gap through imitation learning, residual reinforcement learning, and collision-aware motion planning. Across everyday manipulation tasks, Video2Sim2Real improves simulated success, safety, trajectory coherence, and real-world transfer over prior approaches.