Soft Hand Simulation for Smooth and Robust Natural Interaction

Natural hand-based interaction should feature hand motion that adapts smoothly to the tracked users motion, reacts robustly to contact with objects in a virtual environment, and enables dexterous manipulation of these objects. In our work, we enable all these properties thanks to an efficient soft hand simulation model. This model integrates an articulated skeleton, nonlinear soft tissue and frictional contact, to provide the realism necessary for natural interaction. Robust and smooth interaction is made possible by simulating in a single energy minimization framework all the mechanical energy exchanges among elements of the hand: coupling between the hands skeleton and the users motion, constraints at skeletal joints, nonlinear soft skin deformation, coupling between the hands skeleton and the soft skin, frictional contact between the skin and virtual objects, and coupling between a grasped object and other virtual objects. We have put our effort on describing all elements of the hand that provide for realism and natural interaction, while ensuring minimal and bounded computational cost, which is key for smooth and robust interaction. As a result, we accomplish hand simulation as an asset that can be connected to diverse input tracking devices, and seamlessly integrated in game engines for fast deployment in VR applications.