Automatic 3D virtual fitting system based on skeleton driving

To facilitate the input and improve the practicality of the existing virtual fitting systems, we propose a fully automatic 3D virtual fitting system to fit garment onto human models with various shapes and poses using a 2D full-body image and a 3D garment model. The proposed method constructs the 3D human model from the input 2D full-body image of the user by adopting the SMPLify method. To automatically position garment models onto human models with arbitrary postures, we present a 3D mesh segmentation method based on the discrete Reeb graph to accurately segment the different parts of a garment model, and a skeleton driving method based on mean curvature flow, which automatically adjusts the posture of the garment model according to the skeleton structural difference between the human model and the garment model. In addition, for the purpose of obtaining a more natural dress effect, we further adopt interpenetration removal and physical simulation for the deformed garment model. Compared to existing automatic 3D virtual fitting systems, the experimental results, we obtained based on the Leeds Sports Pose dataset, reveal that the proposed virtual fitting system is stable and effective.

Automated summary

This study introduces a fully automatic 3D virtual fitting system that can automatically fit garment onto human models with various shapes and poses. The system constructs a 3D human model from a 2D full-body image and positions garment models onto the human models with arbitrary postures. Additionally, techniques such as interpenetration removal and physical simulation are used to achieve a more natural dress effect.