Robust facial landmark tracking via cascade regression

Recently, tremendous improvements have been achieved for facial landmark localization on static images. However, detecting and tracking facial shapes in sequential images is still challenging due to the large appearance variations in unconstrained videos. To address this issue, we present a robust facial landmark tracking system via cascade regression, which is able to deal well with some challenges emerging in the sequential images. Specially, our system employs a pose-based cascade shape regression model to predict the facial landmark locations. Pose-based cascade shape regression model decreases the shape variances in the model learning stage, making the learned regression model more robust to the large pose variances. In addition, we explore a pose tracking model to enhance the temporal consecutiveness between the adjacent frames, and leverage the Kalman filter to make the predicted shape more smooth and stable. Finally, we incorporate a re initialization mechanism with the facial landmarks as the position priors into the system, which is able to effectively and accurately locate the face when it is misaligned or lost. Experiments on the LFPW, Helen, 300 W and 300 VW datasets illustrate the superiority of proposed system over the state-of-the-art approaches, and it is worthy emphasizing that our method has won the 300 VW competition in the category one.