Human 3D Facial Pose Estimation and Tracking

To track relative movements of the facial landmarks from a video, we have developed a robust tracking approach, in which head movement is also tracked and decoupled from the facial landmark movements. We first employed an state-of-the-art 2D facial alignment algorithm to automatically localize 68 landmarks for each frame of the face video. Then, a 3D face model is used to extract the depth information from 2D frames in order to achieve 3D landmark tracking. Below, you can see a demo of this work:

3D Facial Expression Tracking_Video

Selected Publication:

“Facial Expression and Peripheral Physiology Fusion to Decode Individualized Affective Experience,” 2nd Affective Computing Workshop of the 27th International Joint Conference on Artificial Intelligence (IJCAI-2018), July 13-19, 2018, Stockholm, Sweden. [Code]