I wrote recently on Face Detection and Face Tracking. Both are based on a rigid approach to detect and track a face. Let’s talk about Non-Rigid Face Tracking! In this section we’ll talk about Shape Model.
Rigid Face Tracking is good for simple Augmented Reality application such as displaying a hat or sunglasses on the user’s face. Non-Rigid Face Tracking consists in finding landmarks at some locations of interest.
The first thing to do is to create or find a database of faces which contains as many faces as possible. Along with these faces, landmarks positions must be set. A database of interest is the MUCT face database (3500+ faces with landmarks).
Once your data are prepared, shapes (made of landmarks) must be a aligned using what we call the “Procrustes Distance based Alignment”. Result of this procedure is depicted on the picture here-under.
Once shapes are aligned, we compute the standard PCA analysis to find main shape distortions. Here is an example of PCA shapes that can be obtained.
- Computer Vision with OpenCV : Mastering OpenCV with Practical Computer Vision Projects. The book is quite interesting but very incomplete (worth a quick overview).