Non-Rigid Face Tracking (1)

I wrote recently on Face Detection and Face Tracking. Both are based on a rigid approach to detect and track a face. Let’s talk about Non-Rigid Face Tracking! In this section we’ll talk about Shape Model.

Rigid Face Tracking is good for simple Augmented Reality application such as displaying a hat or sunglasses on the user’s face. Non-Rigid Face Tracking consists in finding landmarks at some locations of interest.

Face detection rectangle (left) and landmarks (right)

The first thing to do is to create or find a database of faces which contains as many faces as possible. Along with these faces, landmarks positions must be set. A database of interest is the MUCT face database (3500+ faces with landmarks).

Once your data are prepared, shapes (made of landmarks) must be a aligned using what we call the “Procrustes Distance based Alignment”. Result of this procedure is depicted on the picture here-under.

Procrustes Analysis on the MUCT database

Aligned shapes along with the mean shape

Once shapes are aligned, we compute the standard PCA analysis to find main shape distortions. Here is an example of PCA shapes that can be obtained.

Results of PCA

Resulting PCA mean (center) and main rigid basis deformations


One thought on “Non-Rigid Face Tracking (1)

Leave a Reply

Your email address will not be published. Required fields are marked *


− 2 = five

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>