In this post, we continue to consider the Shape Model of the Non-Rigid Face Tracking principle. Previously, we illustrated how to construct a Shape Model. Now we emphasize on the Shape Model analytical expression.

**Edit: **a flash-based demonstraction is available here.

A shape $x$ is expressed as a linear function:

$$\mathbf{x} = \overline{\mathbf{x}} + Pb$$

where $\overline{\mathbf{x}}$ refers to the mean shape, $P$ is a matrix constructed using the few first eigen vectors obtained from the Principal Component Analysis on Shapes, and $b$ are the corresponding modes of variation.

This formulation is our Shape Model also known as the Point Distribution Model (PDM). By varying values of $b$ (usually between $+\pm3\sigma$), we can model different face deformations as depicted here-under.

The PDM model is nice, however faces can appear at any place (left, right, including fat or close to the camera) and at any orientation (even non facing ones) in images. That’s a problem because the PCA analysis on such a database will fail to deliver a correct PDM. In this case, few modes of variations (after truncation) will not be able to explain a large enough variance over the data (see left image in the following illustration).

So we need to align the faces before building our PDM with the face databse. To do that, a transformation is used:

$$ \mathbf{x} = s.R(\overline{\mathbf{x}} + Pb) + t$$

The procrustes alignment procedure gives a correct estimation of the PDM (see Cootes paper to understand how to align a set of shapes).

# A Flash Sample

Non rigid face-tracking. It’s in flash so not as robust as the c++/mobile application.

**Links**:

- T. Cootes, CJ.Taylor, D.H.Cooper and J.Graham – Training Models of Shape from Sets of Examples.

Dear Sir.

I am pleasure due to your face tracking example.

Now, I am analyzing the source code of book of “Mastering OpenCV with Practical Computer Vision Projects”.

And I simulated the book source.

However, I can’s obtain a sucessful face tracking result compared to your flash example.

I wonder several different points or algorithms between book source and your flash example.

Thank for reading.

Sincerely Dong-Ju Kim

Hi,

Flash example is not based on the approach you mentionned above. We rather use Support Vectors to construct the classification of face pixels.

Good luck!

Thank you for reply.

One more,

Can you teach me the reference papers to implement your example?

Sincerely Dong-Ju Kim

Unfortunately, I can’t tell you too much since the technology is being part of a commercial product.

However, I’m sure you’ll find out by yourself interesting papers on the topic.