XZIMG Face Detection and Face Tracking Technology

XZIMG Face Tracking engine is a product developed by XZIMG Research that addresses the Augmented Reality market. Using its components you will be able to create experiences based on the Face Tracking technology.

A component of this Product is dedicated to process images, and a part of this component can be used to process faces that appear in images, by:

  • detecting faces (initialization step) using advanced Non-Rigid face detection to build a 3D model of the user in front of the camera, then by,
  • tracking the built face model recursively using standard natural feature based tracking algorithms (one could refer to the work of vacchetti/fua/lepetit).

Face detection in details

Discrete Haar wavelets features are simple features also called weak features because it is needed to combine a lot of these feature to obtain a good classification model.
The classification model is learnt offline using a database of images containing human faces and images non containing human faces. Each part of these images is processed at different scales to ensure the model is complete enough.
During the real time experience, a face is located in the image using this model. It returns three degree of freedom: the position of the face center ($u$, $v$) in the image and its scale ($s$). This gives us the opportunity to build a reference model of the face that shows up in front of the camera.

Face detection initialization with D'Fusion

Non rigid Face detection initialization. The reference model is used to track the face.

Face tracking in details

It becomes now possible to continue to follow this particular object frame after frame without observing drifts because we constructed a reference model of it. The technique used to track the face is based on a standard natural feature tracking approach, in which:

  • feature points are computed in the current frame; then,
  • matched both with the previous frame feature points and with the reference model feature points; finally,
  • using those matches, is estimated a location of the face, achieved with six degrees of freedom (meaning it returns a position and an orientation).

When the face is lost, the reference model of the face is removed and we restart the face detection process.

Face tracking XZIMG

Face tracking combined with Augmented Reality does the magic!

Conclusion:

The process is fast and quite robust. The unique issue is that it is impossible to initialize the process with a non facing face, while advantages are numerous:

  • people’s face are not recognized so that users preserves their anonymity (face data are automatically removed when the tracking is lost),
  • process is working on (almost) all type of faces (beard faces, happy faces …),
  • it’s fast enough to run efficiently on current smartphones.

Youtube Video:

A project to promote Transformers the movie

Links:

4 thoughts on “XZIMG Face Detection and Face Tracking Technology

  1. I wonder how can we use the natural features to track.
    And i test LK demo in Opencv to define features on my face for tracking.
    However, the tracking result is so bad, i can not use it for tracking

    • Hi, thanks for your comment.

      There exists many different recipes that use natural features to track a face. I’m not allowed to give too many details on that but a good starting point is to read the paper by Vacchetti, Lepetit & Fua mentioned in the links section.

Leave a Reply

Your email address will not be published. Required fields are marked *

*


five − 3 =

* Copy This Password *

* Type Or Paste Password Here *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>