XZIMG Face Tracking engine is a product developed by XZIMG Research that addresses the Augmented Reality market. Using its components you will be able to create experiences based on the Face Tracking technology.
A component of this Product is dedicated to process images, and a part of this component can be used to process faces that appear in images, by:
- detecting faces (initialization step) using advanced Non-Rigid face detection to build a 3D model of the user in front of the camera, then by,
- tracking the built face model recursively using standard natural feature based tracking algorithms (one could refer to the work of vacchetti/fua/lepetit).
Face detection in details
Discrete Haar wavelets features are simple features also called weak features because it is needed to combine a lot of these feature to obtain a good classification model.
The classification model is learnt offline using a database of images containing human faces and images non containing human faces. Each part of these images is processed at different scales to ensure the model is complete enough.
During the real time experience, a face is located in the image using this model. It returns three degree of freedom: the position of the face center ($u$, $v$) in the image and its scale ($s$). This gives us the opportunity to build a reference model of the face that shows up in front of the camera.
Face tracking in details
It becomes now possible to continue to follow this particular object frame after frame without observing drifts because we constructed a reference model of it. The technique used to track the face is based on a standard natural feature tracking approach, in which:
- feature points are computed in the current frame; then,
- matched both with the previous frame feature points and with the reference model feature points; finally,
- using those matches, is estimated a location of the face, achieved with six degrees of freedom (meaning it returns a position and an orientation).
When the face is lost, the reference model of the face is removed and we restart the face detection process.
The process is fast and quite robust. The unique issue is that it is impossible to initialize the process with a non facing face, while advantages are numerous:
- people’s face are not recognized so that users preserves their anonymity (face data are automatically removed when the tracking is lost),
- process is working on (almost) all type of faces (beard faces, happy faces …),
- it’s fast enough to run efficiently on current smartphones.
A project to promote Transformers the movie
- Paul Viola & Michael Jones, Robust Real-time Object Detection, IJCV, 2001.
- L. Vacchetti, V. Lepetit, and P. Fua, Stable Real-Time 3D Tracking Using Online and Offline Information, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Nr. 10, pp. 1385-1391, 2004