Advanced Multimedia Processing Lab -- Projects -- Video Based Face Recognition

            About AMP Lab        Projects        Downloads        Publications        People        Links

Project - Video Based Face Recognition


Top of this page


To take advantage video information and enhance face recognition performance compared to image based recognition.   

Top of this page


Although face recognition has been an active research topic for decades, the traditional recognition algorithms are all based on static images. In this project, we would like to propose algorithms for video based face recognition because it has superior advantages over the image-based recognition. Such as, the temporal information of faces can be utilized to facilitate the recognition task; and further, a better modeling technique can be applied to the video sequence of human faces.

Top of this page


First of all, we would like to take advantage the temporal information in video sequences to enhance the face recognition. We propose to use adaptive Hidden Markov Models (HMM) to perform video-based face recognition. During the training process, the statistics of training video sequences of each subject, and the temporal dynamics, are learned by an HMM, as shown in Figure 1. During the recognition process, the temporal characteristics of the test video sequence are analyzed over time by the HMM corresponding to each subject.  The likelihood scores provided by the HMMs are compared, and the highest score provides the identity of the test video sequence. Furthermore, with unsupervised learning, each HMM is adapted with the test video sequence, which results in better modeling over time.  Based on extensive experiments with various databases, we show that the proposed algorithm provides better performance than using majority voting of image-based recognition results.

Figure 1. Temporal HMM for modeling face sequences

Second, to provide a better modeling for different variations in face videos, we propose an approach to generating a statistical face model based on video mosaicing, as shown in Figure 2. Unlike traditional video mosaicing, we use the geometry of a face to improve the mosaicing result. Given a face sequence, each frame is unwrapped onto certain portion of the surface of a sphere, as determined by spherical projection and the minimization procedure using the Levenberg-Marquardt algorithm or the Condensation method. For example, Figure 3 shows how a side-view face image could be unwrapped onto the surface of a sphere. A statistical model containing a mean image and a number of eigenimages, instead of only one image template, is used to represent the face mosaic. After generating the face mosaic model, we are using it for face tracking and face recognition.

Figure 2. Overview for face mosaic model

Figure 3. Spherical projection for faces.


Top of this page


  • Xiaoming Liu and Tsuhan Chen, Geometry-assisted Statistical Modeling for Face Mosaicing, In the Proceeding of the IEEE International Conference on Image Processing 2003, pp. 883- 886, Barcelona, Spain, September 14-17, 2003.

  • Xiaoming Liu, Tsuhan Chen, Video-Based Face Recognition Using Adaptive Hidden Markov Models, In the Proceeding of the IEEE International Conference on Computer Vision and Pattern Recognition 2003, pp.340-345, Vol.1, Madison, Wisconsin, June 18-20, 2003.

Top of this page


Any suggestions or comments are welcome. Please send them to Xiaoming Liu

Top of this page

This website is maintained by Trista Chen
Copyright 200
1 Advanced Multimedia Processing Lab. All rights reserved.
Revised: March 07, 2004 .