Project - Video Based Face Recognition
To take advantage video information and enhance face recognition performance compared to image based recognition.
Although face recognition has been an active research topic for decades, the traditional recognition algorithms are all based on static images. In this project, we would like to propose algorithms for video based face recognition because it has superior advantages over the image-based recognition. Such as, the temporal information of faces can be utilized to facilitate the recognition task; and further, a better modeling technique can be applied to the video sequence of human faces.
First of all, we would like to take advantage the temporal information in video sequences to enhance the face recognition. We propose to use adaptive Hidden Markov Models (HMM) to perform video-based face recognition. During the training process, the statistics of training video sequences of each subject, and the temporal dynamics, are learned by an HMM, as shown in Figure 1. During the recognition process, the temporal characteristics of the test video sequence are analyzed over time by the HMM corresponding to each subject. The likelihood scores provided by the HMMs are compared, and the highest score provides the identity of the test video sequence. Furthermore, with unsupervised learning, each HMM is adapted with the test video sequence, which results in better modeling over time. Based on extensive experiments with various databases, we show that the proposed algorithm provides better performance than using majority voting of image-based recognition results.
Figure 1. Temporal HMM for modeling face sequences
Second, to provide a better modeling for different variations in face videos, we propose an approach to generating a statistical face model based on video mosaicing, as shown in Figure 2. Unlike traditional video mosaicing, we use the geometry of a face to improve the mosaicing result. Given a face sequence, each frame is unwrapped onto certain portion of the surface of a sphere, as determined by spherical projection and the minimization procedure using the Levenberg-Marquardt algorithm or the Condensation method. For example, Figure 3 shows how a side-view face image could be unwrapped onto the surface of a sphere. A statistical model containing a mean image and a number of eigenimages, instead of only one image template, is used to represent the face mosaic. After generating the face mosaic model, we are using it for face tracking and face recognition.
Figure 2. Overview for face mosaic model
Figure 3. Spherical projection for faces.
Any suggestions or comments are welcome. Please send them to Xiaoming Liu.