Advanced Multimedia Processing Lab -- Projects -- Model Updating

# Project - Model Updating

 Xiaoming Liu xiaoming@andrew.cmu.edu

To have a adaptive model that can learn the dynamic changing appearance of an object.

In patter recognition and computer vision, appearance modeling is an important way for object tracking and recognition. While many modeling methods have been proposed, few of them were dealing with the updating of the model, which is very useful because of the lacking training data and always under-going changing of the object's appearance. Here we propose to update the eigenspace model in order to adaptively learn the changing appearance of an object.

For face recognition applications, when large variations, which may be due to aging, changes in expressions and poses, and variations caused by illumination, etc., appear in the test face images, the traditional modeling algorithms degrade quickly in performance. Although some methods in literature work well for the specific variations being studied, their performance degrades rapidly when other variations are present. In order to approach this general problem, we try to make our face recognition system more intelligent by learning the variations over time while the recognition system is being used.

We use Figure 1 to illustrate the basic idea of our updating method. Since the eigenspace model is composed of the mean and the eigenvectors, the updating of the model can be stated as: given a new sample and the previous model, estimate the new mean and the new eigenvectors. As shown in the following figure, we use a face image to estimate the new mean and the new covariance matrix, which will result in the new eigenvectors.

Figure 1. Learn the changing statistics of face sequences.

The model updating method can be used in detecting the statistics changes of a signal because our method can build an adaptive model for previous samples with respect to the current sample. Thus the large difference between the model and the current sample can be considered as a change. The applications on this topic can be the shot boundary detection for video sequences, the detection of facial expression changes, etc.

Figure 2. Detect the statistics changes of a signal.

Similarly we can use the model updating method for object tracking. As shown in Figure 3, the current tracking result will be considered as a new sample and used for updating the eigenspace model, which can capture the changing appearance of the object and enhance the tracking for future frames.

Figure 3. Face tracking using model updating.

We also propose to use the model updating method to model the visual content in the region of interest (ROI) of a video sequence effectively and adaptively. The proposed Eigen263 modifies the coding framework of H.263 and operates in two modes: Intra and Inter. The decision of either encoding a frame with the Intra or Inter mode depends on how well the current eigenspace represents the ROI in the current frame. If the current eigenspace can represent the ROI with small error, only a few eigencoefficients need to be encoded and transmitted. Otherwise, the ROI is encoded with the Intra mode of the underlying H.263, and is used to update the eigenspace as well.