Advanced Multimedia Processing Lab -- Group Member -- Cha Zhang

Group Member

Cha Zhang
PhD. Student

Personal Homepage: http://www.andrew.cmu.edu/~czhang
Email: czhang@andrew.cmu.edu

Office: Porter Hall B10
Lab: Porter Hall B42
Phone: 412-268-7114
Fax: 412-268-1679

Mailing Address:
Department of ECE, Carnegie Mellon University,
5000 Forbes Avenue, Pittsburgh,
PA 15213-3890

The Mobile Camera Array

Dynamic image-based rendering (DIBR) has been an extremely challenging research topic due to the huge amount of data required. This work tried to build a camera array to capture dynamic scenes and render them in real time. Meanwhile, the cameras are mobile such that they can reconfigure themselves automatically according to the scene content in order to achieve best rendering quality.

Non-Uniform Sampling of Image-Based Rendering and Active Image-Based Rendering

If we take pictures of a natural scene from different directions, we get different pictures. Some are simple, some are complex. Some changes fast when the camera moves a little bit, some changes slowly. Some are important to the user, some are less important. In most cases, we may want to take more pictures from one direction than the others. The non-uniform sampling of image-based rendering has the same flavor. More images are taken when the estimated rendering quality is bad. Other criteria, such as user preference, can be seamlessly integrated into the sampling process.

Spectral Analysis and hexagonal sampling for Image-Based Rendering

Image-based rendering often requires many images in order to avoid aliasing or ghosting effects. The sampling analysis of IBR is thus very important. We study the Fourier spectrum of IBR through a novel method. It is capable to analyze scenes with non-Lambertian surfaces and occlusions, as well as generic IBR representations such as concentric mosaics. We also work on generalized sampling, which view the IBR sampling as a multi-dimensional sampling problem, and apply hexagonal or quincunx sampling on IBR.

3D Model Retrieval

3D scene/object browsing is becoming more and more popular as it engages people with much richer experiences than 2D images. As it is easy to build 3D models from real scene/object today, more and more 3D models are available over the Internet. The growing size of available 3D models makes 3D model retrieval very important. This work focuses on extracting features that well represent 3D models and combining these features in order to get a better retrieval performance.

Active Learning in Content-Based Information Retrieval

Hidden annotation has been proved to be a powerful tool to bridge the gap between low-level features and high-level semantic meanings in a retrieval system. We propose to use active learning to improve the efficiency of hidden annotation. We show that with our algorithm, the performance of the retrieval system improves rapidly along with the number of models that have been annotated. In addition, in all the experiments our algorithm outperforms the random sampling algorithm that picks models randomly to give annotation.

Compression, Real-Time Rendering and Streaming of Image-Based Rendering

Image-based rendering has attracted much attention recently in realistic scene/object representation. By taking thousands of pictures for the real scene/object, image-based rendering can render 3D complex scene/object very realistically yet fast. As many data are involved in image-based rendering, compression is essential through the birth of IBR. We propose some new algorithms to compress IBR at very high compression ratio. The compressed bitstream is random-accessible so that real-time rendering is possible with only the compressed bitstream held in the memory. Moreover, we implemented the world's first application to stream IBR over the bandwidth limited Internent.

	The Mobile Camera Array We build a mobile camera array consisting of 48 network cameras. Virtual views can be rendered from the 48 cameras in real time (5-10 fps). Moreover, a unique feature of our camera array is that the cameras are mobile. Each camera can move horizontally, as well as pan. This mobile camera array opens a wide area of novel research topics. See the mobile camera array project page for more information!
	Active Image-Based Rendering We build a system for capturing Image-based rendering scenes intelligently. The system first use a simple algorithm to estimate the rendering quality of each neighborhood. Then it decides the position where more images need to be taken. It takes more images, and then go back to to the first step to find the next position. This iteration runs until the maximum number if images reaches. The result is a non-uniformly sampled IBR. Please refer to the active image-based rendering project page for more details.
	Spectral Analysis and Sampling for Image-Based Rendering Image-based rendering has become a very active research area in recent years. The sampling problem for IBR has not been completely solved. We study the the spectral analysis and sampling of IBR. In terms of spectral analysis, we obtain more general results than previous work by using the surface plenoptic function. After we get the spectrum, generalized sampling for high-dimensional data is used to pack the data in the frequency domain. For more detailed description, please see the IBR sampling project page.
	Active Learning in Content-Based Information Retrieval We propose to use active learning to improve the efficiency of hidden annotation. Active learning has been studied in the machine learning literature. For many types of machine learning algorithms, one can find the statistically "optimal" way to select the training data. This was given the name active learning. Although in traditional machine learning research, the learner typically works as the recipient of data to do training, active learning enables the learner to use his own ability to respond, to collect data, and to influence the world he is trying to understand. To be more precise, what we are interested here is a specific form of active learning, i.e., selective sampling. The goal of selective sampling is to reduce the number of training examples that need to be labeled by examining unlabeled examples and selecting the most informative ones for the human to annotate. For more details on the project, please see the active learning project page.
	3D Model Retrieval In our system, we propose a new set of features that view the 3D model as a solid binary region. Ten features such as volume-surface ratio, aspect ratio, moment invariants and Fourier transform coefficients are extracted. In the current version, we simply use Euclidean distance to measure the similarity between two models after the features are normalized. For more details on the project, please see the 3D model retrieval project page.

Top of this page

Publications

Journal Papers and Book Chapters:

C. Zhang and T. Chen, "A Survey on Image-Based Rendering - Representation, Sampling and Compression", EURASIP Signal Processing: Image Communication, pp. 1-28, Vol. 19, No. 1, Jan. 2004 (invited paper).
C. Zhang and T. Chen, "Spectral Analysis for Image-Based Rendering Data", IEEE Trans. on CSVT, Special Issue on Image-based Modeling, Rendering and Animation, pp. 1038-1050, Vol. 13, No. 11, Nov. 2003.
C. Zhang and T. Chen, "From Low Level Features to High Level Semantics", The Handbook of Video Database Design and Applications, CRC Press, 2003.
C. Zhang and T. Chen, "A New Active Learning Approach for Content-Based Information Retrieval", IEEE Trans. on Multimedia Special Issue on Multimedia Database, pp. 260-268, Vol. 4, No. 2, Jun 2002.
Y. Wu, C. Zhang and J. Li, "Smart-Rebinning for Compression of Concentric Mosaic", IEEE Trans. on Multimedia, Vol. 4, No. 3, Sep. 2002.
C. Zhang and J. Li, "On the Compression and Streaming of Concentric Mosaic Data for Free Wondering in a Realistic Environment over the Internet", submitted to IEEE Trans. on Multimedia.

Technical Reports:

C. Zhang and T. Chen, "A Survey on Image-Based Rendering - Representation, Sampling and Compression", Carnegie Mellon Technical Report: AMP03-03.
C. Zhang and T. Chen, "Active Scene Capturing for Image-Based Rendering with a Light Field Setup", Carnegie Mellon Technical Report: AMP03-02.
C. Zhang and T. Chen, "Spectral Analysis for Sampling Image-Based Rendering Data", Carnegie Mellon Technical Report: AMP03-01.
C. Zhang and T. Chen, "Generalized Plenoptic Sampling", Carnegie Mellon Technical Report: AMP01-06
C. Zhang and T. Chen, "An Active Learning Framework for Content-Based Information Retrieval", Carnegie Mellon Technical Report: AMP01-04

Conference Papers:

C. Zhang and T. Chen, "View-Dependent Non-Uniform Sampling for Image-Based Rendering", submitted to ICIP 2004.
W. Zhang, C. Zhang and T. Chen, "Security Analysis for Key Generation Systems Using Face Images", submitted to ICIP 2004.
C. Zhang and T. Chen, "The Lumivideo", submitted to ACM SIGGRAPH 2004.
J. Li and C. Zhang, "Distributed Hosting of Web Content with Erasure Coding and Unequal Weight Assignment", to appear in ICME 2004.
C. Zhang and T. Chen, "Annotating Retrieval Database with Active Learning", ICIP 2003, Barcelona, Spain, Sep. 2003.
C. Zhang and T. Chen, "Non-Uniform Sampling of Image-Based Rendering Data with the Position-Interval Error (PIE) Function", VCIP 2003, Lugano, Switzerland, Jul. 2003.
C. Zhang and T. Chen, "A System for Active Image-Based Rendering", ICME 2003, Baltimore, Maryland, USA, Jul. 2003.
C. zhang and T. Chen, "Surface Plenoptic Function: A Tool for the Sampling Analysis of Image-Based Rendering", ICASSP 2003, Hong Kong, China, Apr. 2003.
C. Zhang and T. Chen, "On Generalized Sampling for Image-Based Rendering Data", ICASSP 2003, Hong Kong, China, Apr. 2003.
C. Zhang and T. Chen, "Towards Optimal Least Square Filters Using The Eigenfilter Approach", ICASSP 2002. , Orlando, FL, USA, May 2002.
C. Zhang and T. Chen, "Indexing and Retrieval of 3D models Aided by Active Learning", Demo on ACM Multimedia 2001, Ottawa, Ontario, Canada.
C. Zhang and T. Chen, "Efficient Feature Extraction for 2D/3D Objects in Mesh Representation", ICIP 2001, Thessaloniki, Greece.
C. Zhang, J. Li, "Interactive Browsing of 3D Environment over the Internet", Visual Communication and Image Processing (VCIP) 2001, San Jose, California, USA.
Y. Wu, C. Zhang, J. Li and J. Xu, "Smart-Rebinning for Compression of Concentric Mosaics", ACM Multimedia 2000, Los Angeles, CA, USA.
C. Zhang and J. Li, "Compression and Rendering of Concentric Mosaics with Reference Block Codec (RBC)", Visual Communication and Image Processing (VCIP) 2000, Perth, Australia.
C. Zhang and J. Li, "Compression of Lumigraph with Multiple Reference Frame (MRF) Prediction and Just-in-time Rendering", IEEE Data Compression Conference, pp.253-262, Snowbird, Utah, Mar.2000.

Top of this page