 |
About AMP Lab Projects Downloads Publications People Links
Project - Image-Based Relighting System
|
HyunJung Kate Shim |
Top of this page
| Motivation and Goal |
Image-based relighting (IBL) has become a popular research topic in both computer graphics and signal processing.
With image-based relighting, one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate a 2D light mapping function from each pixel of the scene image.
In this project, we develop a statistical approach to estimating a 2D light mapping function, an SRF (Surface Reflectance Function), with fewer input images than previous IBL work. An SRF is defined as a weighting function that describes how each point light sources contributes for a particular pixel value on the image. By having an SRF, therefore, we can synthesize an image under arbitrary lighting condition. Our goal is to minimize input images to estimate an SRF while we maximize the quality of relit images. This work can be extended to the environment matting.
Our relighting system consists of single camera (canon G5) , 17" CRT monitor (DEL 17") and arbitrary objects. The wire frame outside of the object supports the camera and the object. The frame is necessary to maintain the relative position between the object and the camera. In order to control the lighting environment, we use the black cloth to cover the entire setup while the experiment is running.
We implement the program to control the camera with following parameters; shutter speed, aperture value, ISO speed, image size, image quality, flash, and white balance. We combine the pan-tilt system as well for future work. With specified camera parameters by an user, the camera starts to capture images under the sequence of lighting patterns
The lighting patterns are designed by the analysis of SRFs, which describe how a pixel on the image is illuminated by each point light source. Above figure shows the definition of an SRF (Surface Reflectance Function). The illumination of a pixel value reflected by the surface is recorded on the image plane by taking the inner product of an SRF and a lighting pattern. We collect the data of SRFs from synthetic images and analyze them by Principle Component Analysis (PCA). PCA results provide us the most representative basis functions of SRFs statistics. Derived basis functions is shifted and scaled into the range of image. Designed patterns are optimal to reconstruct SRFs in mean square error sense.
Above image shows a relighting result from synthetic data. The first image is a rendered image by ray tracing algorithm (POV-RAY) and the second image is a relit image using only fifteen input images.
Above figure is a relighting result of the real objects. The first image is a real image and the second is a relit image using only twenty training images. An applied lighting pattern in above examples does not belong to training images.
This is our relighting demo. This video file contains relit images with corresponding lighting conditions. We have used 20 input images to generate above relit images.
- Kate Hyunjung Shim and Tsuhan Chen "EFFICIENT REPRESENTATION OF LIGHTING PATTERNS FOR IMAGE-BASED RELIGHTING", Picture coding symposium 2004
- Kate Hyunjung Shim and Tsuhan Chen "A STATISTICAL FRAMEWORK FOR IMAGE-BASED RELIGHTING", ICASSP 2005
Any suggestions or comments are welcome. Please send them to Kate Shim
|
This
website is maintained by Wende Zhang |