Multiple View Object Cosegmentation using Appearance and Stereo Cues
People
Adarsh Kowdle, Sudipta N. Sinha and Richard Szeliski
Abstract
We present an automatic approach to segment an object in calibrated images acquired from multiple viewpoints. Our system starts with a new piecewise planar layerbased stereo algorithm that estimates a dense depth map that consists of a set of 3D planar surfaces. The algorithm is formulated using an energy minimization framework that combines stereo and appearance cues, where for each surface, an appearance model is learnt using an unsupervised approach. By treating the planar surfaces as structural elements of the scene and reasoning about their visibility in multiple views, we segment the object in each image independently. Finally, these segmentations are refined by probabilistically fusing information across multiple views. We demonstrate that our approach can segment challenging objects with complex shapes and topologies, which may have thin structures and non-Lambertian surfaces. It can also handle scenarios where the object and background color distributions overlap significantly.
Figure: Our approach automatically segments objects in multiple images using the piecewise planar stereo algorithm proposed in the paper and applying multi-view reasoning on the 3D planar segments. (a) One of the 9 images of the COUCH sequence. (b) The segmented object. (c) The 3D plane labeling and (d) its corresponding piecewise planar depth map.
Publications
- Adarsh Kowdle, Sudipta N. Sinha and Richard Szeliski. "Multiple View Object Cosegmentation using Appearance and Stereo Cues", European Conference on Computer Vision (ECCV), 2012. [pdf | supplementary | slides | Video Lecture]
Additional results and comparisons can be found in the supplementary material