Scribble Based Interactive 3D Reconstruction via Scene Cosegmentation
People
Adarsh Kowdle, Yao-Jen Chang, Dhruv Batra, Tsuhan Chen
Abstract
In this work, we present a novel interactive 3D reconstruction algorithm which renders a planar reconstruction of the scene. We consider a scenario where the user has taken a few images of a scene from multiple poses. The goal is to obtain a dense and visually pleasing reconstruction of the scene, including non-planar objects. Using simple user interactions in the form of scribbles indicating the surfaces in the scene, we develop an idea of 3D scribbles to propagate scene geometry across multiple views and perform co-segmentation of all the images into the different surfaces and the non-planar objects in the scene. We show that this allows us to render a complete and pleasing planar reconstruction of the scene along with a volumetric rendering of the non-planar objects. We demonstrate the effectiveness of our algorithm on both outdoor and indoor scenes including the ability to handle featureless surfaces.
Video showing summary of algorithm and 3D visualization of the results [High Res]
Publications
- Adarsh Kowdle, Yao-Jen Chang, Dhruv batra and Tsuhan Chen. "Scribble Based Interactive 3D Reconstruction via Scene Co-segmentation", Oral at International Conference on Image Processing, 2011 (ICIP '11). [pdf | slides]
- Adarsh Kowdle, Yao-Jen Chang and Tsuhan Chen. "i3D:Interactive Planar Reconstruction of Objects and Scenes (Preliminary work)", Western New York Image Processing Workshop, 2009 (WNYIPW '09). [pdf] [slides]
Cornell Multiview Dataset
We make publicly available two sets of multiview datasets as the Cornell Multiview Dataset.
- Object_Multiview: Images of an 'object of interest' captured from multiple viewpoints around the object, with the goal of co-segmentation and obtaining a 3D model of the object of interest
- Scene_Multiview: Images of a scene captured from multiple viewpoints with the goal of dense depth estimation of the scene.
If you use this dataset for research purposes, please cite us in any resulting publication. The following are the relevant publications,
Object Multiview:
- Adarsh Kowdle, Dhruv Batra, Wen-Chao Chen and Tsuhan Chen. "iModel: Interactive Co-segmentation for Object of Interest 3D Modeling ", RMLE Workshop at the European Conference on Computer Vision, 2010 (ECCV '10).
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance", International Journal of Computer Vision, 2011 (IJCV '11).
Scene Multiview:
- Adarsh Kowdle, Yao-Jen Chang, Andrew Gallagher and Tsuhan Chen. "Active Learning for Piecewise Planar 3D Reconstruction", Computer Vision and Pattern Recognition, 2011 (CVPR '11).
- Adarsh Kowdle, Yao-Jen Chang, Dhruv Batra and Tsuhan Chen. "Scribble Based Interactive 3D Reconstruction via Scene Co-segmentation", International Conference on Image Processing, 2011 (ICIP '11)
Software: iScribble
We have made the graphical user interface developed by us available to the community. The above project uses the same GUI with some simple modifications. We feel that this GUI is a powerful yet simple tool which can be easily integrated with Matlab. Please feel free to use it and give your feedback.
You can download the interface from the iScribble project page.
