Active Learning for Piecewise Planar 3D Reconstruction
People
Adarsh Kowdle, Yao-Jen Chang, Andrew Gallagher, Tsuhan Chen
Abstract
In this work, we present an active-learning algorithm for piecewise planar 3D reconstruction of a scene. While previous interactive algorithms require the user to provide tedious interactions to identify all the planes in the scene, we build on successful ideas from the automatic algorithms and introduce the idea of active-learning, thereby improving the reconstructions while considerably reducing the effort.
Our algorithm first attempts to obtain a piecewise planar reconstruction of the scene automatically through an energy minimization framework. The proposed active-learning algorithm, then uses intuitive cues to quantify the uncertainty of the algorithm and suggest regions, querying the user to provide support for the uncertain regions via simple scribbles. These interactions are used to suitably update the algorithm, leading to better reconstructions. We show through machine experiments and a user study that the proposed approach can intelligently query the users for interactions in informative regions, and users following these can achieve better reconstructions of the scene and faster, especially in case of textureless surfaces and scenes lacking cues like lines which the automatic algorithms rely on.
Video showing summary of algorithm and 3D visualization of the results [High Res]
Publications
- Adarsh Kowdle, Yao-Jen Chang, Andrew Gallagher and Tsuhan Chen. "Active Learning for Piecewise Planar 3D Reconstruction", Computer Vision and Pattern Recognition, 2011 (CVPR '11). [pdf | slides]
Cornell Multiview Dataset
We make publicly available two sets of multiview datasets as the Cornell Multiview Dataset.
- Object_Multiview: Images of an 'object of interest' captured from multiple viewpoints around the object, with the goal of co-segmentation and obtaining a 3D model of the object of interest
- Scene_Multiview: Images of a scene captured from multiple viewpoints with the goal of dense depth estimation of the scene.
If you use this dataset for research purposes, please cite us in any resulting publication. The following are the relevant publications,
Object Multiview:
- Adarsh Kowdle, Dhruv Batra, Wen-Chao Chen and Tsuhan Chen. "iModel: Interactive Co-segmentation for Object of Interest 3D Modeling ", RMLE Workshop at the European Conference on Computer Vision, 2010 (ECCV '10).
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance", International Journal of Computer Vision, 2011 (IJCV '11).
Scene Multiview:
- Adarsh Kowdle, Yao-Jen Chang, Andrew Gallagher and Tsuhan Chen. "Active Learning for Piecewise Planar 3D Reconstruction", Computer Vision and Pattern Recognition, 2011 (CVPR '11).
- Adarsh Kowdle, Yao-Jen Chang, Dhruv Batra and Tsuhan Chen. "Scribble Based Interactive 3D Reconstruction via Scene Co-segmentation", International Conference on Image Processing, 2011 (ICIP '11)
Software: iScribble
We have made the graphical user interface developed by us available to the community. The above project uses the same GUI with some simple modifications. We feel that this GUI is a powerful yet simple tool which can be easily integrated with Matlab. Please feel free to use it and give your feedback.
You can download the interface from the iScribble project page.