Multi-View 3D Reconstruction for Scenes under the Refractive Plane with Known Vertical Direction
People
Yao-Jen Chang and Tsuhan Chen
3D reconstruction for a scene under water. The color in the scene reconstruction encodes the height of a point from the lowest (red) to the highest (blue). Notice the curved reconstructions of the bottom plane obtained by using no distortion model and radial distortion model. By incorporating the proposed refractive distortion model, all surfaces are correctly reconstructed.
Abstract
Images taken from scenes under water suffer distortion due to refraction. While refraction causes magnification with mild distortion on the observed images, severe distortions in geometry reconstruction would be resulted if the refractive distortion is not properly handled. Different from the radial distortion model, the refractive distortion depends on the scene depth seen from each light ray as well as the camera pose relative to the refractive surface. Therefore, it's crucial to obtain a good estimate of scene depth, camera pose and optical center to alleviate the impact of refractive distortion. In this work, we formulate the forward and back projections of light rays involving a refractive plane for the perspective camera model by explicitly modeling refractive distortion as a function of depth. Furthermore, for cameras with an inertial measurement unit (IMU), we show that a linear solution to the relative pose and a closed-form solution to the absolute pose can be derived with known camera vertical directions. We incorporate our formulations with the general structure from motion framework followed by the patch-based multiview stereo algorithm to obtain a 3D reconstruction of the scene. We show through experiments that the explicit modeling of depth-dependent refractive distortion physically leads to more accurate scene reconstructions.
Video Summary
Publication
Yao-Jen Chang and Tsuhan Chen. Multi-View 3D Reconstruction for Scenes under the Refractive Plane with Known Vertical Direction. IEEE International Conference on Computer Vision (ICCV), 2011. [Paper][Poster]