iCoseg: Interactive cosegmentation by touch
People
Dhruv Batra, Adarsh Kowdle, Devi Parikh, Kevin Tang, Yu-Wei Chao, Jiebo Luo, Tsuhan Chen
Abstract
The advent of commercial touch-screen devices, like the HP Touchsmart™, the Microsoft Surface™ or the Apple iPhone™, has opened new avenues for human-computer interactions. In this demo, we present one such interactive application – Interactive Cosegmentation of a group of related images. We develop a user-friendly system on the HP Touchsmart™, which enables a user to cut out objects of interest from a collection of images by providing scribbles on a few images.
We present an algorithm for Interactive Cosegmentation of a foreground object from a group of related images. While previous approaches focus on unsupervised co-segmentation, we use successful ideas from the objectcutout literature. We develop an algorithm that allows users to decide what foreground is, and then guide the output of the co-segmentation algorithm towards it via scribbles. Interestingly, keeping a user in the loop leads to simpler and highly parallelizable energy functions, allowing us to work with significantly more images per group. However, unlike the interactive single image counterpart, a user can not be expected to exhaustively examine all (tens of) cutouts returned by the system to make corrections. Hence, we propose iCoseg, an automatic recommendation system that intelligently recommends where the user should scribble next. We introduce and make publicly available the largest cosegmentation dataset with 38 groups, 643 images, and pixelwise groundtruth annotations. Through machine experiments and real user studies with our developed interface, we show that iCoseg can intelligently recommend regions to scribble on, and users following these recommendations can achieve good quality cutouts with significantly lower time and effort than exhaustively examining all cutouts.
Publications
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance", International Journal of Computer Vision (IJCV), To appear.
[Springer Page | Preprint pdf]
- 03/2011: Among top downloaded articles from IJCV in the last 30 days.
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "iCoseg: Interactive Co-segmentation with Intelligent Scribble Guidance", Computer Vision and Pattern Recognition, 2010 (CVPR '10). [pdf]
- Dhruv Batra, Devi Parikh, Adarsh Kowdle, Tsuhan Chen, Jiebo Luo. Seed Image Selection in Interactive Cosegmentation. International Conference on Image Processing 2009 (ICIP '09). [pdf]
- Adarsh Kowdle, Dhruv Batra, Wen-Chao Chen and Tsuhan Chen. "iModel: Interactive Co-segmentation for Object of Interest 3D Modeling ", Workshop on Reconstruction and Modeling of Large-Scale 3D Virtual Environments, European Conference on Computer Vision, 2010 (ECCV '10). [pdf] [Project Page]
Demos
- Jason Lew, Adarsh Kowdle, Dhruv Batra, Tsuhan Chen. "iPhone application for iCoseg", Demo session at ECE day 2010, Cornell University. [poster]
- Dhruv Batra, Adarsh Kowdle, Kevin Tang, Devi Parikh, Jiebo Luo, Tsuhan Chen. "Interactive Cosegmentation by Touch", Demo session at Computer Vision and Pattern Recognition, 2009 (CVPR '09)
Video presentation describing the system
Dataset
We introduce the largest publicly available co-segmentation dataset of 38 groups (643 images), along with pixel ground-truth hand annotations. CMU-Cornell iCoseg dataset.
The image below gives a glimpse of the image groups in the dataset separated into 4 main categories. You can choose to browse through the images and the ground-truth hand annotations here before downloading the dataset.
If you use the 'CMU-Cornell iCoseg dataset' in your research, please cite the following paper,
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance", International Journal of Computer Vision (IJCV), To appear.
- Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jeibo Luo and Tsuhan Chen. "iCoseg: Interactive Co-segmentation with Intelligent Scribble Guidance", Computer Vision and Pattern Recognition, 2010.
Please inform us about your accuracies on this dataset, so that we may make them publicly available for easy comparison and citation.
Acknowledgement
The authors would like to acknowledge and thank the Flickr users whose images are part of this dataset. The usernames are listed below in no particular order.
- cmlburnett
- JB Imaging
- rjt208
- Sou'wester
- phes999
- auTomoTiVe FreAk
- LeszekZadlo
- Mhln
- richardavis
- timd2000
- TAIwiffic
- ocreationsphotography
- smata2
- jadepike4
- kitekid
- rreed3
- Rich Moffitt
- albertosepe
- Dustin Hall
- Rolandito
- Tracy Sasser
- EricaJoy
- Leonardo Paris
- Guido Jansen
- micisi
- John Alarcon
- Vincent
- SanSuiSo
Software: iScribble
We have made the graphical user interface developed by us available to the community. We feel that this GUI is a powerful yet simple tool which can be easily integrated with Matlab. Please feel free to use it and give your feedback.
You can download the interface from the iScribble project page.
Sponsors
|
|
|
