 |
About AMP Lab Projects Downloads Publications People Links
Project - Hierarchical Semantics of Objects (hSO)
Top of this page
| Motivation and Goal |
We introduce hSOs: Hierarchical Semantics of Objects. An hSO is learnt from a collection of images taken from a particular scene category.
The hSO captures the interactions between the objects that tend to co-occur in the scene, and hence are potentially semantically related. Such relationships are typically hierarchical. For example, in a collection of images taken in a living room scene, the TV, DVD player and coffee-table co-occur frequently. The TV and the DVD player are more closely related to each other than the coffee table, and this can be learnt from the fact that the two are located at similar relative locations across images, while the coffee table is somewhat arbitrarily placed. The goal of this work is to learn this hierarchy that characterizes the scene.
The proposed approach, being entirely unsupervised, can detect the parts of the images that belong to the foreground objects, cluster these parts to represent objects, and provide an understanding of the scene by hierarchically clustering these objects in a semantically meaningful way - all from a collection of unlabeled images of a particular scene category. In addition to providing the semantic layout of the scene, learnt hSOs can have several useful applications such as compact scene representation for scene category classification and providing context for enhanced object detection. [ICCV 2007 paper].
The proposed approach can also be used to address unsupervised identification of multiple foreground objects in a scene, which is a subset of the hSO learning problem. [ACCV 2007 paper].
Further details can be found in the related publications below.
-
D. Parikh, and T. Chen. Unsupervised Modeling of Objects and their Hierarchical Contextual Interactions. EURASIP Journal on Image and Video Processing, Special Issue on Patches in Vision, 2008.
-
D. Parikh, and T. Chen. Unsupervised Identification of Multiple Objects of Interest from Multiple Images: dISCOVER, Asian Conference in Computer Vision (ACCV), 2007. [poster presentation]
-
D. Parikh, and T. Chen. Hierarchical Semantics of Objects (hSOs), IEEE International Conference in Computer Vision (ICCV), 2007 [poster presentation].
-
D.Parikh, and T. Chen. Unsupervised Learning of Hierarchical Semantics of Objects (hSOs). Beyond Patches Workshop, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007. (Best Paper Award) [oral presentation]
Any suggestions or comments are welcome. Please send them to Devi Parikh