Advanced Multimedia Processing Lab -- Projects -- hSO

            About AMP Lab        Projects        Downloads        Publications        People        Links

Project - Hierarchical Semantics of Objects (hSO)




Team Member


Devi Parikh

Top of this page


Motivation and Goal

We introduce hSOs: Hierarchical Semantics of Objects. An hSO is learnt from a collection of images taken from a particular scene category.

The hSO captures the interactions between the objects that tend to co-occur in the scene, and hence are potentially semantically related. Such relationships are typically hierarchical. For example, in a collection of images taken in a living room scene, the TV, DVD player and coffee-table co-occur frequently. The TV and the DVD player are more closely related to each other than the coffee table, and this can be learnt from the fact that the two are located at similar relative locations across images, while the coffee table is somewhat arbitrarily placed. The goal of this work is to learn this hierarchy that characterizes the scene.

The proposed approach, being entirely unsupervised, can detect the parts of the images that belong to the foreground objects, cluster these parts to represent objects, and provide an understanding of the scene by hierarchically clustering these objects in a semantically meaningful way - all from a collection of unlabeled images of a particular scene category. In addition to providing the semantic layout of the scene, learnt hSOs can have several useful applications such as compact scene representation for scene category classification and providing context for enhanced object detection. [ICCV 2007 paper].

The proposed approach can also be used to address unsupervised identification of multiple foreground objects in a scene, which is a subset of the hSO learning problem. [ACCV 2007 paper].

Top of this page


Approach and Results

Further details can be found in the related publications below.

Top of this page



Top of this page



Any suggestions or comments are welcome. Please send them to Devi Parikh 

Top of this page

free hit counter