Advanced Multimedia Processing Lab -- Projects -- Low Level Context

            About AMP Lab        Projects        Downloads        Publications        People        Links

Project - Low-level Contextual Patch Saliency

            Image           Discriminative Saliency  Contextual Saliency




Team Member

Devi Parikh

This work was done in collaboration with Larry Zitnick at Microsoft Research (Redmond)

Top of this page


Motivation and Goal

Context has traditionally been exploited for high level tasks such as object recognition, detection, etc. In this work, we explore the use of context for a low level task of determining patch saliency.

Traditionally, saliency has been based on interest-point detectors where a patch is considered to be salient if it has corner-like information; or based on discriminative measures where a patch is considered to be salient if it has information for a classification task. In this work, we consider a patch to be salient if it is predictive of the rest of the image i.e. if a patch is consistent with the context of the rest of the image.

We evaluate our saliency measure by comparing it to other existing saliency measures for the task of object categorization and scene recognition. We work within the bag-of-words paradigm, and use the different saliency maps to sample patches from an image to be used in the bag-of-words histogram. We explore three different sampling strategies that cover the spectrum from exploiting highly salient regions of an image to exploring different regions of the image.

We find that our saliency measure achieves higher recognition rates than some existing saliency measures. Moreover, using our saliency measure, we get higher accuracies using a sparse sampling of the image, while for all other saliency measures, a dense sampling of the image has higher accuracies.

Top of this page


Approach and Results

Further details can be found in our ECCV 2008 paper.

This video demonstrates the evolution of the saliency map when using the sequential sampling strategy.

Top of this page



Top of this page



Any suggestions or comments are welcome. Please send them to Devi Parikh 

Top of this page

free hit counter