Delving Deep into Personal Photo and Video Search

Feb 13, 2017

The ubiquity of mobile devices and cloud services has led to an unprecedented growth of online personal photo and video collections. Due to the scarcity of personal media search log data, research to date has mainly focused on searching im- ages and videos on the web. However, in order to manage the exploding amount of personal photos and videos, we raise a fundamental question: what are the differences and similarities when users search their own photos versus the photos on the web? To the best of our knowledge, this paper is the first to study personal media search using large-scale real-world search logs. We analyze different types of search sessions mined from Flickr search logs and discover a num- ber of interesting characteristics of personal media search in terms of information needs and click behaviors. The insight- ful observations will not only be instrumental in guiding fu- ture personal media search methods, but also benefit related tasks such as personal photo browsing and recommendation. Our findings suggest there is a significant gap between per- sonal queries and automatically detected concepts, which is responsible for the low accuracy of many personal media search queries. To bridge the gap, we propose the deep query understanding model to learn a mapping from the personal queries to the concepts in the clicked photos. Experimental results verify the efficacy of the proposed method in im- proving personal media search, where the proposed method consistently outperforms baseline methods.

  • International Conference on Web Search and Data Mining (WSDM 2017)
  • Conference/Workshop Paper