Representatives from academia and industry gathered in Geneva, Switzerland during the week of July 19th to attend the 33rd annual conference of the ACM Special Interest Group on Information Retrieval (SIGIR 2010). Yahoo continued its unrivaled presence with 15 accepted papers, 5 accepted posters, a series of workshops, tutorials, and a long list of program committee members.
Microsoft Live Labs director Gary Flake gave a keynote on Tuesday morning entitled “Zoomable UIs, Information Retrieval, and the Uncanny Valley.” Flake’s premise is that information retrieval was stuck in the “uncanny valley
, a metaphor he borrowed from the robotics community. Flake offered Grokker (search engine of the past) as an example of a search interface that emphasized visual clustering and got stuck in the uncanny valley. He called it “the sexiest search experience that no one was going to use.” Flake then proposed that moving beyond the uncanny valley would require making the user search experience more of a continuous, fluid interaction rather than simple browsing by taking raw data and combining it with metadata for faceted navigation. He offered some demos, emphasizing Microsoft’s recently released Pivot client, that he felt provided a vision to overcome the uncanny valley.
Yahoo Labs’ Carlos Castillo presented an industry track entitled “When No Clicks are Good News,” largely attended by representatives from search engines and social media companies – signaling a strong interest in novel real-world problems and datasets.
Paper themes varied from analysis of post-click user behavior to context-aware and personalized ranking to measurement and label collection methods. There was a strong emphasis on optimizing for the right metric – beyond simple relevance or click (essentially, what happens after the click). Conversations during breaks echoed a variety of challenges in the information retrieval research community, including the underrepresentation of real-time search at the conference. There were a number of papers and posters where toolbar data was utilized to understand the user’s behavior and potentially evaluate the goal/success of a user session, or assess the utility of the pages visited.
Yahoo researchers Ingmar Weber and Carlos Castillo’s paper “Demographics of Web Search,” was featured in an article in New Scientist
. The paper addressed questions such as who is the searcher in real life and how does search behavior depend on who you are. These are questions that have been endlessly modeled by search engines with the aim of personalizing the user experience as well as for potentially maximizing advertiser revenue with better targeting.
The session on “Relevance and Ranking in Online Dating Systems” by Yahoo’s Fernando Diaz, Sihem Amer-Yahia, and Donald Metzler had a standing-room-only crowd and generated a lot of feedback on Twitter. The paper formulated a two-way relevance model for matchmaking systems but unfortunately found that it did no better than query-independent ranking in the context of a production personals system.
Yahoo researcher Eren Manavoglu presented "Temporal Click Models" in the Link Analysis and Advertising Session. Her team presented a new user click model that can utilize the sequence of clicks in order to learn ad externalities as well as the position bias. Results showed that the sequence information indeed improves the accuracy of click estimation, especially for low and mid range frequency queries.
Another interesting area of research was eye- and mouse-tracking studies. One paper analyzed the impact of the relevance of ads on the search user’s eye tracking behavior. The authors conducted controlled experiments to see how the user’s look at the page when the ads shown were random, relevant or irrelevant.
Overall attendees felt like it was a great conference with a diverse range of topics and are looking forward to the conference in Beijing next year.