Originally posted on December 3, 2009 on the Yahoo Search Blog Earlier this year Yahoo welcomed Yoelle Maarek as our new senior director of Yahoo Research. Prior to joining Yahoo, Yoelle was the Director of Google Haifa Engineering Center, which she opened in July 2006. For more than 20 years, Yoelle has been helping dig into search problems. She talks with the Yahoo Search Blog about new developments in search and challenges in this field. Yahoo Search Blog: Tell us a bit about your research background – what are your main topics of interest? Yoelle Maarek: My research background is core information retrieval, the computer science discipline behind search. I got my PhD in this domain more than 20 years ago, and published my first SIGIR paper in 1989 — way before the Web existed as we know it. At that time, our test collections counted about 300 documents with associated relevance judgments. It’s crazy to think how far we‘ve come. Besides search, I am interested in most Web technologies, with a special taste for user-facing applications. I like to make people wonder what kind of smart algorithms and powerful backend systems were developed to make things work. . I love demo-able applications, anything that makes the user happier and creates either a “wow” effect or significantly simplifies the user’s life on the Web. What are the main future challenges in Search? The challenges are to always make systems more user-friendly, more relevant, and faster. We need to guess what users want even before they know it themselves. I am a strong believer in leveraging larger and larger data sets, and personalizing more and more. We are far from having reached the full potential of technology here, one reason being the fact that our favorite tools and applications do not share enough data. Even more problematic is the privacy issue. We need our users to trust us before we can use their data as we wished. It is probably both a technical and society/cultural challenge, which makes it even more interesting. What are some exciting developments you are seeing in innovating the search experience? I think the search box could be the “next frontier” in search – I am referring to the point I made a bit earlier about “guessing” what users want. The major search engines have started to add query assistance and completion abilities to their search box, as with Yahoo Search Assist, Google Suggest, and even recently by Bing. I believe that these tools are only a first step and that they open the doors to a great deal of innovation. They establish a dialog with users even before users are done formulating their informational or navigational needs. As such, they can influence, facilitate, and direct the users in ways we had not imagined until now. On your Web site, you write “I believe in search and statistics not in NLP.” But some of the developments you mentioned above, like Search Assist, uses Natural Language Processing technologies. What’s wrong with Natural Language Processing? I was only joking. Okay, let’s say half-joking. I like NLP when it is heavily inspired by computational linguistics, where the important word here is “computational.” What I don’t like is a certain old school of NLP that pretends to really understand language and uses heavy semantic networks to encode one vision of the world. It is probably because I don’t think that anyone (human or machine) should define the order of the world. When we were studying the topic 20 years ago, we had to build these monster semantic networks manually. So let’s say that I don’t believe in old fashioned manual NLP, but I am a great believer in NLP systems that do everything automatically. You’ve said that search technology can have social networking effects. Can you explain that a bit? We all know that personalization is a key factor in improving search. However, most have explored personalization for a given individual, which can endanger privacy. My colleague Ricardo Baeza-Yates often says that a more intriguing direction is to consider personalization over intent. Indeed, individuals have various facets and interests in their taste, and we should try to personalize around these facets – around common intents over large populations this should bring more insight and allow us to escape stereotypes. As a woman who likes comedy movies, science, and heroic fantasy literature, as well as my local soccer team, I believe that I have heterogeneous tastes and I would hate not getting relevant soccer information simply because the majority of the Haifa soccer team fans are men, or don’t like science, or … you get the point. So, we should be able to discover implicit social relationships over these common intents. What brought you to Yahoo? Mostly, I was drawn by the chance to work with the top research talent. The research scientists at Yahoo simply dominate the research publication world and it is impressive to see the quality and quantity of Yahoo publications in these forums. I find that Yahoo researchers are not only leading the way but also sharing their results with the community so as to encourage the next generation of thinkers. Yahoo is the only company in that space that is brave enough to do this rather than adopting a paranoid approach. This open approach to research is smart, and it will benefit the company in the long term, but you need vision to understand this. In addition, these research scientists are the most humble, modest, and fun people around. There’s not one trace of arrogance, which is really refreshing. Finally, in addition to the quality of the research people, I see that business-wise Yahoo is ready to take risks and be a game changer so as to take the first spot in all properties. This is the time to progress aggressively and win over market share when others are only protecting their positions rather than moving forward. Now that you’ve been around for a little while, what’s the best part of being a Yahoo? I like the people, the brains, the openness, and the potential to deliver useful content to so many users in so many different properties. For me, my main priority right now is building a world-class team of research scientists. We have been interviewing a lot, extended a few offers and will have our first new hire join soon. In terms of technical directions, we will still focus on search user experience, which is the forte of the team in Barcelona (with their contributions to SearchPad and Search Assist, and their seminal research in query flow graphs). I am also looking together with Yehuda Koren, who is my first report in Haifa and preceded me here, at new directions for research, as we want to develop an additional area of competency for Haifa. This is being defined as I speak and will be strongly influenced by our first hires as we want this area to be driven by them. We will hopefully have more details in the next few weeks. - Jessica Hilberman Yahoo Search Blog

More information at:

http://www.ysearchblog.com/