Key Scientific Challenges, Entry #3: Microeconomics and Social Systems On January 27 we announced the kick-off of our 2010 Key Scientific Challenges Program. To highlight the scientific challenge areas included in the program, we launched a series of guest blog posts earlier this month on Yodel Anecdotal. Read our previous post on privacy and security, “Data, Data Everywhere, but How to Keep it Safe.” Another big challenge are Yahoo’s research scientists are continually examining is microeconomics and social systems. In this entry, Sharad Goel from Yahoo Labs shares some thoughts on how Yahoo is tackling the new opportunities for research into the social sciences that the Web is making possible and why it’s a fascinating field. What do your friends really know about you? How much do they influence your decisions? How often do we stray from the cultural herd? How do groups organize to solve complex problems? Answers to such fundamental questions about social behavior have often eluded us. With microscopes we peered into the intangibly small building blocks of life, and with telescopes we found our place in an unimaginably expansive universe. But without the tools to faithfully document human activity—a challenge that by comparison seems so palpable—we had no way to investigate the inner workings of our own communities. Now with an explosion of information on every aspect of our everyday existence—from what we buy, to where we travel, to whom we know—we can measure what until quite recently was thought unmeasurable. In the Microeconomics and Social Systems Group at Yahoo Labs, we are using this proliferation of data to explore how societies function. It’s a fascinating area of study that is just beginning to shed light on new layers of human behavior, making it a perfect fit for the Key Scientific Challenges Program. In a recent study that’s garnered some attention, for example, we asked, “How eccentric are people?” Looking at consumer preferences across movies, music, and web browsing, we came to the surprising conclusion that ordinary people have pretty extraordinary tastes. In particular, we found that typical Netflix and Yahoo Music users regularly watch movies and listen to songs that are not even available in the largest brick-and-mortar retailers. This result not only challenges stereotypes about people blindly following the herd, but also highlights the importance of offering consumers broad selection. That is, specialty products may dramatically boost user satisfaction by providing buyers the convenience of “one-stop shopping” for both their mainstream and niche interests. In other work, we used web search queries to forecast the commercial success of movies, songs, and video games. Weeks, sometime even months, before a movie opens or a video game is released, one can find traces of pent up consumer demand in the search query logs. We found that these telltale signs of early interest are remarkably good predictors of future success. The catch? Although the search logs do reflect user intent, more mundane indicators, such as production budgets and reviewer ratings, perform equally well at forecasting sales. Thus, the benefit of web search as a prediction tool may have less to do with its superiority over other methods than with its generality, low cost, and real-time nature. At a time when we are drowning in data, at Yahoo Labs we’re asking a simple question: what can you do with it? The answer is limited only by our imaginations.

More information at:

http://ycorpblog.com/