A Sneak Peek Inside Four Silicon Valley Tech Labs - Popular Mechanics

NEWS
Feb 22, 2012

BY JOHN BRANDON Popular Mechanics Yahoo Research Lab Project: C.O.R.E. Visualization On the Internet, there have always been two common ways to find information: You search for what you want or you click on links that appear on a page you visit. But for years, tech giants have been trying to leave that old model behind by using what you search (and other information about you, which it knows because of cookies that track your Web activity) to offer you links that you're more likely to click—and thus keep you clicking and coming back. At Yahoo.com, the stories you see on the front page are heavily customized for your interests–if the site knows you look for football-related content, it'll show you more sports stories as opposed to news about Kim Kardashian. The technology has existed for about three years but now, Yahoo has decided to pull the curtain back on this technology. CORE visualization (CORE stands for Content Optimization and Relevance Engine) is an interface and set of algorithms that show which stories are popular within a given demographic. For example, you can see which articles are popular with, say, females over 55 or teens in New York City. The new interface came out of beta last week. Yahoo editors can easily see what stories are popular. What they want to know is which new stories have the potential to go viral. Raghu Ramakrishnan, a Yahoo chief scientist, told me that the CORE visualization project shows how the algorithms work. When I gave him the example of underwater basket weaving and asked whether a story about that could take off, he said the CORE engine could look for stories about the ocean or watersports or crafts and examine whether the story has merit. The algorithms borrow an idea from statistical modeling, namely the explore-and-exploit model where you gather minimal data, test it, and then keep analyzing that data to see if it is important. Ramakrishnan says the engine watches for initial clicks on stories, how much time a user spends reading, and whether he clicks away from that story quickly. Interestingly, only 1 percent of all visitors see the same group of popular stories. Otherwise, we are all seeing a different mix. What's most interesting about the CORE project, though, is that it points to how Yahoo is using complex algorithms to increase revenue. The more links you click, the more money they make from ads. That's important for the average Web visitor, because it means fresher content and less hunting and pecking for interesting articles. Of course, there is something disconcerting about a machine knowing you so well that every time you visit Yahoo.com you see a perfect blend of interesting content. Yet, we might as well get used to this algorithmic approach. To Ramakrishnan, it points to the future of journalism—using computer AI routines to deliver the most relevant articles for specific users based on location, age, gender, and other factors. (Whether Yahoo engineers can create an automated way to create those articles is still TBD.)