Crowdsourcing the N.C.A.A. Tournament

NEWS
Mar 18, 2010

Originally Published on nytimes.com on March 17 By Joshua Brustein

There are 9.2 quintillion possible outcomes to the N.C.A.A. basketball tournament.

Yahoo has built a new tool for this year’s tournament intended to determine which of these results are most likely.

Called Predictalot, it is a prediction market that allows users to bet on any outcome. It then continuously regenerates the odds of each outcome based on people’s bets.

Using prediction markets to guess the likelihood of future events is not new; solutions to various problems relying on the wisdom of crowds are increasingly common as better computing tools are developed to refine the predictions. Sporting events are seemingly good targets for experimentation, with clear results and many people who are willing to take a guess. Yahoo hopes to develop further markets based on Predictalot that will go beyond sports.

But how good is the crowd at picking N.C.A.A. winners?

Probably the most basic crowdsourcing strategy for the N.C.A.A. Tournament would be to copy ESPN’s national bracket, the aggregate of each individual bracket filled out through the Web site. In this bracket, all the people who make their predictions based on where they went to school or whose mascot is the cutest or how much they hate Duke cancel one another out, giving a consensus prediction for the tournament.

Here, the wisdom of the crowd is pretty conventional. As of Wednesday morning, ESPN’s bracket predicts that there will be only five games in the entire tournament where a lower-seeded team beats a higher-seeded team. In the end, Kansas, the top seed in the country, is expected to win.

What looking at the national bracket illustrates is that, on average, people pick winners by looking at the little numbers next to the teams, and then picking the team with the lower number. This makes sense, since the people who assigned those numbers to those teams know a lot about college basketball.

Yahoo generated its own bracket for The New York Times, based on its prediction market. The bracket is a little more surprising than the national market. It also picks Kansas to win, and includes few first-round upsets. But it strays from the seeding in the later rounds. It predicts, for instance, that two of the top-seeded teams — Duke and Syracuse — will be eliminated before reaching the Final Four.

David Pennock, the principal research scientist at Yahoo Research, said in an interview that the company’s approach allowed for more refinement than simply seeing which team was picked by the most people. Players do not have to complete brackets, they simply bet on the outcomes of certain questions. If someone knows only about Maryland, he can bet only on the outcomes of games involving Maryland. Prediction markets also weigh possible outcomes based on how strongly someone believes in a certain outcome. Yahoo’s bracket allows players to pick how many points they bet on a certain outcome; a 1,000 point bet will affect the market much more than a 10 point bet will.

But deciding that the top-ranked teams are most likely to win is only part of the best brackets. Here is the limit of Predictalot and other crowdsourced solutions.

“It’s trying to predict the most likely outcome, it’s not strategizing for you,” said Mr. Pennock. “If you are actually playing, you’re trying to predict the best bracket that no one else has thought of.”

The crowdsourced brackets, in other words, won’t lose the pools, but they won’t win them, either. Since most office pools do not reward anyone but the top performers, followers of the crowd will still be out $10.

So how to win?

On the Playbook Blog at Wired, Mark McClusky compares experts’ brackets with the wisdom of the crowd. The picks in which the experts and the crowd are furthest apart, he writes, are the best targets to take a chance. A less-refined strategy relying only on the crowdsourced bracket would be to pick upsets on the games where there is more disagreement. On ESPN’s bracket, for instance, 85.5 percent of people think Kansas will beat Ohio State if they meet, but only 66.9 percent feel Syracuse will beat Kansas State. So Kansas State is presumably a safer upset pick than Ohio State to reach the Final Four.

It is almost certain that winning brackets will be those that pick the unexpected, and the wisdom of the crowd can only tell you what’s expected. You can’t beat the crowd if you always follow it.

More information at:

  • http://bits.blogs.nytimes.com/2010/03/17/crowdsourcing-the-n-c-a-a-tournament/?src=twt&twt=nytimesbits