Tech

Netflix has awarded a $1 million bounty to an international team of mathematicians and computer scientists known as BellKor's Pragmatic Chaos. The prize is for improving the algorithm Netflix uses to recommend movies to users based on their past preferences.

The competition gave computer scientists a dataset containing 100 million ratings given to nearly 20,000 films by half a million users, and challenged them to use the information to improve the accuracy of the firm's recommendations by 10 per cent.

The competition has been running since 2006, and entered the final stages in June this year when Netflix confirmed that BellKor's Pragmatic Chaos (BPC) had hit the target, triggering a month of endgame in which all teams could try to better the leaders' score.

The final result went down to the wire. Both BPC and a rival team called The Ensemble, hurriedly formed after the endgame started, submitted efforts in the closing days of the waiting period. The competition organisers say that both proved to be 10.06 per cent better than the benchmark.

Expensive wait

However, The Ensemble's entry came in 20 minutes later than Bellkor's. "That 20 minutes is a million dollars," said Reed Hastings, the Netflix CEO, as he announced the winner in New York earlier today.

Yehuda Koren, a member of the winning team, was quick to praise the competition. "If Kanye West was here he'd probably mention our competitors The Ensemble, with the all-time best performance on the leader board," he said, referring to a quirk of the competition that had the public Netflix Prize Leaderboard declaring The Ensemble to be leading with 10.1 per cent.

The mismatch happened because the public listing was calculated using a public dataset, while the official results use a secret and slightly different dataset to keep the contest fair.

Good loser

Ensemble team member Nicholas Ampazis at the University of the Aegean in Mytilene, Greece, remains upbeat. "Netflix gave us the opportunity to work with the largest ratings dataset ever available to the research community," he told New Scientist.

The dataset will provide wider benefits than just helping Netflix with its business. For example, Tom Tan and Serguei Netessine at the University of Pennsylvania in Philadelphia used it to find empirical evidence against the "long tail" theory, which predicts that online shops make it possible for cumulative sales of unpopular items to be greater than those of the few blockbusters that usually dominate sales.

The same study hinted at what Netflix stands to gain from the high-profile bounty prizes, in addition to publicity. Between 2004 and 2005 the number of products it offered rose by 20,000, but the number of ratings rose by just 2000 over the same period. A better recommendation system would help to quickly match all those unrated products with people who would enjoy them.

Second prize

But implementing BellKor's Pragmatic Chaos's winning entry might be tough, says Ampazis, because gaining a 10 per cent improvement required a blend of several algorithms, performing a large number of calculations in parallel – a process that takes a lot of time and computing power.

Neil Hunt, chief product officer at Netflix, conceded as much, saying that Netflix is currently using a selection of just two or three algorithms to achieve a 6 to 7 per cent improvement on its previous recommender.

Next challenge

Netflix today also launched a second $1 million competition, this time asking entrants to predict movies liked by people who have rated very few or even no titles.

The company will make available a new dataset containing information about Netflix users who have rated very few films. This will be coupled with demographic information, says Hunt, as well as some behavioural data such as the users' known viewing behaviour.

The team that achieves the best results with the data after six months will receive $500,000. The remaining $500,000 will be awarded to the team with the best performance after 18 months.

Journal reference: Tan and Netessine's paper (PDF) can be viewed here

If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.