The Netflix Prize & Aggregate Knowledge

I have been learning a bit about the interesting statistical & programming challenges involved in the creation of recommendations, like those produced by Amazon regarding what sort of books, CDs and other products a user may be interested in based on what that user has purchased in the past as well as what other users who have purchased similar items has liked, etc. Apparently these are some of the priciest and most difficult algorithms to write and they are extremely valuable when they work.

Like Amazon, Netflix offers recommendations to users. They have their own algorithm called Cinematch, which, according to Wikipedia, produces recommendations that are a 9.6% improvement over a straight average of user ratings. The Netflix Prize is contest the company has organized that will give $1,000,000 to the creators of an algorithm that offers a 10% improvement over the 9.6% improvement that Cinematch can already attain. The current standings can be seen here. The current leaders have reached a 7.1% improvement. Apparently Cinematch itself was improved on in 6 days.

Aggregate Knowledge is a Kleiner backed start-up trying to commercialize algorithms like these. Pretty interesting stuff.