Five and a half years ago, Netflix offered data and a $1 million prize to improve their recommendation system by at least ten percent. In 2009, a statistics team at AT&T Labs, BellKor, did that. Unfortunately, Netflix never integrated the algorithm into production.
If you followed the Prize competition, you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later. This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line. We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment. Also, our focus on improving Netflix personalization had shifted to the next level by then.
That’s too bad. Netflix knows their business better than anyone, but I sure wish Keeping Up with the Kardashians wasn’t listed in my top 10 right now.
[via Techdirt]
Did this improved algorithm get incorporated into Jinni.com? Jinni is a beta site (by their own label), and you can link your Jinni recommendations to Netflix, so they may be affiliated. The recommendation engine at Jinni is reviewed much more favorably (and my own personal experience would corroborate this) than the recommendation engine on Netflix.
If you don’t want Keeping Up with the Kardashians in your top 10, maybe you shouldn’t watch it so often…. :-p
I CAN WATCH WHAT I WANT TO.
The blog post points out that the algorithms from the first year of the competition were indeed used, and are still a key part of their recommender. The last year or so of the competition was focused on combining models and teams. The Grand Prize winning solution contained over 800 models from 4 teams – it was engineered to win a contest, and was not surprisingly suitable for a production system.
Similarly, Netflix learned that the best recommender algorithm will be a blend of many individual models, even if the specific Grand Prize solution was not useful to them, the learnings from the competition were extremely valuable.