Following your gut, following the data

December 26, 2018


Statistics  /  , ,

The Wall Street Journal highlighted a disagreement between data and business at Netflix. Ultimately, the business side “won.” However, maybe that’s the wrong framing. Roger Peng describes the differences between analysis and the full truth:

There’s no evidence in the reporting that the content team didn’t believe the data or the analysis. It’s just that their fear of damaging a relationship with an actor overruled whatever desire they might have had to maximize clicks or views. The logic was probably along the lines of “We may take a hit in the short-run but we will benefit from this relationship in the long-run.” Whether that’s true or not is unclear, but it’s a tricky question to answer with data. It’s not even clear to me how you would formulate that question.

Data often pitches itself as the path to definitive answers, but most of the time it gives you possibilities and weighted suggestions. Follow blindly, and you end up with creepy, algorithmically-generated YouTube videos.