Lessons in statistical significance, uncertainty, and their role in science

Aug 20, 2015

Science is hard. Statistics is hard. Proving cause and effect is hard. Christie Aschwanden for FiveThirtyEight, with graphics by Ritchie King, discusses the uncertainty in data and the challenge of answering seemingly straightforward questions via the scientific method.

Leading the article is a description of p-hacking. Mess around with variables enough, and you too can get a p-value low enough to publish results in a distinguished journal.

A fine interactive lets you try this yourself, showing that the political party in office affects the economy. The funny part is that you can easily “prove” that both parties are good for the economy.

Which political party is best for the economy seems like a pretty straightforward question. But as you saw, it’s much easier to get a result than it is to get an answer. The variables in the data sets you used to test your hypothesis had 1,800 possible combinations. Of these, 1,078 yielded a publishable p-value, but that doesn’t mean they showed that which party was in office had a strong effect on the economy. Most of them didn’t.

The p-value reveals almost nothing about the strength of the evidence, yet a p-value of 0.05 has become the ticket to get into many journals. “The dominant method used [to evaluate evidence] is the p-value,” said Michael Evans, a statistician at the University of Toronto, “and the p-value is well known not to work very well.”

I guess that means we have to think more like a statistician and less like a brainless, hypothesis-testing robot.

Worth the full read.

Favorites

The Best Data Visualization Projects of 2014

It’s always tough to pick my favorite visualization projects. Nevertheless, I gave it a go.

Shifting Incomes for American Jobs

For various occupations, the difference between the person who makes the most and the one who makes the least can be significant.

Think Like a Statistician – Without the Math

I call myself a statistician, because, well, I’m a statistics graduate student. However, the most important things I’ve learned are less formal, but have proven extremely useful when working/playing with data.

10 Best Data Visualization Projects of 2015

These are my picks for the best of 2015. As usual, they could easily appear in a different order on a different day, and there are projects not on the list that were also excellent.