Graduate student makes up data for fake findings

Posted to Mistaken Data  |  Tags: ,  |  Nathan Yau

Last month, This American Life ran a story about research that asked if you could change people’s mind about issues like same-sex marriage and abortion — with just a 22-minute conversation. The research was published in Science, but Donald Green asked the publication to retract the paper recently. It seems his co-author and UCLA graduate student, Michael LaCour, made up a lot of data.

Green today told me if there was no survey data, what’s incredible is that LaCour produced all sorts of conclusions and evaluations of data that didn’t exist. For instance, he had “a finding comparing what people said at the door to canvassers to what they said on the survey,” according to Green. “This is the thing I want to convey somehow. There was an incredible mountain of fabrications with the most baroque and ornate ornamentation. There were stories, there were anecdotes, my dropbox is filled with graphs and charts, you’d think no one would do this except to explore a very real data set.”

“All that effort that went in to confecting the data, you could’ve gotten the data,” says Green.

Bizarre.

Stanford and Berkeley researchers found that something seemed off when they tried to reproduce the results. Their full report is available for reading, which includes the data in question and R code that explains their reasoning.

Perhaps one should note that the dataset is not definitely a fake. Noted by the examiners:

No one of the irregularities we report alone constitutes definitive evidence that the data were not collected as described. However, the accumulation of many such irregularities, together with a clear alternative explanation that fits the data well, leads us to be skeptical the data were collected as described.

And it was enough for Green to request a retraction, so there’s that.

With this in mind, it’s especially interesting to visit LaCour’s website, which includes a lot of graphs from the original study. Graphs, which typically lend a feeling of concreteness to a subject — especially scientific research — almost have an opposite effect.

Survey effects

LaCour is currently “gathering evidence” to provide a comprehensive rebuttal.

Update: More details from This American Life. It gets weirder.

Favorites

Unemployment in America, Mapped Over Time

Watch the regional changes across the country from 1990 to 2016.

Years You Have Left to Live, Probably

The individual data points of life are much less predictable than the average. Here’s a simulation that shows you how much time is left on the clock.

Graphical perception – learn the fundamentals first

Before you dive into the advanced stuff – like just about everything in your life – you have to learn the fundamentals before you know when you can break the rules.

Real Chart Rules to Follow

There are rules—usually for specific chart types meant to be read in a specific way—that you shouldn’t break. When they are, everyone loses. This is that small handful.