A team from Imperial College found that in 2009-10, nearly 20,000 adults were coded as having attended paediatric outpatient services, and 3,000 patients under 19 were apparently treated in geriatric clinics. Even more striking, between 15,000 and 20,000 men have been admitted to obstetric wards each year since 2003, and almost 10,000 to gynaecology wards.

It's hard to put your faith in analysis, visualization, policy, and anything else that comes out of data with reports like these. With human error being a known issue, we have to find better ways of inputting and double-checking data. Unfortunate mistakes at the outset only lead to bigger problems down the line.

The Appropriations Bill eliminates the Economic Census, which measures the health of our economy. It terminates the American Community Survey, which produces the social and demographic information that monitors the impact of economic trends on communities throughout the country. It halts crucial development of ways to save money on the next decennial census. In the last three years the Census Bureau has reacted to budget and technological challenges by mounting aggressive operational efficiency programs to make these key statistical cornerstones of the country more cost efficient. Eliminating them halts all the progress to build 21st century statistical tools through those innovations. This bill thus devastates the nation’s statistical information about the status of the economy and the larger society.

A lot of the negative comments following the post are from people who have never used Census data, or any substantial amount of data for that matter, and have no clue how a dataset can feed into a model to make other estimates. Then there's the people who don't want to answer questions about their toilets. I wonder what their Facebook profiles look like.

Princeton history graduate student Benjamin Schmidt explores changes in language through TV anachronisms. In Schmidt's most recent analysis, he examines Megan's use of "callback" in the last episode of Mad Men. Above is the ratio of modern use to period use. Notice callback sticking out in the top left.

The big one from the charts: Megan gets "a callback for" an audition. This is, the data says, a candidate for the worst anachronism of the season. The word "callback" is about 100x more common by the 1990s, and "callback for" is even worse. The OED doesn't have any examples of a theater-oriented use of "callback" until the 1970s; although I bet one could find some examples somewhere earlier in the New York theater scene, that may not save it. It wouldn't really suite Megan's generally dilettantish attitude towards the theater, or the office staff's lack of knowledge of it, for them to be so au courant. "call-back" and "call back" don't seem much more likely.

Other anachronisms include the use of "pay phone" and a frequent use of "on the phone with" which didn't peak until the 1970s.

Don't miss the look into Downton Abbey anachronisms. Also, more details from Schmidt on his methodology.

[via Revolutions]

Besides the direct estimates from the ACS itself, the Census Bureau uses ACS data as the backbone of several other programs. For example, the Small Area Income and Poverty Estimates program provides annual data to the Department of Education for use in allocating funds to school districts, based on local counts and rates of children in poverty. Without the ACS we would be limited to using smaller surveys (and thus less accurate information about poverty in each school district) or older data (which can become outdated within a few years, such as during the recent recession). Either way, it would hurt our ability to allocate resources fairly to schoolchildren nationwide.

Similarly, the Census Bureau uses the ACS to produce other timely small-area estimates required by Congressional legislation or requested by other agencies: the number of people with health insurance, people with disabilities, minority language speakers, etc. The legislation requires a data source like the ACS not only so that it can be carried out well, but also so its progress can be monitored.

Republicans, acknowledging its usefulness, attacked the survey as an unconstitutional invasion of privacy, arguing that the government has no business knowing how many flush toilets someone has, for instance.

"It would seem that these questions hardly fit the scope of what was intended or required by the Constitution," said Rep. Daniel Webster (R-Fla.), author of the amendment.

"This survey is inappropriate for taxpayer dollars," Webster added. "It's the definition of a breach of personal privacy. It's the picture of what's wrong in Washington, D.C. It's unconstitutional."

The ACS is the picture of what's wrong in Washington? This is idiocy.

Related

• Why the American Community Survey is worth keeping
• Fly through a survey of the universe
• In pursuit of the American dream (house)

For over a decade, CNN (Cable News Network) has been providing transcripts of shows, events and newscasts from its broadcasts. The archive has been maintained and the text transcripts have been dependably available at transcripts.cnn.com. This is a just-in-case grab of the years of transcripts for later study and historical research.

Changes in news coverage and CNN's focus over the years, anyone?

[via @A_L]

Related

• 10 Largest Data Breaches Since 2000 – Millions Affected
• Transcript Analyzer for Republican Debate
• Afghanistan war logs revealed and mapped

Representativeness:

People estimate the likelihood of a sample based on how closely it resembles the population.

You can't always judge how likely or improbable a sample is based on how it compares to a known population. For example, let's say you flip a coin four times and get four tails in a row (TTTT). Then you flip four more times and get HTHT. In the long run, heads and tails are going to be split 50/50, but that doesn't mean the second sequence is more likely.

Similarly, a sequence of ten heads in a row isn't the same as getting a million heads in a row.

Gambler's fallacy:

Use of the representative heuristic leads to the view that chance is a self-correcting process.

The history boards at roulette tables mean nothing. They're just for show. Just because a red hasn't come up in a while doesn't mean the roulette wheel is due for a red soon. Each spin is independent of the spins that came before it.

Base-rate fallacy:

People ignore the relative sizes of population subgroups when judging the likelihood of contingent
events involving the subgroups.

You have to consider the base population for comparison. Maybe a company is comprised of 80 percent men and 20 percent women. If your base is the US population, you might consider that inequality, but what if the applicant breakdown was 90 percent men and 10 percent women? In the latter case, a higher percentage of women than men were actually hired.

Availability:

Strength of association is used as a basis for judging how likely an event will occur.

Just because some percentage of your friends are designers doesn't mean that the same percentage of people are designers elsewhere (obviously). Or the example that Garfield uses: a ten percent divorce rate among people you know isn't necessarily the same nationwide or globally.

Conjunction fallacy:

The conjunction of two correlated events is judged to be more likely than either of the events themselves.

The common example from Tversky and Kahneman:

"Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations." A group of people were asked if it was more probable that Linda was a bank teller or a bank teller active in the feminist movement (a sign of the times this poll was taken).

Eighty-five percent of respondents chose the latter, but the probability of two things happening together is always less than or equal to the events occurring individually.

Notice that there's still not much math involved in these examples. It's logic that plays into thinking like a statistician without the math (with statistical foundations). You can get a lot done just by thinking critically about your data.

Related

• Strata of common and not so common colors
• The Sexperience 1000 shows a (statistical) view of what goes on in the bedroom
• Most common iPhone passcodes

What does Rosling make of his statistical analysis of worldwide trends? "I am not an optimist," he says. "I'm a very serious possibilist. It's a new category where we take emotion apart and we just work analytically with the world." We can all, Rosling thinks, become healthy and wealthy. What a promising thought, so eloquently rendered with data.

[Thanks, wife]

Related

• The Joy of Stats with Hans Rosling
• Hans Rosling: Providing Data, Inspiring Change
• Gapminder makes its way to the desktop

If you followed the Prize competition, you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later. This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line. We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment. Also, our focus on improving Netflix personalization had shifted to the next level by then.

That's too bad. Netflix knows their business better than anyone, but I sure wish Keeping Up with the Kardashians wasn't listed in my top 10 right now.

[via Techdirt]

Related

• Netflix Prize Dataset Visualization
• Netflix favorites by location
• The Geography of Netflix Rentals

Before I could actually do any of that I was moved to a highly secret experimental station in the south of England. At the time they were bombing London every night and our job was to help to find out what to do if, one night, they used poisonous gas.

Some of England's best scientists were there. There were a lot of experiments with small animals, I was a lab assistant making biochemical determinations, my boss was a professor of physiology dressed up as a colonel, and I was dressed up as a staff sergeant.

The results I was getting were very variable and I told my colonel that what we really needed was a statistician.

He said "we can't get one, what do you know about it?" I said "Nothing, I once tried to read a book about it by someone called R. A. Fisher but I didn't understand it". He said "You've read the book so you better do it", so I said, "Yes sir".

Box eventually worked with Fischer, studied under E. S. Pearson in college after his discharge from the army, and started the Statistical Techniques Research Group at Princeton on the insistence of one John Tukey.

Related

• Think like a statistician – without the math
• Statistician cracks the scratch lottery code
• Thomas the Tank Engine and Friends, accidental chart