Easily download large-ish survey datasets

Posted to Data Sources  |  Tags: , ,  |  Nathan Yau

Many government organizations release microdata for surveys every year. It comes as anonymized responses from each survey participant for each question in said survey. However, those who want to use this data often run into the challenge of downloading and parsing. It’s rarely straightforward.

So, Anthony Damico provides a big helping of R scripts to easily download data from a bunch of surveys. He calls the site Analyze Survey Data for Free.

Governments spend billions of dollars each year surveying their populations. If you have a computer and some energy, you should be able to unlock it for free, with transparent, open-source software, using reproducible techniques. We’re in a golden era of public government data, but almost nobody knows how to mine it with technology designed for this millennium. I can change that, so I’m gonna. Help. Use it.

The site has been around for a few years but I just discovered it. I’m not sure we’re in a “golden era of public government data” right now (although I’d be happy if you prove me wrong).

I recently used a modified script to download data from the CDC, and it saved me a bunch of time.



Watching the growth of Walmart – now with 100% more Sam’s Club

The ever so popular Walmart growth map gets an update, and yes, it still looks like a wildfire. Sam’s Club follows soon after, although not nearly as vigorously.

Famous Movie Quotes as Charts

In celebration of their 100-year anniversary, the American Film Institute …

A Day in the Life of Americans

I wanted to see how daily patterns emerge at the individual level and how a person’s entire day plays out. So I simulated 1,000 of them.

Most popular porn searches, by state

We’ve seen that we can learn from what people search …