After seeing Tristan Louis' list that tallied the streaming availability of 2011's top 100 box office hits, I was curious what it looked like graphically. So I put together this little number. Blue means available, yellow means not, and gray means it's only available for purchase. The last column for DVD simply means it's available (since DVDs are of course not streaming).

Netflix streaming still isn't a place to find the big movies (as any Netflix customer can tell you), with only five of the top 100 available. There is greater streaming availability from iTunes, Amazon, and Vudu, but those of course aren't fair comparisons to Netflix, given that the latter is subscription-only.

My main takeaway is that if you're deciding between the non-subscription services, it looks like price is the main thing to look at, since there doesn't seem to be much variability in availability (although it could be different for smaller movies). As for Netflix, subscribe for the television and for the movies less so.

[Tristan Louis via Waxy]

After seeing this map on The Guardian, I was curious about what other data was available from the National Highway Traffic Safety Administration. It turns out there's a lot and it's relatively easy to access via FTP. What's most surprising is that it's detailed and fairly complete, with columns for weather, number of people involved, date and time of accidents, and a lot more.

The above shows vehicles involved in fatal crashes in 2010 (which is different from number of crashes or number of fatalities). This data was just released last month, at the end of 2011 oddly enough. It's a calendar view with months stacked on top of one another and darker days indicate more vehicles involved.

Nearly every single data point also has location attached to it, so I tried some mapping, but they look like population density more or less. Here's one that shows crashes that occurred on local roads (orange) and those on freeways, highways, etc (blue). Road patterns start to come out for the major interstates.

I also mapped by weather (rain, fog, snow), time of day (midnight to 4am, 4-8, 8-noon, noon to 4, 4 to 8, and 8 to midnight), and crashes due to drunk driving but it wasn't as interesting as the day-by-day.

If you're a teacher looking for data to use with an assignment or just want to practice, this is a good set, despite the somber topic. You can find the data here, and there's an FTP link in the footer of the page to download more detailed data. You'll also need this guide [pdf] that defines all the variables.

I played around with D3 some more. This time I used data from The World Bank to look at life expectancy over time and by country. The data goes back to 1960 and up to the most current estimates for 2009. Each line represents a country, and you can see a few more details by rolling over each one.

More usefully, you can select regions, as defined by The World Bank, and see how groups of countries have changed or compare with other regions. For example, the shot above shows East Asia and Pacific in pink and Sub-Saharan Africa in green.

The major dips for some of the countries jumped out at me immediately. A few quick searches suggest that they coincide with war, such as the Bangladesh Liberation War in the 1970s, the Iran-Iraq War in the 1980s, and the Rwanda Civil War in the 1990s. There's also a smaller dip for Iraq starting in early 2000.

Some countries, such as Seychelles, didn't have data that went back to 1960. In this case, the line starts where annual data became available.

There are a few interactions that I'd add to make the graphic more complete, but don't have time to get to right now. For example, as I clicked around, I wanted to keep a selected country highlighted to see how it compared to others. It'd be informative to point out periods of war on the graphic, too.

See the full interactive here, and as usual, comments are welcome.

[Life Expectancy]

The Pew Internet and American Life Project published the results of their texting study, showing that young adults text more than anyone else. The report refers to a lot of averages across demographics, but it seems that there were a lot of heavy texters driving up those averages. The medians are a lot lower. The chart above shows the latter.

Even the median for young adults is still high though relative to other groups.

At 29, I'm right at the edge of that young adult group, and I text maybe once or twice a month on average. Forty per day seems outrageously high. Kids these days.

One of my favorite data graphics is an interactive piece by The New York Times that shows how Americans spend their day, based on the American Time Use Survey (ATUS). I've also been wanting to play with Mike Bostock's Data-Driven Documents, or D3 for short, for a while now. So put the two together, and this is what I got.

Main takeaway: we spend most of our time sleeping, eating, working, and watching television.

I followed the NYT aesthetic mostly for myself. I've found that a good way to learn is to try to copy something you like, and then use what you learn to do your own thing.

The NYT version was a stacked area chart with a number of interactions that made the data easier to read. I took a different route and split up some of the main activities, such as sleeping, eating, and working, into separate time series charts to see if it allowed you to see anything new. Showing separate charts at once places more focus on comparisons than on the distributions.

There was one challenge with the data that I didn't anticipate. ATUS provides a table of percentages by hour, but you'll notice with the NYT graphic the numbers are per ten minutes. You can actually calculate this with the ATUS microdata, which is basically the raw survey data from a few thousand respondents. I did this at first but lost patience, because I really just wanted to play with D3 rather than spend all my time building estimates.

I went with the hourly data which are averages for the sample population. The demographic breakdowns were only available in PDF, so I had some data entry fun, too. Luckily, I got my hands on Able2Extract (going off of Matthew Ericson's advice) to ease some of the pain.

Once the graphic was done, I noticed that the transitions are actually a nice way to show differences. For example, if you look at the time use for men and then women, the differences are subtle, but because the change is animated it's easier to spot. I think I knew this already, but probably never thought about it very deeply.

Anyways, it was fun playing around with D3. The beginning learning curve was kind of steep for me, but now that I know my way around a little better, I'm expecting more fun to come.

Mess around with the final graphic here. Let me know what you think.

[How Americans Spend Their Day]

There are many brands on Twitter that exist to uphold an image of the company they represent. As consumers, we can communicate with these accounts, voicing praise or displeasure (usually the latter). Using a simple sentiment classifier¹, I scored feelings towards major brands from 0 (horrible) to 100 (excellent) once a day for five days.

The above for example, shows scores for Netflix, Hulu, and Redbox. Netflix had the lowest scores, whereas Redbox had the highest. I suspect Netflix started low with people still upset over the price hike, but it got better the next couple of days. Then on Saturday, there was a score drop, which I'm guessing was from their downtime for most of Saturday. Hulu and Redbox, on the other hand, held more steady scores.

As for auto brands, Toyota clearly had the lowest scores. However, Lexus, which is actually a luxury vehicle division of Toyota had the highest scores in the high 90s to 100.

How about the major mobile phone companies, AT&T, Verizon, and Sprint? Verizon scored better initially, but had lower scores during the weekend. Not sure what was going on with Sprint.

Between Twitter and Facebook, there was obviously some bias, but Twitter faired slightly better. Twitter scored lower than I expected, but it probably has to do with bug reports directed towards @twitter.

Is Domino's Pizza good now? Papa John's stayed fairly steady while Pizza Hut scores were sub-par.

Finally, as a sanity check, I compared airlines like Breen did in his tutorial. Results were similar with JetBlue and Southwest clearly in the positive and the others picking up the rear.

Any of these scores seem surprising to you?

1. Jeffrey Breen provides an easy-to-follow tutorial on Twitter sentiment in R. The scoring system is pretty basic. All you do is load tweets with a given search phrase, and then find all the "good" words and "bad" words. Good words give +1, and bad words give -1. Then a tweet is classified good or bad based on the total. Then to get a final score, only tweets with total of +2 or more or -2 or less are counted. The final score is computed by dividing number of negative tweets divided by total number of "extreme" tweets. Obviously this won't pick up on sarcasm, but the scoring seems to still do a decent job. I wouldn't make any important business decisions based on these results though.

I came across some UFO sightings data on Infochimps, from the National UFO Reporting Center, and it seemed like a good excuse to mess around in R. I was just playing around, but the globular result was kind of fun to look at, so here it is.

The dataset is 60,000 sightings, but the above shows about 45,000 locations that could be geocoded immediately. The whiter the region, the more sightings there were in the area from 1906 to 2007.

Is it just me, or does the map above match up with this other map of major US airports?

Could some of the sightings actually be... airplanes? Nah. The aliens are coming. Luckily I have a big bat, several glasses of water, and asthma to keep me safe.

As I'm sure you know, Sony has been having all sorts of data breach problems lately — namely a million passwords from the Sony Pictures site, 77 million accounts from the PlayStation Network, and nearly 25 million user accounts from Online Entertainment. I was curious how these recent attacks compared to the largest known data loss incidents, so I headed over to DataLossDB. Sony now holds spots #4 and #10 for largest breaches of all time. That can't be good.

Below: a timeline of all known Sony data breaches so far this year, the biggest on April 26 and the second biggest soon after on May 2. More to come?

The United States Census Bureau just released results from the Survey of Income and Program Participation (SIPP) on marriage and divorce, and my wife and I just celebrated an anniversary this past weekend, so naturally I had to take a look.

The survey of about 39,000 households was actually taken in 2009 (The government can be slow sometimes.), but it provides a glimpse of how marriage and divorce has changed when you compare it to surveys from previous years.

My main takeaway was that people appear to be getting married at an older age, and as you get into the older age groups, the percentages for people who have married at least once are in the high 90s. The former doesn't surprise me simply because it matches with personal observation. The second part though was slightly surprising. For some reason, I always thought there were more people who went their whole lives without getting married.

Can you find any other interesting tidbits in the data?

Ever since seeing the Facebook friendship map and later, the map of scientific collaboration, I've been looking for an excuse to play with great circles. So I thought, why not come back to Aaron Koblin's classic Flight Patterns? But instead of just looking at all flights (above), I broke it down by airline to see where each one flies.

I grabbed the most recent flight data from the Bureau of Transportation Statistics, aggregated by airline, and counted arriving and departing flights between airport pairs. What follows are non-stop domestic flights by major air carriers during February 2011.

Brighter lines represent more arriving and departing flights between the two endpoints, and blue lines are the flights with heaviest traffic. Coloring is relative to within the airline as opposed to overall flight count.

On a quick glance you can spot where the hubs of each carrier are and flights most often flown. We start off with Southwest Airlines, which flies across the country. There's a focus obviously in the southwest.

Delta Air Lines flies just about everywhere, too, but also includes flights to Alaska and Hawaii. Their largest hub is in Atlanta, which explains the focus at Hartsfield–Jackson Atlanta International Airport.

United Airlines, on the other hand, has hubs more north and on the west coast including O'Hare International in Chicago and San Francisco International Airport. It appears they also have flights to all major Hawaiian islands.

Lots of American Airlines traffic in an out of Dallas/Fort Worth International Airport and JFK.

Continental Airlines looks similar to American Airlines, except Continental's headquarters are in Houston, Texas.

Pretty obvious where JetBlue goes. Despite some delays the past couple of times I've flown with them, they're still my favorite. One time Bill Murray was on the flight. If it's good enough for him, it must be good enough for me.

Mesa is a smaller airline that also operates United Express and US Airways Express.

US Airways' largest hub is at Charlotte/Douglas International Airport.

The Alaska Airlines connections look really interesting, streaming out of the the northwest. Most flights go through Seattle-Tacoma International, but there are also flights to and from Portland International. Oh, and of course to and from Alaska.

Atlantic Southeast lives up to its name.

As does Frontier Airlines.

Hawaiian Airlines looks exactly like you'd expect. They exclusively fly to Raleigh, North Carolina. Ah, I kid. I don't know where they fly.

See how these were made and how to apply this to your own data in this step-by-step tutorial.