Michael VanDaniker found that his energy provider, Baltimore Gas and Electric, provides customers with an easy-to-use tool to export their home’s energy usage by the hour. So he downloaded the CSV and had a look back at 2015, through the eyes of heating and cooling. Fun.
Read More

Overall life expectancy continues to increase, but looking at it from the other end — mortality rates — show different trends for different groups, especially women who live in rural areas. Dan Keating and Kennedy Elliott for the Washington Post explain with a collection of time series charts.

For younger age groups, drug overdose and suicide account for virtually all of the increases in death rate. For older groups, additional causes of death are also increasing, particularly heart and lung diseases for rural women, and cirrhosis for people over 45.

Rather than show mortality rates over time, the charts focus on the actual percentage change from 1990. A line that trends upwards is bad.

Peter Aldhous and Charles Seife dug into flight path data, specifically looking for flights manned by the FBI and the Department of Homeland Security.

The government’s airborne surveillance has received little public scrutiny — until now. BuzzFeed News has assembled an unprecedented picture of the operation’s scale and sweep by analyzing aircraft location data collected by the flight-tracking website Flightradar24 from mid-August to the end of December last year, identifying about 200 federal aircraft. Day after day, dozens of these planes circled above cities across the nation.

BuzzFeed’s searchable, animated map shows these circular paths, red for FBI and blue for DHS. There was no definite answer for what those planes are doing. Maybe routine surveillance or maybe lookouts for specific people or events. But still, so interesting.

Many people think of visualization as a plug-in tool that spits out something to look at. Microsoft Excel comes to mind. Some think of visualization as just that final chart to put on a presentation slide. However, there’s always a backstory about how it was made, who made it, why it was made, and most importantly, how the data came about. This is often more important than the finished product.

Artist Jer Thorp wrote about this a while back — about how visualization is a process. More recently, Jake Porway, the director of DataKind, wrote more about the process and how it ties into more rigorous analyses.

When data visualization is used simply to show alluring infographics about whether people like Coke or Pepsi better, the stakes of persuasion like this are low. But when they are used as arguments for or against public policy, the misuse of data visualization to persuade can have drastic consequences. Data visualization without rigorous analysis is at best just rhetoric and, at worse, incredibly harmful.

You need that analysis to figure out what you actually see in a visualization.

For those who make data graphics, this means picking and prodding at the data before you throw up a graph. For example, mean and median can mean a lot of things for a distribution. For those on the consumption side, this means questioning each graphic you see and don’t take every at face value. The bars and lines are usually much more squishy than they appear on the screen.

Government data is, shall we say, not the easiest to use and look at, which is why there are so many ongoing efforts to make it more accessible to both practitioners and the average citizen. There’s no doubt that the data is useful. The Sunlight Foundation does fine work with various projects, Census Reporter provides data at a glance, and efforts like IPUMS make certain large datasets easier to subset and grab.

Data USA, a collaboration between Deloitte, Macro Connections at the MIT Media Lab, and Datawheel, is another hefty project that aims to make government data feel less hairy. It uses data from a number of sources — the American Community Survey, the Bureau of Economic Analysis, and the Bureau of Labor Statistics, to name a few — to create profiles for locations, industries, occupations, and education.
Read More

This is beautiful work by digital art and design studio onformative. They recreate the driving experience with racing data from various tracks.
Read More

The WeMo Insight Switch from Belkin lets you remotely control a power outlet and tells you how much power the devices you plug into it use. More interested in the latter, I got one to see how well it works. Here are my first impressions with about a month of use and a week of data.
Read More

Before your next flight, road trip, or hike, download the Flyover Country app available for Android and iPhone. The app tells you information about where you are at any given moment, or if you’re flying, the ground beneath.

The app exposes interactive geologic maps from Macrostrat.org, fossil localities from Neotomadb.org and Paleobiodb.org, core sample localities from LacCore.org, Wikipedia articles, offline base maps, and the user’s current GPS determined location, altitude, speed, and heading. The app analyzes a given flight path and caches relevant map data and points of interest (POI), and displays these data during the flight, without in flight wifi.

Sedimentary geologist Zoltan Sylvester downloaded Landsat data using Earth Explorer and strung together images of the Ucayali River to see the changes over thirty years.

Thanks to the Landsat program and Google Earth Engine, it is possible now to explore how the surface of the Earth has been changing through the last thirty years or so. Besides the obvious issues of interest, like changes in vegetation, the spread of cities, and the melting of glaciers, it is also possible to look at how rivers change their courses through time.

Yeah, I’m gonna have to look at other areas of the world now. Brb.

Data checking is a pain and can be what stands between you and a good analysis or visualization session. Data Proofer aims to take away some of the pain by automating some of the process.

Every day, more and more data is created. Journalists, analysts, and data visualizers turn that data into stories and insights.

But before you can make use of any data, you need to know if it’s reliable. Is it weird? Is it clean? Can I use it to write or make a viz?

This used to be a long manual process, using valuable time and introducing the possibility for human error. People can’t always spot every mistake every time, no matter how hard they try.

Data proofer is built to automate this process of checking a dataset for errors or potential mistakes.

Gonna have to take this out for a spin.

Robert Simmon provides a hands-on guide to get true color from satellite imagery. The atmosphere makes it a little tricky:

The atmosphere scatters light from the sun before it hits the ground (or a cloud, but we don’t care about those at the moment), and then scatters reflected light again on its way back to a sensor. The atmosphere even scatters light back into a camera that didn’t hit anything on the ground at all.

That would be challenging enough, but the atmosphere changes from one place to another (the air above deserts is typically dry, while the air above a forest is usually moist (even when not cloudy) and often filled with tiny aerosol droplets), and over time (a hazy summer day compared to a crisp fall evening).

A couple of years ago, Eric Odenheimer wondered: If you stand on the beach looking out to the ocean and traveled straight until you reach land, what country would you reach? He only used latitude though. However, in real life, coastline is jagged and points in all directions, so you don’t always face east and west. Cartographer Andy Woodruff took these directions into account and drew a more accurate picture.
Read More

Robert O’Connell for the Atlantic ponders basketball analytics and the rise of Stephen Curry.

Like every sport, basketball has recently undergone a statistical overhaul. A new generation of analysts has pored over the game and come to conclusions about the efficacy of certain players and techniques. Their findings have met mixed acceptance from the old guard of coaches and executives, but at least one of their takeaways is now visible every night in the NBA. The three-point shot, for much of its history a novelty or minor part of teams’ strategies, has become an essential component of almost every team’s offensive attack. As recently as 2012, the average team took about 1,200 threes over the course of a season; last year, that number ballooned to over 1,800.

The difference between the Golden State Warriors and most other teams is that the shots go in, often in spectacular fashion. For this 2015-16 season, the Warriors put up more threes than anyone, but they made 41.5 percent of them so far, whereas everyone else is below 40.

I don’t read visualization books nearly as much as I wish I did, but there are a handful I keep on the shelf for a rainy day, which until recently was basically never here in California. I updated the books page to show some of my favorites.

I also added a few books in my queue that I hope to get to one day. Two are new visualization books that I heard good things about, one is an introduction to statistics (mostly for teaching reasons), and the last is a not-so-new one on design.

By the way, the statistics textbook is available for free as a PDF download.

“Weather data is this endless box of LEGO pieces that arise every day. It’s always a different box.” Sculptor Nathalie Miebach makes these ornate baskets based on weather data.

Peter Dodds teaches a course on complex networks, and he put together a set of tarot cards to illustrate concepts. Fun.

P.S. You can also watch the entire lecture series on YouTube.