EPCOR, the water utility company that runs the fountains up in Edmonton, Canada released this graph yesterday. It's water consumption during the Olympic gold medal hockey game, overlaying consumption of the previous day. How much do Canadians love their hockey? A lot.

The first period ends. Time to pee. The second period ends. Time to pee. The third period ends. Time to pee. Consumption goes way down when Canada wins and during the medal ceremony.

Finally, when it's all said and done, the rest of the country can relieve itself, figuratively and literally.

[via contrarian | thanks, @statpumpkin]

The Economist discusses the return of big government and includes this graph showing total government spending as a percentage of Gross Domestic Product. We see a dip in 2000 and a big jump this past year.

The trouble is that the country labels are cluttered. If you read them left to right, you get mixed up initially. Keep your eyes left and move top to bottom, and you might be okay.

The Challenge

Can you think of a way to make this graph easier to read? Is there a better way to represent the time series?

One catch: you have to work within the size limitation of 290 pixels wide and 300 pixels tall. It's an easy fix with unlimited space. But what can you do when space is scarce? Leave your thoughts in the comments below.

P.S. I was looking for the data this graph uses but got tired of using the OECD stat browser, so we'll just have to use our imagination for this one.

[Thanks, Justin]

Update: Here's GDP (sans spending) by country from 1995 to 2008 if anyone would like to take a wack [thanks, Kim].

The Natural Science Museum of Barcelona has a growing database of 50,000 records of specimens collected over the past 150 years. Bestiario explores this data in their biodiversity treemap and geographical map.

The cool thing about the treemap is that you can zoom in iteratively through the Phylums, Classes, Orders, Families, etc. The interaction is similar to NYT's treemap that showed Obama's budget proposal from earlier this month, except there are a lot more levels, so you zoom, and then you zoom some more.

[Thanks, Jose]

The Obama administration just posted a graph showing monthly job loss from December 2007 (Bush in red) up to last month. Discuss.

[via @nickbilton]

Update: There's a video version now [via infosthetics].

Tableau Software, popular for making data more accessible, mainly in the business sector, just opened up with Tableau Public. The application is similar in spirit to other online data applications like Many Eyes and Swivel. It lets you share data and visualizations online. However, Tableau Public doesn't have a central portal or a place to browse data. Rather it's focused on letting you explore data and stitch modules together on your desktop and then embed your findings on a website or blog.

For example, below is an interactive hosted on Tableau Public and made available online. Those who use Tableau Desktop should be familiar with the layout. You've got a map up top linked to the the time series on the bottom. Filters on the right let you focus on specific parts of the data.

Hot it Works

Like I said, Tableau Public works a little differently than you're used to. You do everything on your own computer first and when you're ready to share, the data and visualization is hosted on Tableau's servers. Copy and paste some javascript to your site and there you go. Easy stuff.

Thoughts

The main trouble with this, and you might have noticed this already, is that the interaction is kind of slow, because everything renders on the server and then is sent back to your browser. So it takes a second or two for every click to process (for me, at least).

For comparison, if we embedded something from Many Eyes, the applet and data are loaded locally, so the visualization interaction is much faster.

Speed can be fixed though. Loading times aside, this is a big shift for Tableau, and it'll be fun to see where it goes from here. It's especially good news for the non-programmer crowd that's interested in data. With the price tag of free, there's nothing to lose and a lot to play with.

Your Thoughts

What do you think? Try for yourself and post your thoughts in the comments below. I'd be especially interested in hearing what regular Tableau users think about the product. How does it compare?

Disclosure: Tableau Software is a FlowingData sponsor.

The treemap was his solution. It's an area-based visualization where the size of each rectangle represents a metric since made popular by Martin Wattenberg's Map of the Market and Marcos Weskamp's newsmap.

Here's a really easy way to make your own treemap in just a couple lines of code. We're looking to make something like this:

Step 0. Download R

Like before, we're going to use R, so you'll want to get it before going any further. Download it for Windows, Mac, or Linux. Don't let the out-dated site full you. You can get a lot done with the free software.

Step 1. Load the Data

We'll use data covering a hundred popular posts on FlowingData. Here it is in CSV format. You don't have to download it though. We'll just load it directly into R. The main thing to take note of is what is there. There's post id, number of views, number of comments, and category.

Okay, let's load it into R using read.csv():

data <- read.csv("http://datasets.flowingdata.com/post-data.txt")

Easy enough. We just used the read.csv() function to load data from a URL. If your data is on your computer, you could also do something like data <- read.csv("post-data.txt"). Just make sure the data file is in your current working directory, which you can change via the "Miscellaneous" menu.

Step 2. Load the Portfolio Library

Only a few more lines of code, and you've got a treemap. It's so easy, because we're going to use the portfolio library in R. First, you have to install it. You can either install the library via the "Package Installer" or you can do it through the command line. Let's do the latter. Type this in the console to install portfolio:

install.packages("portfolio")

Once installed, load it into R:

library(portfolio)

Step 3. Make the Treemap

It's time to make the treemap with map.market(). Type this in the console:

map.market(id=data$id, area=data$views, group=data$category, color=data$comments, main="FlowingData Map")

Tada. You should get something like this:

To sum up, we did this with four lines of code:

data <- read.csv("http://datasets.flowingdata.com/post-data.txt")
install.packages("portfolio")
library(portfolio)
map.market(id=data$id, area=data$views, group=data$category, color=data$comments, main="FlowingData Map")

Step 4. Customize

Now maybe you want to modify something like color. The cool thing about R is that you can see the code for all the functions, edit it, and then use your customized version. If the green and red scheme isn't for you or you don't care about the positive/negative cutoff, then you can change the code to do that. I won't go into detail, but if you type map.market in the console, you'll see the function. You can change color or cutoff around lines 36-46.

For example, you can do a black and white color scheme:

I was alright with the green for this, so I saved it as a PDF and then loaded it into Illustrator as usual. I numbed the green some, cleaned up the labels with a new font and layout, and updated the legend.

And there you go - a treemap with just a few lines of code in our all-trusty R. Rinse and repeat with your own data.

President Obama announced his 2011 budget proposal. How does it compare to last year's budget? Shan Carter and Amanda Cox of The New York Times compare the two plans. Red indicates a decrease in the percentage of the budget dedicated to the respective area, and green is for growth. Zoom in for a better view of the smaller areas.

Big decreases for Medicaid grants, veteran benefits, and unemployment insurance. Major increase for education, Medicare, and administration of justice.

This of course takes budget forecasting into account, which Ms. Cox shows isn't all that accurate sometimes.

[via @nytgraphics]

The Heatmap

In case you don't know what a heatmap is, it's basically a table that has colors in place of numbers. Colors correspond to the level of the measurement. Each column can be a different metric like above, or it can be all the same like this one. It's useful for finding highs and lows and sometimes, patterns.

On to the tutorial.

Step 0. Download R

We're going to use R for this. It's a statistical computing language and environment, and it's free. Get it for Windows, Mac, or Linux. It's a simple one-click install for Windows and Mac. I've never tried Linux.

Did you download and install R? Okay, let's move on.

Step 1. Load the data

Like all visualization, you should start with the data. No data? No visualization for you.

For this tutorial, we'll use NBA basketball statistics from last season that I downloaded from databaseBasketball. I've made it available here as a CSV file. You don't have to download it though. R can do it for you.

I'm assuming you started R already. You should see a blank window.

Now we'll load the data using read.csv().

nba <- read.csv("http://datasets.flowingdata.com/ppg2008.csv", sep=",")


We've read a CSV file from a URL and specified the field separator as a comma. The data is stored in nba.

Type nba in the window, and you can see the data.

Step 2. Sort data

The data is sorted by points per game, greatest to least. Let's make it the other way around so that it's least to greatest.

nba <- nba[order(nba$PTS),]


We could just as easily chosen to order by assists, blocks, etc.

Step 3. Prepare data

As is, the column names match the CSV file's header. That's what we want.

But we also want to name the rows by player name instead of row number, so type this in the window:

row.names(nba) <- nba$Name


Now the rows are named by player, and we don't need the first column anymore so we'll get rid of it:

nba <- nba[,2:20]


Step 4. Prepare data, again

Are you noticing something here? It's important to note that a lot of visualization involves gathering and preparing data. Rarely, do you get data exactly how you need it, so you should expect to do some data munging before the visuals. Anyways, moving on.

The data was loaded into a data frame, but it has to be a data matrix to make your heatmap. The difference between a frame and a matrix is not important for this tutorial. You just need to know how to change it.

nba_matrix <- data.matrix(nba)


Step 5. Make a heatmap

It's time for the finale. In just one line of code, build the heatmap (remove the line break):

nba_heatmap <- heatmap(nba_matrix, Rowv=NA, Colv=NA,
col = cm.colors(256), scale="column", margins=c(5,10))


You should get a heatmap that looks something like this:

Step 6. Color selection

Maybe you want a different color scheme. Just change the argument to col, which is cm.colors(256) in the line of code we just executed. Type ?cm.colors for help on what colors R offers. For example, you could use more heat-looking colors:

nba_heatmap <- heatmap(nba_matrix, Rowv=NA, Colv=NA,
col = heat.colors(256), scale="column", margins=c(5,10))


For the heatmap at the beginning of this post, I used the RColorBrewer library. Really, you can choose any color scheme you want. The col argument accepts any vector of hexidecimal-coded colors.

Step 7. Clean it up - optional

If you're using the heatmap to simply see what your data looks like, you can probably stop. But if it's for a report or presentation, you'll probably want to clean it up. You can fuss around with the options in R or you can save the graphic as a PDF and then import it into your favorite illustration software.

I personally use Adobe Illustrator, but you might prefer Inkscape, the open source (free) solution. Illustrator is kind of expensive, but you can probably find an old version on the cheap. I still use CS2. Adobe's up to CS4 already.

For the final basketball graphic, I used a blue color scheme from RColorBrewer and then lightened the blue shades, added white border, changed the font, and organized the labels in Illustrator. Voila.

Rinse and repeat to use with your own data. Have fun heatmapping.

Statisticians are generally behind the times when it comes to online applications. There are a lot out-dated Java applets and really rough attempts at getting R, a statistical computing environment, in some useful form through a browser. So imagine my surprise when I tried this tool by Jeroen Ooms, a visiting scholar at UCLA Statistics.

It actually works pretty well, and for a prototype, it isn't half bad.

Statistical Graphics as Layers

The engine behind Jeroen's tool is Hadley Wickham's ggplot2, an R implementation of of Leland Wilkinson's Grammar of Graphics. Yes, it's a big chain of events.

Those who use Adobe products, like Illustrator or Photoshop, are familiar with the idea of layers in your graphics.

Without going into all the details, the main idea is that statistical graphics are made up of layers. There's the data, axes, and the plot. You can do this in R with the ggplot2 library, or, with this tool, you can upload data and specify the layers through the clickety interface.

Here's a short demo of ggplot in action:

This is a step-by-step guide on how to make a similar visualization in Actionscript/Flash with your own data and how to customize the design for whatever you need. We're after last week's graphic on consumer spending:

Audience

This tutorial is for people with at least a little bit of programming experience. I'll try to make it as straightforward as possible, but the concepts might be a little hard to grasp if you've never written a line of code. Just a heads up. Of course it never hurts to try.

If you don't care about customization or integration into an application and don't mind putting your data in the public domain, you could also just dump your data into Many Eyes, and use the Stack Graph.

Get Adobe Flex Builder

Like I said, this is all in Actionscript, so before we start anything, I strongly recommend you get Adobe Flex Builder if you don't already have it. You can buy it, get a trial version from the Adobe site, or if you're in education, you can get it for free.

There are ways to compile Actionscript without Flex Builder, but they are more complicated.

Working With Flare

Luckily you don't have to start from scratch. In fact, most of the work has already been done for you using Jeffrey Heer's Flare visualization toolkit. It's an Actionscript library. We're going to work off one of the sample applications: JobVoyager. Once you get your development environment setup, it's just a matter of switching in your data, and then customizing the look and feel.

Okay, let's get started (finally).

Step 0. Download and import Flare

Download the most recent version of Flare, and then unpack the contents into your working directory.

Open Flex Builder. You should see something like this:

Right click on the Flex Navigator (left-hand side) and click on "Import..." You'll get a popup that looks like this:

Select "Existing Projects into Workspace" and click "Next." Browse to where you put the Flare files. Select the "flare" directory, and then make sure "flare" is checked in the project window.

Do the same thing with the "flare.apps" folder. Your Flex Builder window should look like this once you've expanded the flare.apps/flare/apps/ and click on JobVoyager.as.

If you clicked the run button right now (the green button with the white play triangle top left), you should see the working JobVoyager. Get that working, and you're done with the hardest part.

Step 1. Update the data source

Let's dive into the code now so you can adapt the visualization to your own data and customize the aesthetics. Again, we're going to adapt this code to consumer spending data to make last week's graphic.

First we need to change the data source. This is specified on line 57.

private var _url:String = "http://flare.prefuse.org/data/jobs.txt";
 

Change the _url to point at the spending data, which like jobs.txt, is also a tab-delimited file. The first column is year, the second category, and the last column is expenditure:

private var _url:String = "http://datasets.flowingdata.com/expenditures.txt"
 

Now the file will read in our spending data instead of for jobs. Easy stuff so far.

Step 2. Change the years

The next two lines, line 58 and 59 are the column names, or in this case, the distinct years that job data was available. It's by decade from 1850 to 2000. We could make things more robust by finding the years in the loaded data, but since the data isn't changing we can same some time and explicitly specify the years.

Our expenditures data is annual from 1984 to 2008. We'll change lines 58-59 accordingly.

private var _cols:Array = [1984,1985,1986,1987,1988,1989,1990,1991,
		1992,1993,1994,1995,1996,1997,1998,1999,
		2000,2001,2002,2003,2004,2005,2006,2007,2008];
 

Step 3. Data headers

Next we need to change references to the data headers. The original data file (jobs.txt) has four columns: year, occupation, people and sex. Our spending data only has three columns: year, category, and expenditure. We have to adapt the code to this new data structure.

Luckily, it's pretty easy. The year column is the same, so we just need to change any people references to expenditure (vertical axis) and any occupation references to category (the layers). Finally, we'll remove all uses of gender.

At line 74 the data is reshaped and prepared for the stacked area chart. It specifies by occupation and sex as the categories (i.e. layers), uses year on the x-axis, and people on the y-axis.

var dr:Array = reshape(ds.nodes.data, ["occupation","sex"],
     "year", "people", _cols);
 

Change it to this:

var dr:Array = reshape(ds.nodes.data, ["category"],
     "year", "expenditure", _cols);
 

We only have one category (sans sex), and that's uh, category. The x-axis is still year, and the y-axis is expenditure.

Step 4. Sorting

Line 84 sorts the data by occupation (alphabetically) and then sex (numerically). We'll just sort by category:

data.nodes.sortBy("data.category");
 

Are you starting to get the idea here?

Step 5. Proper categories

Line 92 colors layers by sex, but we don't have that split in our data, so we don't need to do that. Remove the entire row:

data.nodes.setProperty("fillHue", iff(eq("data.sex",1), 0.7, 0));
 

We'll come back to customizing the colors of the stacks a little later.

Step 6. Labeling areas

Line 103 adds labels based occupation:

_vis.operators.add(new StackedAreaLabeler("data.occupation"));
 

We want to label based on spending category, so we'll change the line accordingly:

_vis.operators.add(new StackedAreaLabeler("data.category"));
 

Step 7. Interactive filters

Lines 213-231 handle filtering in JobVoyager. First there's the male/female filter and then there's the filter by occupation. We don't need the former, so we can get rid of lines 215-218 and then make line 219 a plain if statement.

Similarly, lines 264-293 at buttons to trigger the male/female filter. We can get rid of that too.

Step 8. Search categories

We're getting really close to fully customizing the voyager to our spending data. Go back to the filter() function at line 213. Again, we need to update the function so that we can filter by spending category instead of occupation.

Here's line 222 as-is:

var s:String = String(d.data["occupation"]).toLowerCase();
 

Change occupation to category:

var s:String = String(d.data["category"]).toLowerCase();
 

Step 9. Color scheme

If you ran the code right now, everything should compile correctly, and it'll look something like this:

Color is specified in two places. First lines 86-89 specify stroke color and colors everything red:

shape: Shapes.POLYGON,
lineColor: 0,
fillValue: 1,
fillSaturation: 0.5
 

Then line 105 updates saturation (the level of red), by count. The code for the SaturationEncoder() is in lines 360-383. We're not going to use saturation though. Instead, we'll explicitly specify the color scheme.

First update lines 86-89 to this:

shape: Shapes.POLYGON,
lineColor: 0xFFFFFFFF
 

We're going to make stroke color white with lineColor. If there were more spending categories, we probably wouldn't do this because it'd be cluttered. We don't have that many though, so it'll make reading a little easier.

Next, make an array of the colors we want to use ordered by levels. Put it towards the top around line 50:

private var _reds:Array = [0xFFFEF0D9, 0xFFFDD49E, 0xFFFDBB84,
    0xFFFC8D59, 0xFFE34A33, 0xFFB30000];
 

I used the ColorBrewer for these colors.

Then we'll add a new ColorEncoder around line 110:

var colorPalette:ColorPalette = new ColorPalette(_reds);
_vis.operators.add(new ColorEncoder("data.max", "nodes", "fillColor", null, colorPalette));
 

Tada, you now have something that looks like what we're after:

Here's the finished product.

Step 10. Download the code and see for yourself

To play with this yourself, just complete step 0 and then replace JobVoyager.as with this file. It has all the updates we've just covered.

Step 11. Where to Go From Here

There's a lot of things you can do with this. You can apply this to your own data, use a different color scheme, and further customize to fit your needs. Maybe change the font or the tooltip format. Then you can get fancier and integrate it with other tools or add more Actionscript, so on and so forth. If anything, you should at least check out what else you can do with Jeffrey's Flare visualization toolkit.

Have fun!