There are a lot of ways to show distributions, but for the purposes of this tutorial, I'm only going to cover the more traditional plot types like histograms and box plots. Otherwise, we could be here all night. Plus the basic distribution plots aren't exactly well-used as it is.

Before you get into plotting in R though, you should know what I mean by distribution. It's basically the spread of a dataset. For example, the median of a dataset is the half-way point. Half of the values are less than the median, and the other half are greater than. That's only part of the picture.

What happens in between the maximum value and median? Do the values cluster towards the median and quickly increase? Are there are lot of values clustered towards the maximums and minimums with nothing in between? Sometimes the variation in a dataset is a lot more interesting than just mean or median. Distribution plots help you see what's going on.

Want more? Google and Wikipedia are your friend.Anyways, that's enough talking. Let's make some charts.

If you don't have R installed yet, do that now.

Box-and-Whisker Plot

This old standby was created by statistician John Tukey in the age of graphing with pencil and paper. I wrote a short guide on how to read them a while back, but you basically have the median in the middle, upper and lower quartiles, and upper and lower fences. If there are outliers more or less than 1.5 times the upper or lower quartiles, respectively, they are shown with dots.

The method might be old, but they still work for showing basic distribution. Obviously, because only a handful of values are shown to represent a dataset, you do lose the variation in between the points.

To get started, load the data in R. You'll use state-level crime data from the Chernoff faces tutorial.

# Load crime data
crime <- read.csv("http://datasets.flowingdata.com/crimeRatesByState-formatted.csv")

Remove the District of Columbia from the loaded data. Its city-like makeup tends to throw everything off.

# Remove Washington, D.C.
crime.new <- crime[crime$state != "District of Columbia",]

Oh, and you don't need the national averages for this tutorial either.

# Remove national averages
crime.new <- crime.new[crime.new$state != "United States ",]

Now all you have to do to make a box plot for say, robbery rates, is plug the data into boxplot().

# Box plot
boxplot(crime.new$robbery, horizontal=TRUE, main="Robbery Rates in US")

Want to make box plots for every column, excluding the first (since it's non-numeric state names)? That's easy, too. Same function, different argument.

# Box plots for all crime rates
boxplot(crime.new[,-1], horizontal=TRUE, main="Crime Rates in US")

Multiple box plot for comparision.

Histogram

Like I said though, the box plot hides variation in between the values that it does show. A histogram can provide more details. Histograms look like bar charts, but they are not the same. The horizontal axis on a histogram is continuous, whereas bar charts can have space in between categories.

Just like boxplot(), you can plug the data right into the hist() function. The breaks argument indicates how many breaks on the horizontal to use.

# Histogram
hist(crime.new$robbery, breaks=10)

Look, ma! It's not a a bar chart.

Using the hist() function, you have to do a tiny bit more if you want to make multiple histograms in one view. Iterate through each column of the dataframe with a for loop. Call hist() on each iteration.

# Multiple histograms
par(mfrow=c(3, 3))
colnames <- dimnames(crime.new)[[2]]
for (i in 2:8) {
	hist(crime[,i], xlim=c(0, 3500), breaks=seq(0, 3500, 100), main=colnames[i], probability=TRUE, col="gray", border="white")
}

Using the same scale for each makes it easy to compare distributions.

Density Plot

For smoother distributions, you can use the density plot. You should have a healthy amount of data to use these or you could end up with a lot of unwanted noise.

To use them in R, it's basically the same as using the hist() function. Iterate through each column, but instead of a histogram, calculate density, create a blank plot, and then draw the shape.

# Density plot
par(mfrow=c(3, 3))
colnames <- dimnames(crime.new)[[2]]
for (i in 2:8) {
	d <- density(crime[,i])
	plot(d, type="n", main=colnames[i])
	polygon(d, col="red", border="gray")
}

Multiple filled density plots.

You can also use histograms and density lines together. Instead of plot(), use hist(), and instead of drawing a filled polygon(), just draw a line.

# Histograms and density lines
par(mfrow=c(3, 3))
colnames <- dimnames(crime.new)[[2]]
for (i in 2:8) {
	hist(crime[,i], xlim=c(0, 3500), breaks=seq(0, 3500, 100), main=colnames[i], probability=TRUE, col="gray", border="white")
	d <- density(crime[,i])
	lines(d, col="red")
}

Histogram and density, reunited, and it feels so good.

Rug

The rug, which simply draws ticks for each value, is another way to show distributions. It usually accompanies another plot though, rather than serve as a standalone. Simply make a plot like you usually would, and then use rug() to draw said rug.

# Density and rug
d <- density(crime$robbery)
plot(d, type="n", main="robbery")
polygon(d, col="lightgray", border="gray")
rug(crime$robbery, col="red")

Using a rug under a density plot.

Violin Plot

The violin plot is like the lovechild between a density plot and a box-and-whisker plot. There's a box-and-whisker in the center, and it's surrounded by a centered density, which lets you see some of the variation.

# Violin plot
library(vioplot)
vioplot(crime.new$robbery, horizontal=TRUE, col="gray")

I bet this violin sounds horrible.

Bean Plot

The bean plot takes it a bit further than the violin plot. It's something of a combination of a box plot, density plot, and a rug in the middle. I've never actually used this one, and I probably never will, but there you go.

# Bean plot
library(beanplot)
beanplot(crime.new[,-1])

A little too busy for me, but here you go.

Wrapping Up

If you take away anything from this, it should be that variance within a dataset is worth investigating. Picking out single datapoints or only using medians is the easy thing to do, but it's usually not the most interesting.

Related

• Mapping Crime in Oxford Over Time
• How to visualize data with cartoonish faces ala Chernoff
• Calendar Heatmaps to Visualize Time Series Data

Here's the technical definition of great circles on Wikipedia:

A great circle, also known as a Riemannian circle, of a sphere is the intersection of the sphere and a plane which passes through the center point of the sphere, as distinct from a small circle. Any diameter of any great circle coincides with a diameter of the sphere, and therefore all great circles have the same circumference as each other, and have the same center as the sphere. A great circle is the largest circle that can be drawn on any given sphere. Every circle in Euclidean space is a great circle of exactly one sphere.

The important bit is that the shortest distance between two points on a sphere is the minor arc of a great circle. When currents and wind don't interfere, ships and aircraft use great circle routes, which makes it perfect to show air carrier coverage. This is what you'll do in this example.

It turns out these maps are relatively easy to make in R once you know how to put the pieces together. The maps that I posted on flight connections for each airline are what we're after. With only about 30 lines of code, you can produce a series of maps that show flights for every major airline, so you get a lot of bang for the amount of effort.

Step 0. Setup

1. I wish they'd update the site; it totally looks like something out of the 1990s, but nevermind that. It's useful software.You're going to use R in this example, so download the free and open-source software if you haven't already. It's a straightforward one-click install¹.

Step 1. Load packages

Open R. You need two packages to do the heavy-lifting: maps and geosphere. If they're not installed, you can do that via the main menu Packages & Data > Package Installer. Once installed, load the two packages as follows:

library(maps)
library(geosphere)

The first package maps, is used to draw the base maps, and the second, geosphere is used to draw the great circle arcs.

Step 2. Draw base maps

The maps package makes it easy to draw geographic areas in R with the map() function. Pass it a database name, and you you get a map in one line of code. For example, to map the United States, type the following in the R console.

map("state")

Here's the map that you get:

Blank state map created in R. Nothing else.

2. Map projections are finicky in R and can be a pain sometimes, but if you want to map with a different projection like say, Albers, look into mapproject() in the maps package.The projection isn't the prettiest thing in the world, but it'll do for now². The bigger problem is that the state database doesn't include Alaska or Hawaii. To include the two often left out states, you use the world database.

map("world")

This gives you a full black and white map of the world.

A blank world map is just as easy to make in R.

Step 3. Limiting boundaries

The data at hand, which you'll get to soon, is only domestic flights for the United States, so you should focus on that area. Use the xlim and ylim variables to limit the map to a rectangle that only covers a range of latitude and longitude.

You can also play with the color of the base map at this point. By default, maps() doesn't fill regions, but it will if you set fill to TRUE. For some reason though, if you set the fill color, you can't change the border color. So instead (if you don't want to edit in Illustrator later) you can set the line width (lwd) to something really skinny. For the purpose of this example, we want the the border lines to get out of the way.

xlim <- c(-171.738281, -56.601563)
ylim <- c(12.039321, 71.856229)
map("world", col="#f2f2f2", fill=TRUE, bg="white", lwd=0.05, xlim=xlim, ylim=ylim)

And here's what you get with the above code. A simple and clean map of the United States that includes Alaska and Hawaii.

Focusing on all states. That includes Hawaii and Alaska.

Step 4. Draw connecting lines

Now that you have a map, you can draw connecting lines. This is really easy with gcIntermediate() from the geosphere package. Pass it the latitude and longitude of the two connecting points, and gcIntermediate() spits out the coordinates of points on the circle.

The n argument indicates how many points you want the function to return. The more points you indicate, the smoother the resulting line will be, but up to a certain point, you won't see much difference. The addStartEnd argument indicates that you want to include the start and end points in the great circle coordinates. Lastly, use lines() to actually draw the line.

lat_ca <- 39.164141
lon_ca <- -121.640625
lat_me <- 45.213004
lon_me <- -68.906250
inter <- gcIntermediate(c(lon_ca, lat_ca), c(lon_me, lat_me), n=50, addStartEnd=TRUE)
lines(inter)

This draws a great circle arc from California to Maine.

A single connection.

Similarly, you can add another line simply by doing the same as above with different latitude and longitude coordinates. For example, you can draw a line from California to Texas, and you can use the col argument to set the line color to red.

lat_tx <- 29.954935
lon_tx <- -98.701172
inter2 <- gcIntermediate(c(lon_ca, lat_ca), c(lon_tx, lat_tx), n=50, addStartEnd=TRUE)
lines(inter2, col="red")

Okay, now let's do two connections.

Step 5. Load flight data

So now you know how to do the hard part. You just have to iterate over latitude and longitude pairs to get the full map for a specific carrier. Start by loading the data with read.csv(), as shown below:

airports <- read.csv("http://datasets.flowingdata.com/tuts/maparcs/airports.csv", header=TRUE) flights <- read.csv("http://datasets.flowingdata.com/tuts/maparcs/flights.csv", header=TRUE, as.is=TRUE)

3. I used a five-line Python script to aggregate data that I downloaded from the Bureau of Transportation Statistics. The original file was about 50mb. You can aggregate in R, but it's usually a better idea to not load large-ish files in R. Luckily airport latitude and longitude coordinates were available on the page for Data Expo 2009. Otherwise, I would've geocoded them myself, which I started to do and then got stalled by API limits.This is processed data that I cleaned up for this tutorial. It's flight counts between each airport, categorized by airline³.

Step 6. Draw multiple connections

Data in. To map all connections for say, American Airlines, you filter as shown in line 3. Then you loop over each row of data, which has latitude/longitude for two two airports and the number of flights between them.

map("world", col="#f2f2f2", fill=TRUE, bg="white", lwd=0.05, xlim=xlim, ylim=ylim)

fsub <- flights[flights$airline == "AA",]
for (j in 1:length(fsub$airline)) {
	air1 <- airports[airports$iata == fsub[j,]$airport1,]
	air2 <- airports[airports$iata == fsub[j,]$airport2,]

	inter <- gcIntermediate(c(air1[1,]$long, air1[1,]$lat), c(air2[1,]$long, air2[1,]$lat), n=100, addStartEnd=TRUE)

	lines(inter, col="black", lwd=0.8)
}

Here's the mess of black lines that you get from the above code.

Rough image of American Airlines flights.

Not bad, but you can do better than that with some rearranging and coloring appropriately.

Step 7. Color for clarity

You learned how to change the color of lines back in step 4. You change the col argument in lines(). You hard-coded the color though. You could instead create a vector of colors that scaled from, say, light gray to black, and then pick a shade from farther down the vector for connections with more flights. That way connections with more flights would be more prominent.

If only there were a way to create a color scale automagically. Oh wait, there is. It's called colorRampPalette(). Pass it the base colors you want to use, and it'll fill in everything in between. More specifically, it creates a function that you can pass a number two, indicating how many shades you want to use. In the code below, we use 100 (lines 1 and 2).

Then you do the same as you did the previous step, but instead of setting all lines to black, you calculate the color based on how many fewer flights the current connection has compared to the maximum flight count.

pal <- colorRampPalette(c("#f2f2f2", "black"))
colors <- pal(100)

map("world", col="#f2f2f2", fill=TRUE, bg="white", lwd=0.05, xlim=xlim, ylim=ylim)

fsub <- flights[flights$airline == "AA",]
maxcnt <- max(fsub$cnt)
for (j in 1:length(fsub$airline)) {
	air1 <- airports[airports$iata == fsub[j,]$airport1,]
	air2 <- airports[airports$iata == fsub[j,]$airport2,]

	inter <- gcIntermediate(c(air1[1,]$long, air1[1,]$lat), c(air2[1,]$long, air2[1,]$lat), n=100, addStartEnd=TRUE)
	colindex <- round( (fsub[j,]$cnt / maxcnt) * length(colors) )

	lines(inter, col=colors[colindex], lwd=0.8)
}

Below is the map that you get. The problem is the longer, less prominent flights are obscuring the more popular connections, because they're being drawn on top. The above code just draws lines in the order that the data comes.

Use color to emphasize more prominent flights.

To fix this we use the same method that Paul Butler used for his Facebook map. We just order connection from least to greatest flight counts. That way less popular connections are drawn first and therefore, will be on the bottom, while the darker connections will be drawn on top.

pal <- colorRampPalette(c("#f2f2f2", "black"))
pal <- colorRampPalette(c("#f2f2f2", "red"))
colors <- pal(100)

map("world", col="#f2f2f2", fill=TRUE, bg="white", lwd=0.05, xlim=xlim, ylim=ylim)

fsub <- flights[flights$airline == "AA",]
fsub <- fsub[order(fsub$cnt),]
maxcnt <- max(fsub$cnt)
for (j in 1:length(fsub$airline)) {
	air1 <- airports[airports$iata == fsub[j,]$airport1,]
	air2 <- airports[airports$iata == fsub[j,]$airport2,]

	inter <- gcIntermediate(c(air1[1,]$long, air1[1,]$lat), c(air2[1,]$long, air2[1,]$lat), n=100, addStartEnd=TRUE)
	colindex <- round( (fsub[j,]$cnt / maxcnt) * length(colors) )

	lines(inter, col=colors[colindex], lwd=0.8)
}

Layer dark on top of light so that it's easier to read.

That's much better. At this point, you can play around with color, by changing the shades in colorRampPalette(). Below uses a light gray (#f2f2f2) to red, but you can do whatever you like. You can even use more than two colors.

pal <- colorRampPalette(c("#f2f2f2", "red"))

Red? Sure, you can do that, too.

Step 8. Map every carrier

The only thing left to do is make a map for each carrier. You can do it manually by changing the airline code and the rerunning the script, but there's an easier way. Find all unique carriers with the unique() function, and iterate over each one. The place that you put "AA" you replace with carriers[i] to indicate the current carrier in the loop. The code below will create a PDF for each carrier and save to your current working directory.

# Unique carriers
carriers <- unique(flights$airline)

# Color
pal <- colorRampPalette(c("#333333", "white", "#1292db"))
colors <- pal(100)

for (i in 1:length(carriers)) {

	pdf(paste("carrier", carriers[i], ".pdf", sep=""), width=11, height=7)
	map("world", col="#191919", fill=TRUE, bg="#000000", lwd=0.05, xlim=xlim, ylim=ylim)
	fsub <- flights[flights$airline == carriers[i],]
	fsub <- fsub[order(fsub$cnt),]
	maxcnt <- max(fsub$cnt)
	for (j in 1:length(fsub$airline)) {
		air1 <- airports[airports$iata == fsub[j,]$airport1,]
		air2 <- airports[airports$iata == fsub[j,]$airport2,]

		inter <- gcIntermediate(c(air1[1,]$long, air1[1,]$lat), c(air2[1,]$long, air2[1,]$lat), n=100, addStartEnd=TRUE)
		colindex <- round( (fsub[j,]$cnt / maxcnt) * length(colors) )

		lines(inter, col=colors[colindex], lwd=0.6)
	}

	dev.off()
}

Here is the map for American Airlines again, produced by the code above. I fiddled with color some to match the maps I created for the original flight post.

A dark map background with gray to white to blue paths.

That's all there is to it. Can you think of other datasets this method could be applied to? Give this tutorial a whirl and post your results in the comments.

For more examples, guidance, and all-around data goodness like this, sign up for FlowingData membership.

Related

• Great Data Visualization Tells a Great Story
• Geographic breakdown: Where do major airlines fly?
• Where the aliens are flying their UFOs

A bubble chart can also just be straight up proportionally sized bubbles, but here we're going to cover how to create the variety that is like a scatterplot with a third, bubbly dimension.

The advantage of this chart type is that it lets you compare three variables at once. One is on the x-axis, one is on the y-axis, and the third is represented by area size of bubbles. Have a look at the final chart to see what we're making.

Step 0. Download R

We're going to use R to do this, so download that before moving on. It's free and open-source, so you have nothing to lose. Plus it's a need-to-know-name of 2011, so you might as well get to know it now. You can thank me later.

Step 1. Load the data

Assuming you already have R open, the first thing we'll do is load the data. We're examining the same crime data the we did for our last tutorial. I've added state population this time around. One note about the data. The crime numbers are actually for 2005, while the populations are for 2008. This isn't a huge deal since we're more interested in relative populations than we are the raw values, but keep that in mind.

Okay, moving on. You can download the tab-delimited file here and keep it local, but the easiest way is to load it directly into R with the below line of code:

crime <- read.csv("http://datasets.flowingdata.com/crimeRatesByState2005.tsv", header=TRUE, sep="\t")

You're telling R to download the data and read it as a comma-delimited file with a header. This loads it as a data frame in the crime variable.

Step 2. Draw some circles

Now we can get right to drawing circles with the symbols() command. Pass it values for the x-axis, y-axis, and circles, and it'll spit out a bubble chart for you.

symbols(crime$murder, crime$burglary, circles=crime$population)

Run the line of code above, and you'll get this:

Circles incorrectly sized by radius instead of area. Large values appear much bigger.

All done, right? Wrong. That was a test. The above sizes the radius of the circles by population. We want to size them by area. The relative proportions are all out of wack if you size by radius.

Step 3. Size the circles correctly

To size radiuses correctly, we look to the equation for area of a circle:

Area of circle = πr²

In this case area of the circle is population. We want to know r. Move some things around and we get this:

r = √(Area of circle / π)

Substitute population for the area of the circle, and translate to R, and we get this:

radius <- sqrt( crime$population/ pi )
symbols(crime$murder, crime$burglary, circles=radius)

Circles correctly sized by area, but the range of sizes is too much. The chart is cluttered and unreadable.

Yay. Properly scaled circles. They're way too big though for this chart to be useful. By default, symbols() sizes the largest bubble to one inch, and then scales the rest accordingly. We can change that by using the inches argument. Whatever value you put will take the place of the one-inch default. While we're at it, let's add color and change the x- and y-axis labels.

symbols(crime$murder, crime$burglary, circles=radius, inches=0.35, fg="white", bg="red", xlab="Murder Rate", ylab="Burglary Rate")

Notice we use fg to change border color, bg to change fill color. Here's what we get:

Scale the circles to make the the chart more readable, and use the fg and bg arguments to change colors.

Now we're getting somewhere.

By the way, you can make a chart with other shapes too with symbols(). You can make squares, rectangles, thermometers, boxplots, and stars. They take different arguments than the circle. The squares, for example, are sized by the length of a side. Again, make sure you size them appropriately.

Here's what squares look like, using the below line of code.

symbols(crime$murder, crime$burglary, squares=sqrt(crime$population), inches=0.5)

You can use squares sized by area instead of circles, too.

Let's stick with circles for now.

Step 4. Add labels

As it is, the chart shows some sense of distribution, but we don't know which circle represents each state. So let's add labels. We do this with text(), whose arguments are x-coordinates, y-coordinates, and the actual text to print. We have all of these. Like the bubbles, the x is murders and the y is burglaries. The actual labels are state names, which is the first column in our data frame.

With that in mind, we do this:

text(crime$murder, crime$burglary, crime$state, cex=0.5)

The cex argument controls text size. It is 1 by default. Values greater than one will make the labels bigger and the opposite for less than one. The labels will center on the x- and y-coordinates.

Here's what it looks like.

Add labels so you know what each circle represents.

Step 5. Clean up

Finally, as per usual, I clean up in Adobe Illustrator. You can mess around with this in R, if you like, but I've found it's way easier to save my file as a PDF and do what I want with Illustrator. I uncluttered the state labels to make them more readable, rotated the y-axis labels, so that they're horizontal, added a legend for population, and removed the outside border. I also brought Georgia to the front, because most of it was hidden by Texas.

Here's the final version. Click the image to see it in full.

Cleanup and a key make the chart more informative.

And there you go. Type in ?symbols in R for more plotting options. Go wild.

For more examples, guidance, and all-around data goodness like this, buy Visualize This, the new FlowingData book.

Related

• Bars as an Alternative to Bubble Charts
• Fail: Area Circles on Wall Street
• 7 Basic Rules for Making Charts and Graphs