With all the new technologies we’ve come to rely on, it’s easy to forget just how much data we’re automatically logging on our own devices or some central server in the boonies.

GPS is one such example. Some of us can’t imagine going out of town without it. What you might not know is that while that GPS device tells you where to turn left, it is also storing where you go in its memory. Scotland Yard has started using this data to solve crimes:

Scotland Yard analysis of the [GPS] devices has helped solve dozens of investigations into kidnappings, grooming of children, murder and terrorism. Information about a suspect’s whereabouts at particular times, their journeys and addresses of associates can all be discovered – if they have been using a GPS. The devices retain hundreds of records of locations and routes in their memory.

So all you criminals out there, make sure you use GPS whenever possible. We all know your actions are a desperate cry for attention.

[Thanks, Tim]

Like what you see? Subscribe to the feed to stay updated on what’s new in data visualization.

When I saw Toby’s Walmart growth video a while back, I was intrigued by what other time-location data Freebase had. A few commented on how it’d be interesting to map the spread of Starbucks along with Walmart and other businesses. I agreed. So I looked, but as it turns out, there’s not a whole lot of opening dates for business other than Walmart. In fact, about 2/3 of the Walmart locations don’t even have dates. Sigh. Maybe another day. Instead, I used the Walmart data as a learning exercise.
Read More

A little over a week ago, I was in Bremen for the Data Viz VI conference. Read that Data Viz 6 – not Data Viz V.I., as I thought through the first three days.

I asked, “Is this the first one of these?”

“What do you mean? This is the sixth one. That’s why it’s called Data Viz SIX.”

“Ah, ok, I did not get that.”

Anyways, Adalbert and company put together an excellent conference, and I’m glad I was lucky enough to attend. It was the absolute best statistical conference I’ve ever been to. That’s saying a lot, because it’s the only statistical conference I’ve ever been to. But seriously, it was a good conference.

Looking Backward, Looking Forward

Michael Friendly opened up with the almost obligatory talk on the history of statistical graphics and where the field is headed. Anyone who’s opened up a Tufte book will have seen a lot of the examples he’s used (e.g. Napoleon’s march and John Snow’s map), but the history behind some of the graphics was interesting. Sometimes statistical graphics tend to lose that back story and becomes all about the values, so it’s always nice to hear the human part of datasets.

Visual Analytics Tools for Analysis of Movement Data

My ears perked up when I saw “analysis of movement of data” in Gennady Andrienko’s talk. I work with a lot of GPS data. I was reminded of the many ways to split up spatio-temporal data – by geographic section, by chunks of time, etc. It’s easy to get caught up in the literal GPS traces on the map, so the talk was a good reminder. I do, however, wish Andrienko used more dynamic examples and branched out from Google Maps as the primary mapping tool. This was probably because his work is more computation-heavy than focused on interaction. Because of that, I was left wanting more than I got.

GGobi for Exploratory Data Analysis

I had the chance to chat a bit with the group behind GGobi, an exploratory tool that lets you “tour” multidimensional data via different projections. (That is one nice group of people, let me tell you.) Off the top of my head, there were four separate talks from the group, showing the various applications GGobi can be applied to. It’s kind of hard to explain in brief, so I’d encourage you to check out the free software from the GGobi site. If anything, it’s fun to see your data move ala John Tukey.

Parallel Coordinates – Good or Bad?

Al Inselberg promoted parallel coordinate plots (PCP) as the ultimate of statistical graphics. I got the sense that not everyone feels the same way. I remember during my second quarter as a graduate student, I proposed PCPs for a project. I was quickly rebuffed with a no way, those are horrible, and I simply moved on. After getting a personal demo from Inselberg though, I might have to take another look. Although, PCPs are certainly no panacea.

Collaboration Wanted

Still, my main take away from Data Viz VI was the need for collaboration between design, computer science, and statistics. As we’ve seen on FlowingData, there’s a lot of great visualization coming from all three camps, but I wish there were more collaboration between all. As Di pointed out, this can sometimes be difficult because statisticians need certain tools (i.e. R) to be tightly coupled with whatever visualization they’re developing. But outside the pure analytical tool, I see a sweet spot at the epicenter of statistics, design, and computer science, which is certainly something to get excited about.

For those who want more out of the commonly-used mapping APIs from Google, Yahoo, Microsoft, etc, but don’t want to get too heavy on the programming, Mapstraction is for you. Mapstraction is a javascript mapping abstraction library that lets you easily use different mapping APIs all at once (or switch between them).

This means you can use functionality from one API and apply it to another, or you can just put a whole bunch of synced maps on one page like above. Other features include geocoding, polylines, marker filters, and GeoRSS and KML, so go for it. Go map crazy.

[via ReadWriteWeb]

The New York Times shows how presidential candidates have spent more than $900 million so far with this bubbly graphic by Lee Byron, Hannah Fairfield and Griff Palmer. The area of a circle represents the amount of money spent in any particular category. For example, the biggest chunk of funds ($337 million) was spent on media and consulting.

I know what a lot of you are thinking and are maybe even about to write something in the comments – “Bubbles suck at showing amount. Bars are much easier to read.” Some might even be thinking about a pie chart in lieu of the bibbly bobbilies. Here’s what I have to say: the bubbles are fun, so mission accomplished. That is all.

The Girl Effect – “the idea that adolescent girls are uniquely capable of raising the standard of living in the developing world” – is portrayed in this beautiful video using animated typography. I think the music plays a pretty big role in making this work too.
Read More

It’s July 4th weekend which means lots of burgers and hot dogs across America. It also means it’s time for Nathan’s annual hot dog eating contest on Coney Island. From 2001 through 2006, 144-pound Takeru Kobayashi dominated the competition, but last year Joey Chestnut brought the crown back to the states with 66 hot dogs and buns (HDBs) in 12 minutes. Who will take the crown this year? Will Kobayashi reclaim the title or will Chestnut keep it in America? Oh the suspense.

Take a look at the history of the event – dating all the way back to 1916.

Earlier this month, I announced FlowingData’s summer project for personal data visualization. It’s been a lot of fun seeing what all of you have sent in; it’s like tidbits out of your lives that you’ve decided to share with me. Really fun. Thank you everyone who’s sent stuff so far.

A Few More Details

For those who haven’t put in your entry yet, the deadline is September 1, 2008, so you’ve still got time. For those who already sent me something, remember, you can send me as many as you like. There’s a $40 Amazon gift certificate at stake here – plus personal introspection and a new understanding of your behaviors and patterns.

Your entry can be pictures, graphs, pencil sketches, Illustrator sketches, Flash or Processing projects, or whatever you like. The only real requirement is that it’s about you or what’s happening around you. I eagerly await your flowing data. Email me your entries with “summer project” in the subject line.

Recent Reader Entries

For some inspiration, here’s a subset of the several recent reader entries that I’ve enjoyed so far.

Tim has been keeping track of his aches and pains:

Stacy and Joel recently had a get together (which sounded like a lot of fun), and everyone drew their connections to the others at the party:

Kevin sent a snapshot of his profile from his in-development web application, BedPost. He’s keeping track of um, well, you can just read it:

In the time that it takes you to watch this movie, the US government will have spent $500,000 towards the war in Iraq. At least that’s what this Atari-sounding clip says. Watch as millions of dollars are put into perspective – 84 brand new schools, a flag pin for every man, woman, and child in America, and a hummer plus 10 years of gas.
Read More

The colmeia group recently installed their Caffeine Viewer project where they hacked their coffee maker to log their “insane coffee consumption” in real-time. Every time a person presses a button on the coffee maker data are logged, but there’s a slight twist – the data are available to everyone via the caffeinated API. That’s some serious self-surveillance. There are also a few visualizations, but mainly, they invite others to create their own.
Read More

Whaaa? Cool beans.

As you might have noticed, I haven’t been live blogging the Data Viz VI conference here in Bremen. I arrived Tuesday evening and on Wednesday, the first day of the conference, I woke up at 9:00am (which is midnight PDT), and my body said, “Nathan, I hate you. Go back to bed.” I said no, and now I’m being punished. That’s pretty much how it’s been.

The actual conference, however, has been really interesting. Di Cook demoed GGobi via high school dropout salary data; Michael Friendly gave a nice talk on the golden age of statistical graphics; Gennady Andrienko talked a bit on clustering spatio-temporal data; and there have been plenty of other interesting ones in the mix. One criticism – Minard’s map, showing the march of Napoleon, has been mentioned at least five times. Enough already.

My Talk

I gave my talk on visualization for self-surveillance. I felt slightly off-topic talking more on design than on traditional statistical visualization, but no one threw any tomatoes at me, so that’s okay. The emphasis was on collecting data about ourselves, looking for patterns, and gaining some insight on the way we live with my current project as the case in point.

Animation in R

Yesterday, Andreas Buja got the audience’s attention by using R for animation. He used R to show fishing boat activity off the Pacific coast simply using getGraphicsEvent(). The coding syntax was very similar to Actionscript where there is a listener, and when an event fires off, a function is called. For example, you can tell R to do something when the user clicks on the mouse. The animated map amazed a lot of people. I was mildly amused.

Design and Statistics

I’ve always known about the big divide between statistics and design for data visualization, but I didn’t really know how big the gap was until now. For example, Processing, which is the default tool for a lot of designers, is foreign to statisticians. At the same time, most designers have never touched or heard of R. From where I sit, I see two separate worlds trying to do the same thing – tell stories with data. Both sides have much to learn from the other. They just don’t know it yet.

This is not to say that the two haven’t done great things separately, because they have. But the potential is high when they merge. Throw computer science in there, which has found it way into seemingly everything as a necessity, and you’ve got something good on its way.

With it being FlowingData’s birthday, it seems like a good time to get some input from all of you. FlowingData isn’t just a personal blog for me anymore. It’s for all the readers too, so I’d love to know what you all are interested in hearing about. If what you’d like to see isn’t one of the poll choices below, please do leave a comment.

It’s been one year since my rambling post on creating effective visualization. It seems so long ago. What was I thinking? Since then, FlowingData has grown to 313 posts, 801 comments (plus tens of thousands of spam), and 2,600+ subscribers, and continues to slowly climb the ranks on Technorati.

It was exciting when FlowingData hit the 1,000-subscriber mark back in March and even more so when some really big blogs linked here and FlowingData was on the front page of del.icio.us. Who would have thought data visualization and statistics was so popular? I certainly didn’t know – which was why I started FlowingData in the first place.

Most Popular FlowingData Posts

I’ve featured a lot of great data visualization and statistics over the past year by some very intelligent and talented people. Here are the 10 most popular posts over FlowingData’s short one-year history.

1. 17 Ways to Visualize the Twitter Universe
2. Showing the Obama-Clinton Divide in Decision Tree Infographic
3. 10 Largest Data Breaches Since 2000 – Millions Affected
4. Ebb and Flow of Box Office Receipts Over Past 20 Years
5. 21 Ways to Visualize and Explore Your Email Inbox
6. Chart of the Day: A Breakdown of Facebook Applications
7. Love, Hate, Think, Believe, Feel and Wish on Twitter
8. 6 Influential Datasets That Changed the Way We Think
9. Area Codes in Which Ludacris Claims to Have Hoes
10. How to Learn Actionscript (Flash) for Data Visualization

Thank You

Thank you everyone for reading, linking, and suggesting topics. The blog wouldn’t be the same without you. We’re well on our way to reaching 5,000 subscribers. If you know someone else who’d be interested in FlowingData, please do pass the word along. I’ll super appreciate it.

Happy birthday, FlowingData!

For the next few days, I’m in Bremen, Germany for a conference on statistical graphics. The official name is Statistical Graphics: Data and Information Visualization in Today’s Multimedia Society.

While unsure about some conferences, I’m about 100% sure that I will enjoy this one. The schedule looks very promising with speakers whose papers I’ve read, but have never had the chance to meet. I’ll be speaking on data visualization for self-surveillance and personal discovery, but mostly I’m interested in what I’m going to learn the next few days.

EuroCup 2008 is also on the schedule. It’ll be my first European soccer viewing experience. Very exciting – especially since Germany is in the semi-finals.

Live Blogging the Conference

I’m going to try to live blog the event (the conference, not the soccer). There are 35 talks over the four days, so I imagine I’ll be note-taking like a fiend. However, a fare warning – there is a chance the jet lag will get the best of me. Just a slight chance.

Mark Coleran has hands down one of the best jobs in the world. He makes infographics for feature films. His résumé includes Mr. and Mrs. Smith, Lara Croft Tomb Raider, The Island, Harry Potter and Blade 2. The infographics don’t have to show real data; they just have to look cool. Well, I’m sure that’s not all there is to it, but I bet awesomeness is a leading requirement. Coleran fills it well.
Read More

FlowingData on Alltop – Alltop describes itself as the digital magazine rack of the Internet collecting stories from “all the top” places on the Web. You’ll now find FlowingData on both the Design and Science racks. While you’re there, check out all the other cool sites.

Excel Contest for Science and Engineering – Jon Peltier, a frequent FlowingData commenter, is running a contest on modeling science and engineering. The key phrase is – A winner will be drawn at random.

Video Game Addicts Not Shy Nerds – A study “showed” that only 1% of problem gamers (in their sample) had poor social skills. What a load off my back.

Surveying the Family Feud Surveys – The WSJ Numbers Guy takes a look at the 100-person surveys on the long-running game show. Survey says?!?

As intelligence goes up, happiness goes down. See, I made a graph. I make lots of graphs.

— Lisa Simpson. The Simpsons. Episode 257. January 7, 2001.

This organic visualization, code_swarm by Michael Ogawa from UC Davis, has been making the rounds on the Web lately, and rightfully so. The data: history of commits to a software project. However, instead of focusing on the actual code, the spotlight is on the relationships between developers and their code.

Watch as developers commit code to the repository, the types of files they commit, and watch the life-like organism grow. Below is a video demo of code_swarm that shows the development of the Eclipse IDE:

The way code swarms, flashing and zooming towards its developer, provides a very human aspect to something that can often feel cold, mechanical, and lifeless. Just one of the many reasons why I love data visualization.

[Thanks, Simon]

Last month I asked FlowingData readers, “What are your favorite data visualizations in recent memory?” I’d heard of some while others were brand new to me. Here are some of your responses.

Richard said, “Hans Rosling, no question.” Of course referring to the famed Gapminder.

Tom said, “I’m really liking [Akamai] right now.” Srikanth replied, “That one is pure awesome.”

Srikanth also liked Lee Byron’s Daylight.

Tony said, “Definitely this one about Manny’s quest for 500 homers!”

Chris provided two of his favorites, Flickr Galaxy and Life of a Cell. “The Flickr galaxy awesome, showing a great user interface and a glimpse of 3d on the web… and I’m also a big fan of the ‘Life of the Cell’ video.”

“I’m a big fan of the Baby Name Voyager… simple, attractive, interactive, informative, elegant” says CTV.

“Nice use of Google Chart API,” says Clint.

Tim said, “The best I’ve seen in recent years.” I agree.

Thanks to everyone who responded to provide us all with some eye candy (and a bit of humor).