This guest post is by Rahul Bhargava, a Senior Software Engineer at nTAG Interactive, makers of interactive name badges for conferences and meetings. Email him : rahul [ @ ] ntag . com

A common thread in many of the great visualizations Nathan shares on Flowing Data is that they are created for external consumption – someone designs a neat way to represent a dataset to a larger, naive audience. I want to talk about the under appreciated utility of writing quick visualizations for yourself, to help you debug your own complicated or data-dense problems. This is not a new discussion, but I want to remind all the programmers out there that a speedily-created visual representation of your debugging log data might be the quickest way to find your problem! Below are some examples of what we’ve done at nTAG, and some techniques we’ve found particularly useful. Please post a comment about what you do.
Read More

Todd provides 5 reasons why data visualization isn’t more prevalent:

1. People don’t know what data visualization is.
2. Bad visualization has skewed perception of what data visualization is and what it can be used for.
3. People can’t interpret charts or new data representations.
4. Visualization is difficult to create, but easy to copy.
5. People won’t pay for visualization.

While all the reasons do have some truth, there are a couple things worth adding.

People Do Know What Data Visualization Is

People have some kind of idea of what data is and know that you can get information out of it somehow. Maybe it’s with a graph or it could be with something more elaborate, but most people will get it. They know what data visualization is. They just don’t know what it’s called. In other words, they know. They just don’t know they know.

People Will Pay (A Lot) for Visualization

With all the data out there and the constantly increasing volumes of it, more people want to understand without having to learn formal statistical methods. How can they understand it? Visualization of course. The growing number of examples I’ve covered here on FlowingData show that there is a growing demand. After all, a lot of stuff I’ve covered here was commissioned.

Not Too Worried

Anyways, even though not everyone knows about data visualization (yet), I’m not too worried about it. There’s just too much data for people not to care… or am I wasting my time? No. If they don’t care, we’ll show them why they should.

At the bottom of each article on National Nine News (Australian MSN), there’s a button to “Flock It!” which is like favorit-ing a news story.

The more people who flock a story, the higher up the flock list the story goes. In the sidebar of each story is an interactive graphic that shows readers flocking around the news and stories getting highlighted. The larger the bubble, the more people who have flocked it; story bubbles light up orange when someone flocks it. The site isn’t showing any larger sizes, but a full screen version could be fun. Maybe a screensaver.

MSN seems to have have this whole news exploration thing going on lately. I like it.

[Thanks, Andrew]

Today, Kevin Merritt, founder and CEO of blist, provides some background on putting data in the hands of mainstream users.

blist is not a company of modest ambitions. We want to democratize working with data much as PowerPoint and Visio have empowered mainstream users to create their own presentations and diagrams. Before these breakthroughs in innovation, mainstream users sketched free hand and asked professionals in central resource pools (art departments and engineering departments) to turn drawings into foil transparencies and blueprints.
Read More

Matthew Block and Jonathan Corum from The New York Times use a network graph to map diseases and the genes they have in common. Color indicates the type of disease, circles represent diseases, and gray squares are genes that the diseases have in common. The graphic has a nice magnifying glass zooming feature, so that you too can be a biologist.

A quick announcement: I’m headed back to California for a few days and may or may not be online. While I’m gone, I have a couple of interesting guest posts scheduled, so I’m looking forward to reading what you all think when I get back :).

Also, I have two guest post spots left for when I leave on vacation, so anyone is welcome to email me their ideas.

If I were to skip straight to the part in The Shawshank Redemption when Andy Durfesne climbs out of the pipe of poo (and put it on mute), someone who never saw the movie might see an escaped convict who steals money from a warden and fleas to some random place in Mexico called Zihuatanejo. Out of grief, the warden kills himself and Ellis Boyd “Red” Redding eventually teams up with Andy to commit more crimes.

Those of us who have seen the movie though know this isn’t the case. Why? Because we saw the whole movie and have context.

Context Matters

As Andrew, a FlowingData reader, put it, “For statistics to be useful, it needs to be explained in a context.” When I get my hands on some data, whether I’m analyzing or visualizing, I want to know the context of data first. I want to know who collected the data, how it was collected, when it was collected, and what was done to it before it arrived in my hands. Without that meta-information, I could easily make an incorrect assumption about the data or misrepresent it somehow in a visualization – which is very bad.

Simply put, we use visualization and statistics to tell stories with data. If we don’t have all the information, then we can’t tell a complete story.

Thank you to everyone who responded to last week’s poll: What Field of Expertise Do You Study or Work In? At the time I’m writing this, there were 326 responses. While I knew all of you came from lots of different fields, I was surprised by how diverse this group really is, which made me really happy.

Here are the results. I tried to extract some of the “Other” responses from the comments and placed them into new categories.
Read More

NewsWare was launched yesterday on msnbc.com. It’s a set of apps, games, and widgets to interact with the news. The three main points of interest are the Spectra (pictured above) and two games that resemble a couple of popular arcade games infused with news.
Read More

In a deviation from the usual pie chart and standard tree map, this graphic from The New York Times resembles something of a stained glass window – a really pretty piece of work. Amanda Cox, with Matthew Bloch and Shan Carter, designed the interactive graphic that lets you explore how American consumers spend their money.
Read More

Visualization Criticism – A criticism on the criticism on visualization. Robert Kosara, Fritz Drury, Lars Erik Holmquist, and David Laidlaw argue that we need to critique to further develop viz theory.

Data Visualization Talks Online – Talks for your viewing pleasure from the likes of Ben Fry, Eric Rodenbeck, Jonathan Harris, and others. A couple hours of weekend learning.

Why Things Cost $19.95 – An interesting article from Scientific American on the “psychological rules of bartering.” Any guesses on this somewhat arbitrary pricing?

People from lots of different fields have emailed me during the course of FlowingData, and I continue to be surprised by the wide reader diversity. So naturally, I’m interested to know what fields all of you study or work in. If you select the Other option, please do leave your “other field” in the comments, or if you are some mix of everything, I’d be interested to hear about that too.

Of the 23,160 Facebook applications, I use about 5, but I probably wouldn’t notice if someone randomly removed all of them from my profile in the middle of the night. Kids these days. I used to play BlockStar, but haven’t used it since it changed to Tetris (formerly BlockStar) and haven’t played Scrabulous since my 1,000,000th consecutive loss. What Facebook applications do you use (or not use)?

Speaking of Facebook, have you joined the FlowingData group yet?

Inspired by Jonathan Harris and Sep Kamvar’s We Feel Fine, and using data from summize, twistori shows what people love, hate, think, believe, feel, and wish for on Twitter. Given the conversational feel of Twitter, twistori shows an almost natural flow of emotion and like Twittervision, is sort of mesmerizing.

[via Twitter]

I studied electrical engineering and computer science in undergrad and now as a stat student, I still work with a lot of engineers and scientists. Something that has always confused me as I walk through the engineering (and statistics) halls of conference posters is the use of the rainbow color scale.
Read More

Ben Fry maps every road segment in All Streets, according to the U.S. Census Bureau’s TIGER/Line data. There’s no actual map or drawn borders; instead Ben chooses to let the data do all the work, and the results are very pretty. Sometimes you don’t need a map to map.

I was somewhat surprised to see California’s low road density compared to the eastern half of the country, but I guess that’s because of all the freeways. What’s more surprising though is that line down the middle. Roads all of a sudden go dense somewhere around North Dakota. Is that really what it’s like? Does farming suddenly stop and urban life begins in these areas?

Poor Alaska and Hawaii, with too few roads, were left out.

Patrick Ball, a human rights statistician, finds truth in numbers while analyzing and consulting to find patterns and uncover scale in crimes against humanity.

The tension started in the witness room. “You could feel the stress rolling off the walls in there,” Patrick Ball remembers. “I can remember realizing that this is why lawyers wear sport coats – you can’t see all the sweat on their arms and back.” He was, you could say, a little nervous to be cross-examined by Slobodan Milosevic.

Mr. Ball was the first expert witness called in the case against the former Serbian president, who was representing himself against mass atrocity charges at the International Criminal Tribunal for Yugoslavia. Ball had spent 10 months crunching numbers about migration patterns in the former Yugoslav province of Kosovo; his findings suggested that hundreds of thousands of refugees who fled to Albania were spurred by the violence of Mr. Milosevic’s army. By the time Ball entered the tribunal chamber, in March 2002, the ousted leader had a reputation for grand orations rather than direct questions; when Milosevic veered off track, the judge would interrupt. “Milosevic would say, ‘Dobro,’ and go on….” Ball remembers. “It means, ‘OK, very well,’ but it was clearly a, ‘Very well, we’ll have you shot later.’ I hear [that] in my dreams periodically.”

Ball is a statistician – not exactly a profession usually associated with human rights defense. But the Human Rights Data Analysis Group that he heads at Benetech, a technology company with a social justice focus, is bringing the power of quantitative analysis to a field otherwise full of anecdote.

That’s right. Statistics is awesome. I dare you to disagree.

[via Statistical Modeling]

Dan Beech represents worldwide poverty in this video, which is actually a 3-dimensional bar chart with some flare:

Welcome to Povertyville, Slumtown, and Low Income city. I’m not sure what to think. Should I laugh? Should I cry? I don’t know. What do you think?

In this genre of over-produced graphs, Povertyville reminds me of the real estate roller coaster, a dramatic 3-D time series plot:

Early next month, I’m going to be traveling a bit. I’m headed back to California for about a week for some work-related stuff. Soon after, my wife and I will be celebrating our one-year anniversary on some tropical island where I will be basking in the glory of all-inclusive. The following week, I’ll be at the International Summit for Community Wireless Networks.

I’m going to write posts in advance, but I’d also like to feature some high quality posts from FlowingData readers (like yourself) while I’m gone.

What I’m Looking For

I’m pretty open as long as it’s within the scope of FlowingData, but here are some ideas I’m interested in finding:

• Anecdotes on how you use data, statistics, or visualization to discover new things.
• The design process (from data-culling to final product) from those who are working on or who have worked on data visualization projects.
• Tips and tutorials on how to tackle certain types of data.

I’m not looking for heavy promotion of a product (although I don’t mind if you mention it). I want to keep the focus on learning and not so much on buying. Also, I’m looking for original content only. I say this just because I want to stay legit with search engines, so please, no duplicate content.

Email Me Your Post

To submit a post, send it to me via email. Put “FlowingData Guest Post” in the subject line, and put your post in the actual email or a plain text file. No Microsoft Word documents, and if your post is already with HTML markup, all the better.

I’m not really sure how many posts to expect, but I’ll use as many submissions as possible, if not all of them. My hope is that I’ll be able to highlight some more flowing data and as well as help us all learn a thing or two. Looking forward to what you all have in store.