Frederik Seiffert provides this nifty tool, LastHistory, to visualize your Last.fm listening history. Mouse over songs and find repeated track sequences. The visualization itself isn't all that useful, but it gets interesting when you hook your calendar and photos in with music. LastHistory lets you replay songs synched with your photos, and your slideshow suddenly gains a new dimension.

I don't use Last.fm, so LastHistory isn't much use to me, but I'd be interested in hearing what others think. Give it a try for your self though (for Mac only), and leave your thoughts in the comments below. Is it a reason to start using Last.fm?

World Progress Report - 6 days left

Open data is great, but it's useless if you don't know what to do with it. Sunlight Labs, a group focused on using technology to support open government, recently released ClearMaps. It's an Actionscript framework for interactive cartographic visualization.

In addition to giving designers and developers more control over presentation the project aims to address some of the common technical challenges faced when building interactive, data driven maps for the web. ClearMaps is designed as a lightweight, flexible set of tools for building complex data visualizations. It is a framework not a plug-and-play component (though it could be a starting point for those wishing to make reusable tools).

It's still in the early stages, but developers will want to check this out I am sure.

[Thanks, Kevin]

World Progress Report - 6 days left

What if you could see all the individual bits of information scattered across the Web in one view and then interact with it in a meaningful way? This is what Microsoft Live Labs' new Pivot experiment tries to do.

Pivot makes it easier to interact with massive amounts of data in ways that are powerful, informative, and fun. We tried to step back and design an interaction model that accommodates the complexity and scale of information rather than the traditional structure of the Web.

The goal is to let users make connections between pages, data points, photos, etc that go beyond links, with what the developers call collections. The below video is a demonstration and explanation:

Pivot's ability to display lots of thumbnails and then reorganize and zoom in on them is the tool's foundation. The transition between each view involves a flutter of thumbnails, which sort of provides a link between data arrangements. The browsing behavior looks a lot like that of Photosynth, a Live Labs project that lets you browse giant bundles of photos.

Jeffrey Heer et. al. wrote a paper on these transitions a while back. I can't really say whether it works or not. I suspect it's more about a fun factor once you get into higher volumes of data than it is about making connections. That's not to say it's not important, of course. After all, most of the Web is about entertainment in some form or another.

All in all, it's an interesting concept, and it will be fun to see where the Live Labs team takes the project.

Pivot is currently by invitation only, but I have a handful of invites (10 to be exact) for you guys. Download Pivot from here, and then use this activation code: 3C5D 19BD B7DA 3186. Come back here and let us know what you think in the comments.

[Thanks, Jeff]

World Progress Report - 6 days left

Like it or not, Google Maps mashups continue to be a Web favorite. It's just so easy to use. Stick a few lines of javascript in your web page, and voila, you've got an interactive map. That's for point-wise data though. It gets a little trickier beyond "you are here" pointers. Cartographic.js, in its first release, aims to make thematic mapping with the Google Maps API easier.

The above is a bubble map for Twitter user locations, and below is a simple choropleth demo.

There's nothing earth-shattering here, but the above only required a couple lines of code, which I think is something every developer likes to hear.

World Progress Report - 6 days left

What It Is

RoamBi is a free data visualization application for the iPhone by MeLLmo. You download datasets to the app and it creates visualizations so you can drill down into the data. The app is pitched as a mobile business tool for viewing sales reports and the like, but the sample visualizations included with the app suggest another possibility: RoamBi could easily be a killer app for statistics-minded sports fans, such as sabermetrics devotees!

Visualizations

RoamBi gives you several ways to visualize a dataset and they all look gorgeous:

There is the Cardex, which is a Rolodex view of your dataset where you actually flip through cards, with each card being a row in your dataset. I imagine is a useful way to view business reports but it is hardly a sophisticated visualization.

The PieView is a pie chart with slices that can be selected to view more information about the segment.

The SuperList is essentially a spreadsheet viewer: the data is displayed in cells and you have the ability to filter rows and display some or all of the columns.

Finally, there is the CataList. This is perhaps the most complex, and most visual, of RoamBi's options. Like the name suggests, it shows the dataset in lists and grouped by category. For instance, one sample dataset has information for the top 50 grossing films. The categories include Total Revenue, Average Weekly Revenue, Weekend Gross Revenue, and Weekend Gross Revenue Per Theater. Within each category you are presented with the list of movies sorted from highest to lowest with revenue over time line graphs for each movie.

What RoamBi Is Not

Perhaps this doesn't need saying, but the RoamBi iPhone app is not a data manipulation tool. Any data set creation must be done elsewhere and then uploaded to a RoamBi server for access from the iPhone app. Also, there is no complex querying mechanism, so your datasets must be relatively small or easily aggregated. This being said, the app does a very good job of efficiently using the iPhone's screen real estate, so you will be surprised at the amount of data you can navigate using the app.

RoamBi.com

RoamBi servers are a key component in using the iPhone app, as they are the only way to create your own visualizations to download to the iPhone app. While RoamBi sells an enterprise server, I imagine most users will use the free RoamBi.com service (called RoamBi Connect within the iPhone app). To create a visualization on RoamBi.com you create a free account and launch the Flash-based creation tool. Once you select a dataset, whether via uploading an Excel or CSV file or via importing a Salesforce.com report, you tell RoamBi the columns and categories of your data. It selects basic defaults and as you edit the parameters the visualization is updated in an interactive simulation of the iPhone app. As a side note, it is strange you cannot share the simulation, as this would be an easy way to share your visualizations across the web to people who do not have iPhones and it shouldn't require any additional work on MeLLmo's part.

Unfortunately, I found the RoamBi.com visualization process somewhat obtuse and often was unable to assemble and group my data into the visualization I wanted. For instance, I was unable to duplicate RoamBi's sample movie sales visualization, despite importing the same raw dataset into my own CataList visualization. However, I believe someone more experienced with Excel or similar tools will have more success.

Conclusion

I see the RoamBi iPhone app as a tool to view datasets, rather than examine data visualizations. Perhaps this is peculiar distinction, but I mean that you are never far from the original dataset and cannot do sophisticated things to the data either mathematically or visually. However, for simpler data visualizations RoamBi is a well-executed and free product that let's you easily drill down into your data from your iPhone. If you have a dataset you're looking to share on the go, perhaps at a trade show or conference, and you have an iPhone, you should give RoamBi a try.

Have you tried RoamBi on your iPhone? Tell us what you think in the comments below.

World Progress Report - 6 days left

But why not use Python for visualization too? That way you can have everything in one language and all the gears can fit together a little easier. Beginning Python Visualization (BPV) by Shai Vaingast is a guide to help you do this.

While you might need a little bit of programming experience to fully make use of this book, Vaingast provides plenty of examples and explanations for you to easily learn how to use Python's visualization options.

What Beginning Python Visualization Covers

I should back up a little bit first. BPV doesn't just cover the actual visualization options. It starts from the very beginning (well, close to it at least) - at the data. It goes into some about how to collect data and then goes deeper into reading, parsing, and formatting your data once you've got it.

There are a lot of online tutorials on how to do that stuff, but it was good to have everything in one place and to find out the right way to do things from an expert.

After the data munging section, BPV gets into the visualization topics you'd expect. It starts with your basic graphs and how to draw in Python. It then covers more scientific visualization with splines, interpolation, and some signal processing.

You can look at the table of contents on the Amazon page for more details on book content.

Who this is For

BPV is for a semi-technical audience. If you have absolutely no programming experience, then this book probably isn't for you.

My main reason for saying this is not because the examples are confusing - they're pretty straightforward; rather, a lot of the code and explanations in BPV rely on Python packages like SciPy, NumPy, and Matplotlib. That means you have to install these before you can go through the examples, which took a while for me to get working, but maybe that's just me.

The Bottom Line

Once I got the packages installed correctly though, it was smooth sailing. Everything is explained clearly, and I'm sure I'll be referring back to it as I use Python more.

Bottom line: If you're a coding beginner, this book could be useful to you if you can get someone to install a few Python packages on your system. If you do know a little bit of Python though and want to use it to visualize data, Beginning Python Visualization is a good book to have that can serve as a guide and a handy reference.

UPDATE: Get a free copy

Leave a comment below by this Sunday at 8pm EST, and I'll select someone at random for a free e-copy of Beginning Python Visualization. Make sure you use a valid email address so I can contact you if you win. Thanks, Apress!

World Progress Report - 6 days left

As we've seen, javascript is growing into a viable solution for visualization on the Web. John Resig ported Processing to javascript about a year ago and we saw some projects in javascript to show off speed in Google Chrome.

Most recently, Nicolas Garcia Belmonte released version 1.1 of his InfoVis Toolkit, which provides a basic set of tools for creating interactive visualizations on the Web. Some of the features:

• Multiple Data Representations - Treemaps, Radial Layouts, HyperTrees/Graphs, SpaceTree-like Layouts, and more...
• Major Browsers Support - IE6+, Firefox2+, Safari3+, Opera9.5+
• Open Source - Licensed under the BSD License
• Library Agnostic - You may use the JIT with your favorite DOM manipulation framework
• Extensible - All visualization classes are mutable, so you can easily add/override any method you want.
• Composable - Visualizations can be combined in order to create new visualization methods.

Here are a couple of things you can do with the toolkit:

Visit the site for more demos.

World Progress Report - 6 days left

Axiis, an open source data visualization framework in Flex, was released a few days ago under an MIT license. I haven't done much in Flex, but from what I hear, it's relatively easy to pick up. You get a lot of bang out of a few lines of code. Axiis makes things even easier, and provides visualization outside the built in Flex graph packages.

Axiis gives developers the ability to expressively define their data visualizations through concise and intuitive markup. Axiis has been designed with a specific focus on elegant code, where your code can be just as beautiful as your visual output.

Above is the wedge stack graph. Here's your standard area graphs:

See what other visualizations you can create with Axiis here.

Visualization Toolbox Continues to Grow

I remember when I was first getting into data visualization like all 3 years ago, and all I knew was Excel and R. My toolbox grew a little bit when I discovered this small PHP library called, oh I don't know, sparklines or something like that. Fast forward to now, and we've got so much at our disposal in javascript, PHP, Python, Actionscript/Flash, Flex, Silverlight, Processing, and the list goes on and on. Data.gov just launched, Freebase is growing, infochimps is doing great work, and heavy computing resources are now available at low cost to individuals. I keep saying this, but this is exciting times for data. Seriously.

World Progress Report - 6 days left

In the promotion of its speedy javascript, Google announces the Chrome Experiment. As part of the Experiment, design group Use All Five give us Small Talk, which is a social weather map that uses tweets that contain terms like rainy and sunny. Circles are sized by number of tweets, and tweets are colored by dominant weather tweet, so what you get right now is very blue on the east and sort of orange in the west. Oh how I long for the sun.

The cool thing about this (and the other projects from Chrome Experiment) is that it's implemented in javascript.

Pan and zoom...

Click on the bubbles...

Yes, javascript just keeps getting faster and more impressive. It's no longer just a way to show dynamic status messages and popups. It's much more than that. Javascript is becoming a viable visualization solution.

[Thanks, Levi]

World Progress Report - 6 days left

This is a guest post from Elad Israeli and Roni Floman of SiSense, which specializes in easy-to-use business intelligence.

Pundits joke that Google Adwords is driving Microsoft Excel sales. Two rivals are vying for domination; yet one's desktop software is used to optimize keywords sold by the other.  The reason is very simple: the Google AdWords interface doesn't support the rigorous analysis of multiple AdWords keywords and their optimization. Importing the Google AdWords data into Excel lets you do just that… albeit within the constraints of Excel.

Let's try to explain this by looking at the visualization and business intelligence assumptions behind the Google use case and the Microsoft use case.

The Good News

The Google Analytics and AdWords interfaces have gone a long way towards democratizing the de-facto use of business intelligence practices and good visualization analysis approaches (we'll ignore the occasional pie chart). Not only does the small business have access to millions of eye-balls through Internet Advertising; it also gets a good lesson in business intelligence, succinctly visualized.

Every non-beginner Google AdWords or Analytics user knows what a dimension and a measure are. These two concepts, key to multidimensional analysis, will soon become household names, courtesy of Google (well, almost). Custom reporting in Google Analytics assumes that once you've become acquainted with these terms you can now use them for better reporting, using the custom reporting feature. With the addition of motion charts every user now has access to the visual expression of very complex (time-based) relationships.

The visual that drives the Google universe (although the goal conversions funnels are pretty cool) is the Time Series Chart. Anyone who wants as much traffic as possible into their site is bound to be mesmerized by the Time Series Chart. This includes the author of this article, the publisher of the article, everyone with an online presence. "Hey – look at the spike last Monday, at lunchtime! People must have had their lunch while looking at my site…" Time series charts are what helps you look for trends and see beyond seasonal fluctuations. Time series, together with unstructured data, can be even more helpful, since you can tie the unstructured events (thanksgiving, a press release) to the data and see whether the data may explain some of the seasonality. Google is beginning to hint at this with the notes feature. Posting the unstructured data (press release, new ad campaign) can help you correlate that event to pattern changes in the clicks, visits or any other measure you're tracking.

The Bad News

So now we know why there's always that scraggly time series chart indicating traffic, clicks or conversions at the top strip. It is a great solution visually, but it is not good enough to do the continuous improvement work you need to do to optimize your web site traffic or adwords campaigns.
The reason is really simple if you think about it: optimizing web site traffic or ad campaigns can't be assisted at all by the time series view. It needs a drill down. And a good drill down can be defined as follows: compare as many items across as many measures and dimensions that you can. And this is exactly why you'll find yourself migrating to excel….

Google did a formidable job of delivering valuable reports. But the ability to drill into pivot-type views (which are best for optimization work) just takes too many steps. And, as we're going to argue in this article, pivot type views are best when optimizing your AdWords campaigns.

Typically, data is flat, meaning that it consists of columns and rows. A pivot table can help you quickly summarize the flat data, giving it depth, and get the information you want. It does this by offering another level of analysis, across rows and columns. The usage of a pivot table is extremely broad and depends on the situation. To get the right depth, you need to know what you’re looking for, and in an AdWords context, the question you'd like to answer is "which of my keywords/campaigns is most valuable to me?" or, in other words, "how can I rank my keywords, campaigns or traffic sources against each other and compare across the broadest metrics".

Even with the recent Google release of custom reporting, you can see how certain measures behave over a dimension but not how several dimensions or measures interact. So Google's new custom reporting feature gives you a very basic pivot. All the defined segments still retain the rigidity of the underlying Google Analytics structure. You can define a segment as an and/or or and+or but not a measured new dimension (a brand new x divided by y), even a simple one such as cost/conversion.

Below you can see the Google Analytics custom report interface – it lets you define what you want in the report (I'm using it instead of directly referring to adwords interfaces).
You can define dimensions and get to see measures that apply to them. Yet, you still need to do quite a lot of drill down and sorting to effectively compare campaigns. The reason is that you cannot compare dimensions side by side – you can't compare the keywords within each campaign, for instance. In Google you cannot see the top keywords (a fliter) per each campaign. You need to always drill down and sort.

This is the custom report. Comparing campaigns still requires drill downs, not all the information (such as filters) appears. Also, without any measured values we cannot see what a measured value (x/y) for each campaign is, although it is an important measure. So we cannot look at a good KPI, such as looking at the average cost of the worst keywords, which can be an important piece of information when you want to whittle out the bad ones. You can measure yourself by requiring a lower average cost of bottom twenty keywords for every week that goes by.

So you drill down in Google, and define analytics, and use all the reports AdWords gives you, and end up exporting to excel, and using the good old pivot.

World Progress Report - 6 days left