Magically Reformat Data to Get it How You Need it

Posted to Apps  |  Nathan Yau

One of the more painful parts of analysis or visualization is that you have to get the data in a proper format. Real data almost never comes how you want it. Magic/Replace from DabbleDB lets you reformat data via their spreadsheet interface and a few sprinkles of magic. The solution is really quite elegant.

You copy and paste CSV or TSV from a spreadsheet and submit. You then see a column editor and a preview window. This is where the magic happens. In the column editor, you can edit a column so that it fits a certain format and Magic/Replace will show you a preview of what the others will look like. For example, say you have a column of phone numbers and they’re in the (555) 555-5555 format, but what you really want is 555-555-5555. Change a single row, and voila, Magic/Replace does the rest. It really is “data cleanup for everyone” – not that the data were dirty to begin with.

[Thanks, Jose]

2 Comments

  • Looks very nice!
    But IMHO, if you’ve got the time, it’s better to just learn regular expressions and work with a decent text editor that can apply them. Once you get the hang of it, it’s as powerful and fast as anything.

  • It does look nice. Dabbledb look interesting. You may also want to check out Mass Edit from MIT. It seems to do everything at least in the Magic/Replace video and is non-corporate. I came across it quite a while ago when I was checking out their SIMILE project (also some great free tools) and was very impressed http://groups.csail.mit.edu/uid/mass-edit/

Favorites

Reviving the Statistical Atlas of the United States with New Data

Due to budget cuts, there is no plan for an updated atlas. So I recreated the original 1870 Atlas using today’s publicly available data.

Causes of Death

There are many ways to die. Cancer. Infection. Mental. External. This is how different groups of people died over the past 10 years, visualized by age.

Who is Older and Younger than You

Here’s a chart to show you how long you have until you start to feel your age.

Graphical perception – learn the fundamentals first

Before you dive into the advanced stuff – like just about everything in your life – you have to learn the fundamentals before you know when you can break the rules.