A different analytical wall

Oct 14, 2010

In reference to the wall between reporting data and understanding it, Martin Theus proposes a different one:

Once you start to explore the data, the whole thing stops to be linear but gets to be very iterative, jumping over the wall every now and then. I.e., you may find out that the data cleaning is insufficient, or the model you have in mind needs some other transformation of the data, or you might want to collect additional or other data altogether.

The wall does exist, but I think it is more separating two kinds of people / thinking.

Theus finishes:

One thing is for sure: we won’t succeed if analysts continue to build useful but technically insufficient tools and computer scientists still build fancy tools that merely help the analysts.

Or even better: analyst and tool builder become the same person. That’ll take much longer though, so communication is a good place to start.



  • > Or even better: analyst and tool builder become the same person.
    That is a good approach, but the commonly used tools come from mega companies like IBM, SAS or SPSS, having loads of programmers who never spoke to a real user, and sales personal who care for their bonus more than the customer’s needs.

    There is hope though, when you look at the rise of R. High quality and managed open source projects might be the solution for the future.

  • Business Systems Analysts come close to a resource who can bridge the gaps between, business, data, and tool requirements. Truth is, every environment is different. In one environment an MS Excel solution where the most complex feature is advanced filtering may suffice. Others may demand pivot table functionality. Others may need a cube containing aggregates from a 50,000,000 data source – and drill through to boot. Obviously that’s the reporting side. Advanced analytics is another story.

    The reporting/analytics domain requires experienced resources who learn quickly, are passionate, can communicate, ‘understand’ how tools work, and not be afraid of jumping in to learn a new technology when required.


Top Brewery Road Trip, Routed Algorithmically

There are a lot of great craft breweries in the United States, but there is only so much time. This is the computed best way to get to the top rated breweries and how to maximize the beer tasting experience. Every journey begins with a single sip.

Divorce Rates for Different Groups

We know when people usually get married. We know who never marries. Finally, it’s time to look at the other side: divorce and remarriage.

Shifting Incomes for American Jobs

For various occupations, the difference between the person who makes the most and the one who makes the least can be significant.

A Day in the Life of Americans

I wanted to see how daily patterns emerge at the individual level and how a person’s entire day plays out. So I simulated 1,000 of them.