Casey Reas and Chandler McWilliams asked visual designers why they write their own software and how it affects their process:
The answers reflect the individuality of the designers and their process, but some ideas are persistent. The most consistent answer is that custom software is written because it gives more control. This control is often expressed as individual freedom. Another thread is writing custom software to create a precise realization for a precise idea. To put it another way, writing custom code is one way to move away from generic solutions; new tools can create new opportunities.
Most of the interviewees are media artists, but there are a couple of names you’ll recognize. My favorite, Amanda Cox, uses a Mad Libs metaphor:
Mad Libs is a game where key words in a short story have been replaced with blanks. Players fill in the blanks with designated parts of speech (“noun”, “adverb”) or types of words (“body part”, “type of liquid”), without seeing the rest of the story. Occasionally, hilarity ensues, but no one really believes that this is an effective method for generating great literature.
I’m looking at you, non-programming statistician.
Update: The article isn’t there anymore, so you can read the cached page for now.
404! :-( Feel free to delete this comment if/when the link works.
Love the mad libs metaphor! We write all of our own code and draw upon (and extend!) open source tools only when it is appropriate to do so. Being both creative and precise is exceptionally important for data scientists, statisticians, and graphic designers in an era where there are ever more black box solutions to choose from.
What would you think if you heard someone say any of these:
* “We maintain a custom in-house word processor because it gives us more freedom and control”
* “We maintain a custom in-house spreadsheet program because it gives us more freedom and control”
* “We maintain a custom in-house image editor because it gives us more freedom and control”
Crazy, right? We take software like word processors and image editors for granted because there are mature tools on the market, and thousands of engineers at Microsoft and Adobe wake up every morning to make them more usable and flexible. The only reason that people in data visualization are forced to write their own software is that the existing GUI solutions uniformly suck. These are fat times for cross-disciplinary programmers, but I think in the future a statistician who can code will be like having a bus driver who knows how to build his own bus.
I think the power granted by being able to program and create novel applications / tools to solve problems will always be a desired skillset to have in this field. The 3 examples you list do have many alternatives that people create and maintain themselves – precisely because the increased control allows these custom tools to be much more powerful then their generic counterparts — in certain situations.
* tools like Jekyll, scrivener, and custom latex platforms allow for automation, collaboration, and other features that Word doesn’t provide and won’t ever provide well
* image processors like custom setups using imagemagick again allow for automation and power in certain situations.
So, I don’t think those statements are all that crazy. Like Amanda Cox says: “generic solutions work perfectly well in some cases”. But I would say that no matter what future tools exist in this realm, coding will always be valuable. This is because a programming language is really just another tool – not much different than Word or Excel. (This is especially apparent of languages like R and Processing that cater somewhat to the non-computer science crowd).
Being able to use these tools effectively is more akin to a bus driver knowing how to fix or improve his bus. Certainly not something every bus driver will be able to do, but a wonderfully powerful skill for those who can. (I was going to say your bus driver metaphor “breaks down” a bit – but I couldn’t fit the pun in anywhere).
I agree with you that there will always be a “long tail” of specialized applications, in data science as in other fields. People who can make their own solutions will continue to add value for fulfilling custom needs.
However, that same quote from Amanda Cox indicates to me that the generic tools have a long way to go. If generic tools were doing their job it would read, “generic solutions work perfectly well in MOST cases”.
I’m just trying to point out that it should seem odd that programming ability is extremely valuable for performing what seem to be mundane tasks. Case in point: the graphic above, an election map with horizontal arrows. With the current set of publicly available tools, that would take an experienced programmer weeks to get that right. They’d have to worry about things like district boundaries, state boundaries, placement of city labels, choice of map projections, linking records, calculating centroids, computing fade effects, and the half-century old question of where to put Alaska and Hawaii. If proper tooling were available, in my opinion, a custom election map like that would be as easy as making a custom Pie Chart in Excel.
In the early days of automobiles it was essential to have a driver who knew a little bit about engines in the not unlikely event that something went wrong. We are in the early days of data visualization, and it is essential to know a little bit about programming in the not unlikely event that the generic tools can’t do the job right. But just as knowing automobile mechanics is no longer a prerequisite to hauling a bus full of kids to school every morning, programming ability is bound to shift from a “need-to-have” skill to a “nice-to-have” skill for practitioners of data science.
google cached version: http://webcache.googleusercontent.com/search?q=cache:_khg5NpLzL4J:www.creativeapplications.net/theory/designing-programs-theory/+site:www.creativeapplications.net+amanda+cox&cd=1&hl=en&ct=clnk&gl=us
Thanks. Not sure why the original isn’t up right now.
@ Evan Miller
It depends on what kind of user you are.
There are many visual (rapid development) programming environments, but they don’t make programming obsolete. In fact, they make it even more important. You very quickly run up against the limitations of the software. What non-technical people can do in it is limited albeit useful in simple situations.
For web design, there’s are visual layout programs but they are also very technical once you want to fine-tune. Moreover, the way code is generated and displayed in these tools makes customizations challenging. For instance, the generic code produced by the application is not transparent to every person, since every programmer or designer prefers their own style, and you spend hours looking for a needle in a haystack. That is why many people still write xml or html or javascript by hand in more structured development environment. It’s simpler.
It also depends on the type of software. Image editors are all unique, which is the beauty of using new tools. Some are very experimental, some are more standardized because of the market they target. Each handles differently, like a different car. One does not fit all. Individual “feel” aside, there is no one piece of software that will ever do everything, just as there is no one brush or surface that will satisfy all painters, just as there is no one musical instrument that will satisfy all musicians. This fields is to a great extent artistic, which is about experimentation.
Word processing or spreadsheets is about function. It’s office software meant for the standard tasks. It’s a completely different class of software. That said, departments that use Word or Excel are littered with customized solutions. So your crazy statements are not that far from the truth. People get into VB scripting, macros, and the like to get the standard software to do what they want. In fact, you’d be amazed by the extent to which some departments go with adding onto Excel or Word (this is not including all the commercial add-on software you can buy to add functionality to office applications).