I had to install a new caching implementation a few days ago to appease the hosting gods. Since then I've gotten a few emails reporting garbled text instead of an actual page. However, I haven't been able to repeat the problem, and those nice enough to notify me said it went away after a few clicks. I tweaked a little bit and think it's not an issue anymore, but if anyone sees garbage, please do let me know. Thanks.
This is a guest post by Simit Patel of InformedTrades, which offers free advice on trading stocks.
While many investors use economic and fundamental factors to identify investment opportunities -- i.e. whether a company has good management and is in a growth industry, or how it will be affected by macroeconomic conditions -- ultimately the price of an asset comes down to two things: supply and demand. The demand for buying vs the demand of selling. By visualizing the movement of price assets, we can gain an understanding of the psychology of the market as a whole, and thus what direction the price will go.
The U.S. election is over. The post-election analyses begin. The above map shows presidential voting at the county level. The more red a county is, the stronger the support for John McCain and similar for Barack Obama and blue. Below is cotton production in 1860. Each dot represent 2,000 bales. That's some strong correlation. In fact, here is the election map with the cotton overlay:
This of course is a case of strong correlation - not causation. That is to say, if you get your county to grow more cotton, it doesn't mean that you're increasing the probability that voters will sway towards Democrat. As Strange Maps points out, it is in fact a correlation to African-American population (of which 91% voted for Obama). There has been some migration during the post-slavery area, but families have largely settled in the areas their families before them grew up in.
[via Strange Maps | Thanks, Albyn]
Jeff Clark of Neoformix has been doing some cool stuff with words lately. Above is a word portrait of Albert Einstein looking very chipper. Einstein's entire face is composed of the word "genius" at varying shades and sizes. Inspired by Gui Borchet, briefly explains the process done in uh, Processing:
The Word Portraits that I have been creating lately use an algorithm that analyzes a starting image and finds rectangular patches of a reasonably consistent color. These are then filled in the generated image with words or letters painted with the average color in the rectangle.
The algorithm can of course be generalized to not just words and can be used with non-human images as well. Ginger the Cockapoo serves as the case study in which Jeff reconstructs an image of the dog with rectangles, the letter O, leaf-like shapes, and filled circles.
Take a look through Jeff's other postings for more word portraits of Barack Obama and George Boole - inventor of a logical calculus of truth values.
Remember the awesome interface in Minority Report? You know, the one where Tom Cruise is sifting through files and information as if he were directing a symphony? Oblong, whose co-founder served as science adviser on the Steven Spielberg movie, created something a lot like it. It's called g-speak.
Oblong Industries is the developer of the g-speak spatial operating environment.
The SOE's combination of gestural i/o, recombinant networking, and real-world pixels brings the first major step in computer interface since 1984; starting today, g-speak will fundamentally change the way people use machines at work, in the living room, in conference rooms, in vehicles. The g-speak platform is a complete application development and execution environment that redresses the dire constriction of human intent imposed by traditional GUIs. Its idiom of spatial immediacy and information responsive to real-world geometry enables a necessary new kind of work: data-intensive, embodied, real-time, predicated on universal human expertise.
Here's the impressive demo reel:
Now here's the Minority Report clip for comparison's sake:
Of course g-speak is still in development and has a lot of work ahead before it's useful to explore "massive datasets" but it's a good first step nevertheless. Plus, it just looks fun to play with. I wonder what it'd do if I gave it an obscene gesture.
Martin briefly discusses a presentation at a recent visualization workshop. The speaker blurts, "I don't care about the data, I am just interested in the method." This begs the question
Can you design worthwhile visualization without worthwhile data?
I can see why the speaker said what he did, but you know what, if you don't care about the data then I probably won't either, and most likely, I won't care about your visualization. What do you think? Can useful visualization techniques come out of using whatever datasets?
I asked the same question on Twitter a couple of days ago. Here are a few of the responses:
@skylark64: you can, but shouldn't... Then again, maybe it is worthwhile to someone.
@vrypan: But that's the question in the first place! "what's my data worth?" If you know the answer, tools have little importance.
I think I know where this conversation is headed.
The title reads, "Barack Obama is going to appoint the nation's first CTO. What are the top priorities?" I don't know about you, but I'm putting in my vote for open government data. Who knows if anyone from Washington actually sees this (probably not), but can you imagine how fun it'd be to have APIs to data that defines the United States, or any country for that matter? Something more accessible than thousands of scattered Excel spreadsheets and PDF files? It's good to hope, or as my friend Andy Dufresne would say, hope will set you free. Yeah, that makes absolutely no sense. This is what happens when I stay up all night long.
[via Boing Boing | Thanks, Georgina]
The thing about cartograms is that it's hard to make out what you're seeing. You lose most sense of geography and size comparison is near impossible. They're more of a pretty picture than an analytical tool. Axis Maps proposes an alternative to cartograms, and the example of course uses presidential election data.
Instead of morphing counties so that they are sized by area, Axis uses transparency or more accurately, alpha levels. Uh, wait, is that more accurate? Oh I dunno. Someone correct me if I'm wrong. Anyways, as a result, counties with higher populations glow brightly and those with smaller populations fade into the darkness that is oblivion. I like it. More importantly though - what do you think?
In Google Flu Trends, Google uses related searches to predict flu activity in your area "up to two weeks faster than traditional flu surveillance systems." The above graph shows query-based flu estimates compared against flu data from the U.S. Centers for Disease Control.
We have found a close relationship between how many people search for flu-related topics and how many people actually have flu symptoms. Of course, not every person who searches for "flu" is actually sick, but a pattern emerges when all the flu-related search queries from each state and region are added together. We compared our query counts with data from a surveillance system managed by the U.S. Centers for Disease Control and Prevention (CDC) and discovered that some search queries tend to be popular exactly when flu season is happening. By counting how often we see these search queries, we can estimate how much flu is circulating in various regions of the United States.
Anyone who has a blog can attest to this. No matter how much you tweak your blog's design, you can always find something else to "fix." Maybe it's the spacing between two lines, the color of a left border, or the alignment of a picture. However, after this most recent change in FlowingData's layout - mainly to the homepage and sidebar - the OCD in me can rest easy. Here's a quick runthrough.
The featured post on the homepage melds much better now with the rest of the design. My original goal was to make the featured post stand out, but it stood out too much - like a sore thumb, as my mom would say. I also moved "popular posts" above the fold and got rid of underlining. I'm going for clean here.
No More Google Adsense
I got rid of Google Adsense. Once I removed that big ad block, I felt like the look and feel of FlowingData completely changed. Everything feels so unified now, and I like it - a lot. It's nothing against Google. They provide excellent services (and the monthly check wasn't bad either). It's just that I like having full control over everything that appears on FlowingData, and now I do. FlowingData sponsors, Eye-Sys and Tableau Software, are in the sidebar because I think they're worth a look. Not because Google says they're related to my content.
Welcome to FlowingData's Newest Sponsor
On that note, check out SiSense, FlowingData's newest sponsor. It's analytics software that makes dashboard creation easy (for both online and desktop). There's no code required. It's got drag and drop analysis and charting and easily hooks in data from Excel, CSV, MySQL, Oracle, and Google Spreadsheets. So if you're looking for something lighter weight than Tableau or Eye-Sys and want an easy way to share data amongst your workgroup, then SiSense might be what you're looking for.
What do you think of the new layout? Let me know in the comments.
Vertigo put together a great collection of 672 Obama headlines using Silverlight's deep zoom capabilities. The cool thing here isn't so much the number of headlines or the mosaic of pictures. It's how you can interact with the newspapers' front pages. It's not just a mosaic of thumbnails. You can pan and zoom really smoothly with a roll of the scroll wheel and mouse drag and a click. Zoom all the way in to read the actual articles without it taking forever for high-resolution images to load.
Take a look see at Blaise Aguera y Arcas' TED talk for where this technology is headed:
[via Data Mining]
I don't normally put up job postings, but this opportunity is too cool not to. Stamen Design, in San Francisco, has an opening for a full-time developer to "make their ideas feasible." If you follow visualization on the Web, no doubt you've come across some of their work - somewhere in between analytical and art. There's the Digg Labs stuff, Trulia Hindsight, Twitter Blocks, Cabspotting, and plenty of other fun stuff.
Here's part of the job description:
You'll be working with a small team of designers and engineers who will be looking to you to make their ideas feasible. You're excited by the possibility of cutting and bending data to fit it through the thin straw of the internet. You can look at a source of information and model it as resources, rows and columns, messages and queues. You have the programming experience necessary to write data processors and servers, the system administration experience to inhabit and actively guide a constantly-shifting technical environment of free & open source software, and the patience & grace to grant that PHP and spreadsheets might be appropriate tools when circumstances require the quick and the dirty.
You must have the willingness and ability to discuss the finer points of HTTP, SQL, RESTful API's, response formats and resource consumption. You understand that the perfect is often the enemy of the good, and your pragmatism & flexibility show themselves in functional systems. You can see the connections between technical infrastructure and the interactive design & visualization it supports.
We're less concerned with how long you've worked than with how good you are. You will need to have been paid to do good work; the skill that comes from delivering work for money can't be learned in any other way. You maintain a state of constant learning to keep up with new work in your field, participate in communities of practice connected to your expertise, and experiment with new techniques in personal projects.
Go here for the complete details.
I pledge you, I pledge myself, to a new deal for the American people," Franklin D. Roosevelt told supporters in 1932 while accepting the presidential nomination. When he took office the following year, he spent his first 100 days enacting a dizzying number of reforms designed to stabilize an economically depressed nation. Since then, a president's first 100 days have been an indicator of what he is able to accomplish. In January 2009, the clock starts again.
For each President, going back to FDR, a line represents his first 100 days in office. Each circle corresponds to a significant event. The profiles underneath the timelines start with a notable quote from the President's inauguration speech (except Truman and Johnson, who were sworn in after the death of Roosevelt and Kennedy, respectively) followed by events marked with cute, little icons that show what type of event it was. The % of popular vote, days in office, and political inheritance are there too.
Yeah, it's a crud load of information (presented quite nicely). I hope you have a big monitor.
Cartograms got a lot of coverage in 2004 when Michael Gastner, Cosma Shalizi, and Mark Newman used them to show the Bush/Kerry election results. Naturally, they've put together a similar series of (very red) maps and cartograms for the just past 2008 election.
In case you're unfamiliar with cartograms, they're essentially maps with morphed areas according to some metric. The election series for example are adjusted for population, so that states are sized by population instead of physical area. The drawback of course is that after a certain point, the image starts to look a lot like a Pollock painting.
For those watching the clock and waiting for the weekend to hurry up and start, here's the Diagram Diaries Flickr group. The 654 diagrams should keep you occupied for a good while. Enjoy, and have a good weekend, everyone.
Sprint, in a promotion to their mobile Internet service, created this amusing futuristic dashboard. "All aboard the now machine," the computer says. "How about a big bowl of now?... Please keep your hands inside the moment...your hair has grown 5 millionths of centimeter in the last second." It's got tickers for eggs being produced, emails being sent, spam emails being received, recent news from The New York Times, CNN, newsvine, top Google searches of the day, and most importantly, seconds until doughnut day. That'd be a nice little screensaver - or something I'd have running 24/7 on a giant plasma.
Oh, and yes, that is my face in the middle.
Following up on my post last week about using Twitter to track eating and weight, some of you voiced some interest in creating your own Twitter bot. This post covers how you can do that.
The Gist of It
Creating my own Twitter bot was pretty straightforward (much more than I thought it'd be), mostly because Twitter provides an API and the resources to make it that way.
I wanted something really simple that I could play around with. I just wanted to be able to send a direct message to my Twitter bot, and from there, it would store my data. OK, so here are the basic steps I took:
- Create Twitter account for bot
- Turn on email notification for direct messages only
- Check email periodically for new direct messages
- Parse direct messages and store in database
The New York Times adds another item to the list of things to watch tonight as election results start to pour in. The Times invites readers to enter one word that describes their current state of mind and who they support. You are allowed to enter a word once per hour. The result is the above self-updating word cloud as new words from readers flow from left to right. You can filter among McCain supporters, Obama supporters, or everyone. McCain supporters appear to be worried, scared, and nervous while Obama supporters are excited, hopeful, and optimistic. Both sides are anxious. What's your current state of mind?
A quick reminder -- it's election day in the United States. If you're registered to vote, take the few minutes out of your day and go put in some ticks at your nearest voting place. If not for the presidency, at least go vote for your local candidates and propositions.