• Jobs Charted by State and Salary

    Posted to Data Underload  |  Tags: , ,

    Prominent industries in a state can say a lot about an area. Is there a lot of farming? Is there a big technology market? Couple the jobs with salary, and you also see where the money's at. You see a state's priorities.

    For example, look at California. You see an increased prominence of farmworkers and laborers, whereas the farming, fishing, and forestry sector is nearly nonexistent in many other parts of the country. I expected a lot more in the midwest states, but relative to the other occupations in those states, the farming sector doesn't seem that big from an employee perspective.

    For a drastic change, switch to Washington, D.C., where people who work in the legal and business sectors are much more common. I realize it's a comparison between a city and states, but whoa, that's a lot of lawyers packed in one place.

    Move the median salary up a bit, and you get a sense of overall salaries (and a correlating cost of living, kind of) as you check out different states.

    Anyway, it's an interesting first look at employment data from the Bureau of Labor Statistics. I'll have to poke more.

  • Data Cuisine uses food as the medium

    Posted to Data Art  |  Tags: ,

    Ditch the computer screen for your data. It's all about the food. Moritz Stefaner and prozessagenten, process by art and design ran a second round of the Data Cuisine workshop to explore how food can be used as a medium to communicate data. Naturally, you've got your basic visual cues, but when you introduce food, you open lots more possibilities.

    [W]e have all kinds of sculptural 3D possibilities. We can work with taste — from the basic tastes of sweet, sour, salty, bitter, umami to complex combinations or hotness. There is texture — immensely important in cooking! Then we have all the cultural connotations of ingredients and dishes (potatoes, caviar, …). We can work with cooking parameters (e.g. baking temperature or duration). Or the temperature of the dish itself, when served!

    The above shows piece of bread shows youth unemployment in Spain. See more data dishes here.

  • NSA programs with goofy names

    Posted to Infographics  |  Tags: ,

    Julia Angwin and Jeff Larson for ProPublica made a chart of NSA programs revealed in the past year. Programs were plotted subjectively from foreign to domestic surveillance on the horizontal axis and targeted to bulk surveillance on the vertical. So you get more controversial the further you move up towards the top right corner.

    Interesting stuff.

    The best part though is the goofy program names, as illustrated by Alberto Cairo. ParanoidSmurf and his siblings Nosey, Tracker, and Dreamy; EgotisticalGoat and EgotisticalGiraffe; WillowVixen. First off, who names these programs? And second, how do I get in on the naming action (without becoming creepy)?

  • Subpar Captain America

    Posted to Statistics  |  Tags: ,

    Animation Domination High-Def has a Captain America video of things that America is not so good at, relative to other countries. And they even cited their data source, the CIA World Factbook. How about that.

  • Bass Shapes visualizes sound in hand-drawn style

    Posted to Data Art  |  Tags:

    Media artist Nick Hardeman's audio visualization app Bass Shapes was rejected by the Mac App Store because "it's not useful." So Hardeman released the software as a free OS X download instead. It's a beauty.

    The app takes in sound input from your microphone or an external audio source through Soundflower (also free), and the visuals come to life. Watching Bass Shapes, you'd swear that you were seeing a custom, hand-drawn animation that served as some kind of old-school-ish intro to an animated film. But you'd be wrong.

    Download Bass Shapes and try it yourself.

  • Visualizing algorithms

    Posted to Visualization  |  Tags: ,

    Mike Bostock, who you might recognize from such things as Data-Driven Documents or the New York Times, writes on the value of visualizing algorithms for entertaining, teaching, learning, and debugging.

    Algorithms are a fascinating use case for visualization. To visualize an algorithm, we don’t merely fit data to a chart; there is no primary dataset. Instead there are logical rules that describe behavior. This may be why algorithm visualizations are so unusual, as designers experiment with novel forms to better communicate. This is reason enough to study them.

    But algorithms are also a reminder that visualization is more than a tool for finding patterns in data. Visualization leverages the human visual system to augment human intellect: we can use it to better understand these important abstract processes, and perhaps other things, too.

    At the very least, you'll have fun scrolling through the animated visuals that show how various algorithms work, but read the whole thing. It's good.

  • Super MIDI

    Posted to Visualization  |  Tags: ,

    Watch the notes rain down on those piano keys. Made with MIDItrail, a MIDI player with a 3-d visualization component. The program (and this video) has been around for a while, but it's new to me.

    Do people still make MIDIs? Before MP3s were a thing I downloaded and played more MIDI files than I care to admit.

  • Normal vs. paranormal distributions

    Posted to Miscellaneous  |  Tags:

    Paranormal distributionBy Matthew Freeman. This is important.

  • Real-time lightning map

    Posted to Mapping  |  Tags: ,

    Blitzortung is a community of volunteers who install inexpensive lightning sensors and transmit their data to a central server. In return, those who run the sensors have access to the network's data. The map that runs on the site shows the data in near real-time, providing a view of lightning strikes around the world. Pretty neat that this exists.

  • Detailed UK census data browser

    Posted to Mapping  |  Tags: ,

    DataShine Census provides a detailed view into United Kingdom 2011 census data. Population, housing, income, commute, and other variables are available.

    The DataShine mapping platform is an output from an ESRC Future Research Leaders Project entitled "Big Open Data: Mining and Synthesis". The overall project seeks promote and develop the use of large and open datasets amongst the social science community. A key part of this initiative is the visualisation of these data in new and informative ways to inspire new uses and generate insights. Phase one has been to create the mapping platform with data from the 2011 Census. The next phases will work on important issues such as representing the uncertainty inherent in many population datasets and also developing tools that will enable the synthesis of data across multiple sources.

    They're off to a good start.

  • Duck vs. rabbit plot

    Posted to Miscellaneous  |  Tags:

    Not sure where this is from, but feel that tingle in the back of your head? That's the feeling of your mind blowing up.

  • Mirador: A tool to help you find correlations in complex datasets

    Posted to Software  |  Tags: ,

    Mirador, a collaborative effort led by Andrés Colubri from Fathom Information Design, is a tool that helps you find correlative patterns in datasets with a lot of variables and observations. It's in the early stages of development, but is available to use and test on Windows and Mac. Colubri explains the process, from its early stages to its current iteration.

    Although fields like Machine Learning and Bayesian Statistics have grown enormously in the past decades and offer techniques that allows the computer to infer predictive models from data, these techniques require careful calibration and overall supervision from the expert users who run these learning and inference algorithms. A key consideration is what variables to include in the inference process, since too few variables might result in a highly-biased model, while too many of them would lead to overfitting and large variance on new data (what is called the bias-variance dilemma.)

    Leaving aside model building, an exploratory overview of the correlations in a dataset is also important in situations where one needs to quickly survey association patterns in order to understand ongoing processes, for example, the spread of an infectious disease or the relationship between individual behaviors and health indicators.

    Download Mirador to try it for yourself.

  • Burger Place Geography

    Posted to Data Underload  |  Tags: , ,

    After looking at pizza places, coffee, and grocery stores, I had to look at burger chains across the country. The data was just sitting there. (Thanks, AggData.)

    As before, the map above shows the nearest burger chain out of the selected seven. I chugged along every twenty miles, checked within a 10-mile radius, and then colored each dot accordingly.

    With pizza places you saw a lot of regionality despite the national coverage of Pizza Hut. You saw a lot of Domino's on the east, Little Caesar's in California, and Godfather's in the midwest. Similarly, with coffee, Dunkin' Donuts reigned in the east and Caribou is popular in the midwest.

    However, more than a handful of burger chains cover the country somewhat evenly, which gets you this map that resembles sprinkles on a cupcake, save a couple areas of interest. In the Oklahoma and Arkansas areas Sonic Drive-in dominates, and Jack in the Box established itself well in California. We saw a similar geographic pattern in Stephen Von Worley's burger map a few years ago.

    But still, McDonald's is sprinkled throughout, which shouldn't surprise since it has more than twice the locations than its nearest competitor Burger King. Keep in mind this includes all the Golden Arches in Wal-Marts, airports, and college food courts.

    Because of this expansive burger coverage by McDonald's and the other major chains, it's more useful to look at the locations separately, shown below. I also included all the other chains with at least a hundred locations.

    burgers

    As you expect, it looks like population density in the beginning. Chains are gonna open where the people are. Once you get past Wendy's though, you start to see region-specific chains.

    I'd say Dairy Queen is well-established nationally, but it's interesting to see a gap with Oklahoma, Arkansas, Mississippi, and Louisiana. Do the folks there not like Dairy Queen? Maybe Sonic has a stronghold on the states in an epic battle for burger supremacy. Or it's just a totally mundane reason like Dairy Queen started in Illinois, expanded east, and then saw growth opportunity in Texas.

    In any case, the separation is more obvious when you look at just Dairy Queen versus a competitor like Sonic, using the same distance formula as the first map.

    sonic-vs-dq

    The rest of the chains kind of have their regional pockets: Whataburger in Texas, Checkers in Florida, and of course, In-N-Out in California.

    Then there's all the local joints, which I didn't even touch on yet. I'll have to leave that for another day though. In case someone is interested, Yelp seems like a good place to start. I poked around the review data for a bit, and it was interesting that the local places almost always reigned review-wise, and profiles for chains basically serve as somewhere for people to complain.

  • Test your statistical wits about stuff in the world

    Posted to Statistics  |  Tags:

    Many of us aren't aware of how one country compares to others or public policy that has been around for decades. How Wrong You Are is a simple quiz game by Moiz Syed and Juliusz Gonera that tests such knowledge.

    How Wrong You Are is a collection of important questions that people are sometimes misinformed about. We poll you to measure how right — or how wrong — the public is about these important questions.

    Every week, we will add a new question. These are all questions that we hope you already know. But if you don't, don't worry! You learned something. Share your results, successful or not. Chances are, if you didn't know this question, other people might not, either.

    Play the game here. At the very least, you'll learn something new.

  • New York City taxi trips mapped

    Posted to Mapping  |  Tags: , ,

    While we're on the topic of NYC taxi data, Eric Fischer for Mapbox mapped all 187 million trips. Each observation contains the start and end location of a trip, so blue dots represent the former and orange represent the latter. My favorite bit is on the data collection artifacts, such as the map above.

    The patterns at JFK and LaGuardia airports show interesting artifacts of the data collection process. Almost all of the trips there must have really begun or ended right at the terminals, but many of them are attributed to the roads leading to and from the airports, where the last good GPS fix must have occurred.

    See also the New York Times animated map from several years ago that shows taxi activity during days of the week.

  • Drone crash database

    Based on data compiled from a combination of military records, Defense Department records, and drone manufacturers, Emily Chow, Alberto Cuadra and Craig Whitlock for the Washington Post provide a quick view into drone crashes.

    More than 400 large U.S. military drones crashed in major accidents worldwide between Sept. 11, 2001, and December 2013. By reviewing military investigative reports and other records, The Washington Post was able to identify 194 drone crashes that fell into the most severe category: Class A accidents that destroyed the aircraft or caused (under current standards) at least $2 million in damage.

    The top row represents where a drone crashed, the second row who owns it, and the third tells the type. Mouse over any of the tick marks, and you get details for the corresponding crash.

  • Lessons from improperly anonymized taxi logs

    Posted to Statistics  |  Tags: ,

    Through a Freedom of Information request Chris Whong received and eventually released NYC taxi logs starting in 2013 (about 173 million trips). Vijay Pandurangan looked at the data a little closer and deanonymized the logs to link hashed license numbers to the driver names. It didn't take much to do it. Pandurangan described the process and lessons organizations can learn when they release data.

    Someone on Reddit pointed out that one specific driver seemed to be doing an incredible amount of business. When faced with anomalous data like that, it's good practice to weed out data error before jumping to conclusions about cheating taxi drivers. Also, I couldn't shake the feeling that there was something about that encoded id number: "CFCD208495D565EF66E7DFF9F98764DA." After a little bit of poking around, I realised that that code is actually the MD5 hash of the character '0'. This proved my suspicion that this was actually a data collection error, but also made me immediately realise that the entire anonymization process was flawed and could easily be reversed.

    He also provided the code snippet he used to do it.

  • Clubs that connect World Cup national teams

    Gregor Aisch for the New York Times explored how the soccer clubs that play all year connect the national teams in this year's World Cup.

    The best national teams come together every four years, but the global tournament is mostly a remix of the professional leagues that are in season most of the time. Three out of every four World Cup players play in Europe, and the top clubs like Barcelona, Bayern Munich and Manchester United have players from one end of the globe to the other.

    My browser buckled a few times as I scrolled, but even without smooth transitions, it's an interesting dive into player connections.

  • Modern Love

    Posted to Miscellaneous  |  Tags: ,

    Based on a column by Tim McEown, the animated video Modern Love by Freddy Arenas elegantly illustrates a relationship.
     Continue Reading 

  • Mathematical cake slice

    Posted to Miscellaneous  |  Tags: ,

    Sir Francis Galton, creator of the concept of correlation and regression toward the mean, wrote a letter to the editor of Nature in 1906 on the best way to cut a circular cake. The result is moist cake with every slice, even if you eat it days later. Alex Bellos for Numberphile demonstrates in the video below.

    I don't get it. I typically just eat a full cake in one sitting with a really big fork.