• Feltron Annual Report 2013

    Posted to Self-surveillance  |  Tags: ,

    In his ninth edition of the personal report, Nicholas Felton looks at communication through his phone, email, Facebook, and physical mail.

    Also, don't miss the short video from the New York Times. Felton is half-jokingly asked if he's obsessive compulsive which always amuses me.

    It reminds me of when I asked someone about her pedometer, and she gladly talked about how she logged her steps every day for nearly a decade. Days with a lot of steps reminded her of trips or long walks. So naturally, I brought up my dissertation work on personal data collection. I thought she would be totally into it, but she was skeptical. She wondered why anyone would want to collect data on their location, computer usage, or sleep habits. And again, this was right after she told me about her decade of step logs.

    There's a disconnect.

    Actively looking at your data seems to cross you over to the obsessive side. I haven't quite figured it out yet, but the separation between the active and passive seems to be getting fuzzier. Maybe one day there'll be a guy in an interview who doesn't collect data about himself, and everyone is curious why.

  • When the world sleeps

    Posted to Statistical Visualization  |  Tags: ,

    An additional hour of sleep can make a huge difference in how you feel the next day (especially when you have kids). It's the ability to concentrate for long periods of time versus the ability to stare at a clock until your next break. I got the Jawbone UP24 band to try to improve on that, and I still wear it every night to better understand my sleep habits.

    So, it only seems natural for Jawbone to look closer at how people sleep as a whole in a couple of interactive graphics. Select your city to see how people sleep in your neck of the woods.

    Every now and then we see a set of graphics that shows America's sleep habits, based on data from the American Time Use Survey. The Jawbone data is likely more accurate though, which makes it more interesting. The former depends on survey participants' memories and doesn't factor out things like reading in bed. The latter is actual sleep.

  • Everywhere Jonny Cash went, man

    Posted to Mapping  |  Tags: ,

    Johnny Cash says he went to a lot of places in his song, "I've Been Everywhere." Iain Mullan had some fun with the location list for Music Hack Day London and mapped each place as the song plays.

    Also related to songs and location: where Ludacris claimed to have hoes.

  • Unintentional Venn diagram suggests opposite meaning

    Posted to Mistaken Data  |  Tags: ,

    Most people probably wouldn't think much about this poster that shows the values of Thomson Reuters. But when you think of the graphic as a Venn diagram, it's hard to see much else.

  • Not automatic

    It's an absolute myth that you can send an algorithm over raw data and have insights pop up.

    — Jeffrey Heer in For Big-Data Scientists, 'Janitor Work' Is Key Hurdle to Insights by Steve Lohr

  • Crisis Text Line releases trends and data

    Posted to Data Sources  |  Tags:

    Crisis Text Line is a service that troubled teens can use to find help with suicidal thoughts, depression, anxiety, and other issues via text messaging. The long-term hope was to anonymize and encode these text messages so that researchers and policy-makers could better understand something typically kept private to the individuals.

    Following through, the organization recently released a look into their data and a sample of encoded messages. (There's a link to download the data at the bottom of the page.)

    The visual part of the release shows when text messages typically come in, and you can subset by issue, state, and days. It could use some work, but it's a good start. Hopefully they keep working on it and release more data as the set grows. It could potentially do a lot of good.

  • Talking Ferguson on Twitter and localness

    Posted to Mapping  |  Tags: ,

    For trending topics, Twitter likes to show an animated map of how a lot of people talked about something at once. They pushed one out for Ferguson tweets. Naturally, the map looks a lot like population density. So instead, Eric Huntley aggregated and normalized for a more useful view.

    Ultimately, despite the centrality of social media to the protests and our ability to come together and reflect on the social problems at the root of Michael Brown's shooting, these maps, and the kind of data used to create them, can't tell us much about the deep-seated issues that have led to the killing of yet another unarmed young black man in our country [5]. And they almost certainly won't change anyone's mind about racism in America. They can, instead, help us to better understand how these events have been reflected on social media, and how even purportedly global news stories are always connected to particular places in specific ways.

    You won't find answers to the more important questions on Twitter.

  • Map of military surplus distribution

    Posted to Mapping  |  Tags: , ,

    With the situation in Ferguson, the New York Times mapped the distribution of military surplus through Defense Department program. Equipment, especially assault rifles, have gone to most parts of the United States.

  • FuelBand Fibers visualizes daily activities beautifully

    Posted to Data Art  |  Tags: , ,

    Leading up to a Nike women's 10k run, design studio Variable made FuelBand Fibers, an artistic interpretation of a week of activity from seven individuals.

    To celebrate effort of preparing for the run Nike has chosen 7 influential runners equipped with FuelBands. Up to the minute Nike Fuel data was then collected 24/7 and delivered to Variable to tranform into beautiful artworks. This is how Fibers were born. 7 digital fibers growing when the person is working out, one for each day of training, stylized uniquely for the given runner.

    So each fiber represents a day, each essentially a timeline from bottom to top. Thickness represents activity, and colors represent times when a person led in the FuelBand community. The results: organic.

    See more details on what the visuals show and how it was made on the project page.

    Reminds me of Universal Everything, which also focused on individuals' movements. [via Creative Applications]

  • How charity: water uses data to do more good

    Posted to Statistics  |  Tags:

    Scott Harrison, the founder and CEO of charity: water, describes how the organization uses data to improve what they do, both on the ground and internally.

    The first is program data, which is the information we collect on the water programs and projects we fund in 22 countries around the world. The second is data on our donors and supporters — for example, how much time or money they've given, which projects or aspects of our work they are most interested in and how they interact with our website. And finally, we collect internal data on our work in order to increase our effectiveness and efficiency as an organization.

  • Variance be damned

    Posted to Statistics  |  Tags: ,

    Daniel Colman won $15.3 million in the The Big One for One Drop poker tournament, but he seems annoyed about it. What he had to say after the win:

    First off, I don't owe poker a single thing. I've been fortunate enough to benefit financially from this game, but I have played it long enough to see the ugly side of this world. It is not a game where the pros are always happy and living a fulfilling life. To have a job where you are at the mercy of variance can be insanely stressful and can lead to a lot of unhealthy habits.

    Clearly we're missing some details here — stuff that would make a person disgusted with winning — but how about that response. Stability and normality for the win. Or, well, in this case, variance for the win. Nevermind, forget it. [via Deadspin]

  • CSV Fingerprint: Spot errors in your data at a glance

    You get your CSV file, snuggle under your blanket with a glass of fine wine, all ready for the perfect Saturday night. Then — what the heck — there's a bunch of missing data and poorly formatted entries. Don't let this happen to you. CSV Fingerprint by Victor Powell provides a simple, wideout view of your CSV file, color-coded for quick quality control.

    To make it easier to spot mistakes, I've made a "CSV Fingerprint" viewer (named after the "Fashion Fingerprints" from The New York Times's "Front Row to Fashion Week" interactive ). The idea is to provide a birdseye view of the file without too much distracting detail. The idea is similar to Tufte's Image Quilts...a qualitative view, as opposed to a rendering of the data in the file themselves. In this sense, the CSV Fingerprint is a sort of meta visualization.

    Try it with your own CSV data. Never let a subpar CSV file ruin your Saturday night again.

  • State of birth, by state and over time

    We've seen migration within the United States before, but Gregor Aisch, Robert Gebeloff, and Kevin Quealy for the Upshot take a more time-centric look at how people moved state to state, over the past century.

    The following charts document domestic migration since the turn of the last century, based on census data. For every state, we've broken down the population by resident's state of birth. The ribbons are color-coded by region, and foreign-born residents are included at the bottom, in gray, to complete the picture for each state.

    The good thing about the ribbon approach, other than the flow-like aesthetic that lends well to the topic, is that you can see the change in order through the years. Unlike a stacked area chart, each layer isn't restricted to an original ranking, so you can for instance, see that a lot of people born in California moved to Idaho starting in the 1960s.

    On top of that, there are lots of nice details like ribbons move to the top when you mouse over, labels that follow the time series pattern, and a thicker highlight bar at each point in time. All of these make the data easier to read.

    Check out the details of your state and others.

  • Interactive documentary takes you through space and orbits

    Posted to Mapping  |  Tags: , ,

    Impressive work in A Spacecraft for All:

    This Chrome Experiment follows the unlikely odyssey of the ISEE-3, a spacecraft launched in 1978 to study the Sun, but better known for its amazing accomplishments beyond that original mission. "A Spacecraft for All" is an interactive documentary combining film and 3D graphics, allowing you to follow the spacecraft's story as you trace it along its entire 36 year journey.

    The combination of video and interactive sometimes feels gimmicky, but this feels like they belong together. The interactive portion lets you casually interact in space and look at orbit paths, and the video portion explains what you're looking at. Guidance comes when necessary.

  • Where the poor live, a decade comparison

    Posted to Mapping  |  Tags: ,

    To better understand race and poverty, MetroTrends maps where people live whose income is below the poverty line.

    The history, geography, and politics of individual metro regions all matter profoundly, and any serious policy strategy must be tailored to local realities.

    To help take the policy conversation from the general to the specific, we offer a new mapping tool. It lets you explore changes from 1980 to 2010 in where poor people of different races and ethnicities lived, for every metropolitan region nationwide.

    Each dot, color-coded by race, represents 20 people. So when you slide between views for 1980 and 2010, you see how areas have grown more or less diverse, increased or decreased in covered areas, and perhaps areas in need of more attention.

  • Looking For the Closest Casino

    Posted to Data Underload  |  Tags: ,

    The New York Times covered casinos in the Northeast and noted that more than half of the population is within 25 miles of a casino. Naturally, I wondered what it is like in other parts of the country, so I sampled uniformly across the United States looking for the nearest casino. The results are shown in the map above.

    Using the Google Places API, I took about 7,500 samples in twenty-mile increments, east to west and north to south. Of those samples, 36 percent of them are 25 miles or less from a casino.

    This is significantly lower than the Northeast's 50 percent mark, but keep in mind that the sample also includes uninhabited areas. Lots of deserts and mountains. So the national percentage for population is likely higher. After all, some 80 percent of the country's population lives in urban areas.

    However, it's tough to say what the actual percentage is based on this data. Google classifies places as casinos based on their own criteria, and not everything is an actual casino by a legal definition. For example, there's one Utah casino location in my sample, but gambling is illegal in the state, and there are no Indian reservation casinos. This particular place is a poker party business.

    So think estimate rather than concrete count.

    But, speaking of reservations, also of note: how this map coincides with Indian reservations, especially in the west.

    Indian reservations

    In any case, if you're in the mood to lose money, alongside a three-dollar prime rib dinner, it's likely you're not too far off.

  • Vector maps on the web with Mapbox GL

    Posted to Software  |  Tags: ,

    Online mapping just got an upgrade:

    Announcing Mapbox GL JS — a fast and powerful new system for web maps. Mapbox GL JS is a client-side renderer, so it uses JavaScript and WebGL to dynamically draw data with the speed and smoothness of a video game. Instead of fixing styles and zoom levels at the server level, Mapbox GL puts power in JavaScript, allowing for dynamic styling and freeform interactivity.

    For the non-developers: Online maps are typically stored pre-made on a server, in the form of a bunch of image files that are stitched together when you zoom in and out of a map. So developers have to periodically update the image files if they want their base maps to change. It's a hassle, which is why base maps often look similar. With Mapbox GL, making changes is easier because the development pipeline is shorter.

    More details on the JavaScript library here.

  • Mapping the spread of drought, nationally

    Posted to Mapping  |  Tags: ,

    Although California has perhaps had it the worst, drought also affects other states, mainly the southwestern ones. Mike Bostock and Kevin Quealy for the New York Times have been updating an animated map weekly. It shows the spread of drought severity, across the United States. But, be sure to scroll down to also see drought levels over time, shown as stacked area chart.

    See also: NPR drought tracking from a couple of years ago.

  • California drought in small multiples

    Posted to Visualization  |  Tags: , ,

    To show the increased levels of drought throughout the state of California, Kyle Kim and Thomas Suh Lauder for the LA Times showed weekly change in drought levels with 188 color-coded California maps. There's also an animated version, but why do that when you can scrollllll?

    A couple of years ago, the New York Times did something similar, but with a two-category color scale and on a national scale.

  • Geography.

    Posted to Mistaken Data  |  Tags: ,

    By way of David Kennerr, something in this CNN frame seems off.