• Texting data to save lives

    February 6, 2014  |  Data Sources

    Remember that TED talk from a couple of years ago on texting patterns to a crisis hotline? The TED talker Nancy Lublin proposed the analysis of these text messages to potentially help the individuals texting. Her group, the Crisis Text Line, plans to release anonymized aggregates in the coming months.

    Ms. Lublin said texts also provided real-time information that showed patterns for people in crisis.

    Crisis Text Line's data, she said, suggests that children with eating disorders seek help more often Sunday through Tuesday, that self-cutters do not wait until after school to hurt themselves, and that depression is reported three times as much in El Paso as in Chicago.

    This spring, Crisis Text Line intends to make the aggregate data available to the public. "My dream," Ms. Lublin said, "is that public health officials will use this data and tailor public policy solutions around it."

    Keeping an eye on this.

  • Map: US bus and Amtrak routes

    February 6, 2014  |  Mapping

    Bus routes

    In case you're wondering how to travel the country without a car (in a way other than running), this map from the American Intercity Bus Riders Association [pdf] shows you all the bus and Amtrak routes that span the United States. Keep in mind that these trains don't run 24/7, so plan accordingly.

  • Olympic events placed in New York for scale

    February 5, 2014  |  Infographics

    Bryant park ski jump

    The New York Times published a fun piece that places Winter Olympic events in the city. Events include the luge in Times Square, ski jump in Bryant Park, and speed skating down Broadway.

    The Winter Olympics sometimes gets flack for being the thing in between the more popular Summer Olympics, but I think it has a lot to do with scale and perception of the events. People know how fast they run, but don't always get how steep the mountains are. I used to go downhill skiing, and from a distance the hills didn't look especially daunting, but when I stood at the top of the black diamond, it looked pretty scary.

  • Where people run

    February 5, 2014  |  Data Underload

    Where people run

    There are many exercise apps that allow you to keep track of your running, riding, and other activities. Record speed, time, elevation, and location from your phone, and millions of people do this, me included. However, when we look at activity logs, whether they be our own, from our friends, or from a public timeline, the activities only appear individually.

    What about all together? Not only is it fun to see, but it can be useful to the data collectors to plan future workouts or even city planners who make sure citizens have proper bike lanes and running paths.
    Continue Reading

  • Quotes poster updates

    February 4, 2014  |  Announcements

    For those who ordered a famous quotes poster: I'll be updating the printing and shipping status on this page.

    I sent the poster to the printers on Friday, approved the digital proof yesterday, and the posters might be printing as I'm writing this. I still expect to start mailing to you in the middle of this month, assuming posters and (a lot of) shipping supplies are in my hands as scheduled. Thanks!

    For those who did not order a poster but still want one: It's not too late.

  • Places in the US with the most pleasant days per year

    February 4, 2014  |  Mapping

    Pleasant places to live

    We've seen plenty of maps the past few weeks that show how bad the weather is, in just about everywhere but California. Kelly Norton looked at it from the other direction and estimated how many pleasant days per year areas of the US get, based on historical NOAA data.

    I decided to take a stab at what constitutes a "pleasant" day and then aggregate NOAA data for the last 23 years to figure out the regions of the United States with the most (and least) pleasant days in a typical year. The results, I think, are not that surprising and pretty much affirm the answer given off the cuff by many of my west coast friends when asked about the best places, "Southern California?" For the areas with the least pleasant days, I admit I would have guessed North Dakota. However, it’s much of Montana that gets an average of a couple of weeks of pleasantness each year.

    Of course the map changes (mainly the geographic range) depending on the definition of a "pleasant" day. In this case it's defined as one where the mean temperature is between 55 and 75 degrees.

  • Digitally revamped atlas of historical geography, from 1932

    February 3, 2014  |  Mapping

    Snow cover

    In 1932, Charles O. Paullin and John K. Wright published Atlas of the Historical Geography of the United States, a reference of almost 700 maps about a varied set of topics, such as weather, travel, and population. The Digital Scholarship Lab at the University of Richmond brought the atlas to digital life.

    In this digital edition we've tried to bring—hopefully unobtrusively and respectfully—Paullin and Wright’s maps a bit closer to that ideal. First, with the exception of the historical maps from the cartography section and a handful of others (those that used polar projections, for example), we’ve georeferenced and georectified all of the maps from the atlas so that they can be overlaid consistently within a digital mapping environment. (Georeferencing is a process of linking points on a map to geographic coordinates, and georectification is a process of warping a map using those coordinates to properly align it within a particular projection, here web mercator.) High-quality scans of all of the maps as they appeared on the plates are available too.

    Not only are the maps overlaid on a slippy map, but the lab also added simple interactions with tool tips and animation so you can look more specifically at the data.

    I could spend all day (or several days) looking through this. [Thanks, Lee]

  • Amount of snow to cancel school

    January 31, 2014  |  Mapping

    Snow day

    Someone ended an email to me last week with "Stay warm." Not to sound like a jerk, but I happened to be answering email outside with my t-shirt on and sweater slung over the chair. I was also half-wondering whether I should change into shorts. Anyway, this map by Alexandr Trubetskoy, or reddit user atrubetskoy, might be of interest to many of you not in California. It shows an estimated amount of snow required to close school for the day, by county.
    Continue Reading

  • How R came to be

    January 30, 2014  |  Statistics

    Statistician John Chambers, the creator of S and a core member of R, talks about how R came to be in the short video below. Warning: Super nerdy waters ahead.

    I've heard this story before, but it was nice to hear it again, since it is about something I use almost every day. I would also like to hear about the invention of the toilet. [via Revolutions]

  • History through the president’s words

    January 30, 2014  |  Infographics

    History through the Presidents Words

    The Washington Post visualized the use of specific words throughout the years during State of the Union addresses.

    Since 1900, there have been 116 State of the Union addresses, given by 20 presidents, with some presidents giving two addresses a year. Studying their choice of words, over time, provides glimpses of change in American politics—"communism" fades, "terrorism" increases—and evidence that some things never change ("America" comes up steadily, of course. As does "I.").

    For some reason the interactive won't load for me now (It did yesterday.), but there's also a PDF version that you can download. Although the PDF only goes back to 1989 Bush, so try for the interactive version first. It was an interesting one. Update: Works again.

    Can you believe it? We made it through an entire SOTU without a single word cloud. Come to think of it, I can't even remember the last time I saw one. I almost feel cheated.

  • Last day to pre-order quotes poster

    January 29, 2014  |  Projects

    Famous movie quotesIt's been an interesting few days. I thought a few people would find the famous quotes graphic amusing, but I didn't expect so many to share my odd sense of humor. Thanks.

    If you haven't pre-ordered a poster yet, today's the last day to get it at a discounted price.

    Put your order in here.

    I'm going to proof the poster a few more times tonight and then send it to the printers. They should take about a week to get the finished posters to me. From there, I'll be (really) busy signing and rolling.

    I still expect mid-February shipments to you. International shipping takes a little longer of course, depending on where you are.

  • Learn R interactively with the swirl package

    January 29, 2014  |  Software

    R, the statistical computing language of choice and what I use the most, can seem odd to those new to the language or programming. And I think this what holds a lot of people back and what keeps people stuck in limited software. The swirl package for R helps beginners get over that first hurdle by teaching you within R itself.

    swirl is a software package for the R statistical programming language. Its purpose is to teach users statistics and R simultaneously and interactively. It attempts to do this in the most authentic learning environment possible by guiding users through interactive lessons directly within the R console.

    Assuming you installed R on your computer already, install the package (and the other packages it depends on), make a call to swirl(), and you get a guide through the basics.

  • What a computer sees while watching movies

    January 28, 2014  |  Data Art

    Benjamin Grosser visualized how computers "watch" movies through vision algorithms and artificial intelligence in Computers Watching Movies.

    Computers Watching Movies was computationally produced using software written by the artist. This software uses computer vision algorithms and artificial intelligence routines to give the system some degree of agency, allowing it to decide what it watches and what it does not. Six well-known clips from popular films are used in the work, enabling many viewers to draw upon their own visual memory of a scene when they watch it.

    Above is the bag scene from American Beauty. Contrast this with the more frantic Inception scene, and you get a good idea of how it works. See computer-watching scenes for several more movies here.

  • Members Only
    How to Map Paths in R

    How to Map Geographic Paths in R

    As people and things move through a place, it can be useful to see their connected paths instead of just individual points.
  • FlowingData Job Board, January 2014

    January 28, 2014  |  Job Board

    Looking for a job in data science, visualization, or statistics? There are openings on the board.

    Senior Game Analytics Specialist for Activision Publishing, Inc. in Santa Monica, CA

    Data Scientist for Thumbtack in San Francisco, CA

    Instructional Technologist for Quantitative Applications for Reed College in Portland, OR

  • Public transit times in major cities

    January 27, 2014  |  Mapping

    Public transit travel time

    Last year, WNYC made an interactive map that shows transit times in New York, based on where you clicked. Geography graduate student Andrew Hardin expanded on the idea for San Francisco, Seattle, Boulder, and Denver, with additional options and more granular simulations.
    Continue Reading

  • Facebook debunks Princeton study

    January 24, 2014  |  Statistics

    Princeton declining interest

    Researchers at Princeton released a study that said that Facebook was on the way out, based primarily on Google search data. Naturally, Facebook didn't appreciate it much and followed up with their own "study" that debunks the Princeton analysis, blasted with a healthy dose of sarcasm. They also showed that Princeton is on their way to zero-enrollment.

    This trend suggests that Princeton will have only half its current enrollment by 2018, and by 2021 it will have no students at all, agreeing with the previous graph of scholarly scholarliness. Based on our robust scientific analysis, future generations will only be able to imagine this now-rubble institution that once walked this earth.

    While we are concerned for Princeton University, we are even more concerned about the fate of the planet — Google Trends for "air" have also been declining steadily, and our projections show that by the year 2060 there will be no air left

    Crud. Dibs on the oxygen tanks.

  • Bird flight paths

    January 23, 2014  |  Data Art

    Dennis Hlynsky, an artist and a professor at the Rhode Island School of Design, recorded videos of flying birds and in post-processing shows previous flight positions for less than a second. The results are beautiful. It's like the video version of long-exposure photography.

    This is just one video in the series. Also see this, this, and this. [via Colossal]

  • Using data to find a girlfriend

    January 23, 2014  |  Statistics

    Remember when Amy Webb created a bunch of fake male profiles to scrape data from two dating sites and analyze it to find a husband? Mathematician Chris McKinlay took a similar route to find a girlfriend (and now fiancee). However, unlike Webb who used a relatively small sample, McKinlay scraped data for thousands of profiles in his area and analyzed the data more thoroughly, in search of the perfect mate.

    For McKinlay's plan to work, he’d have to find a pattern in the survey data—a way to roughly group the women according to their similarities. The breakthrough came when he coded up a modified Bell Labs algorithm called K-Modes. First used in 1998 to analyze diseased soybean crops, it takes categorical data and clumps it like the colored wax swimming in a Lava Lamp. With some fine-tuning he could adjust the viscosity of the results, thinning it into a slick or coagulating it into a single, solid glob.

    He played with the dial and found a natural resting point where the 20,000 women clumped into seven statistically distinct clusters based on their questions and answers. "I was ecstatic," he says. "That was the high point of June."

    He selected the two clusters most to his liking, looked at what interested the women, and then adjusted his profile accordingly. He didn't lie. He just emphasized the traits that he possessed and that women tended to like. Then he waited for women to notice him.

    It's kind of like he built a targeted advertising system for himself and then cast a really wide net. Even though McKinlay is engaged now, I still wonder if it actually worked or if something similar might have happened if he left it to chance. I like to believe in the latter. He did after all go on dates with 87 other people before finding a match.

  • A century of passenger air travel

    January 22, 2014  |  Mapping

    Aviation for 100 years

    Kiln and the Guardian explored the 100-year history of passenger air travel, and to kick off the interactive is an interactive map that uses live flight data from FlightStats. The map shows all current flights in the air right now. Nice.

    Be sure to click through all the tabs. They're worth the watch and listen, with a combination of narration, interactive charts, and old photos.

    And of course, if you like this, you'll also enjoy Aaron Koblin's classic Flight Patterns.

Unless otherwise noted, graphics and words by me are licensed under Creative Commons BY-NC. Contact original authors for everything else.