Alec Barrett for TWO-N describes the benefits and some of the intricacies of animated transitions in data visualization.

This visual essay is inspired by the question: What is happening conceptually between the start and end of a transition? I look at reasons for using animated transitions (besides “it looks cool”) and at the kinds of variables that can be transitioned. I conclude that we can think of animated transitions in two categories: those where the space between the start and end states consists of real/realistic data and grammatically valid states for that visualization, and those where it does not.

The essay by the way was published on Observable, a new-ish way to publish “interactive notebooks for data analysis, visualization, and exploration.” Worth a look if you’re into publishing and sharing code.

Maps typically show a view from straight above, which is good for navigation and to see regional patterns over large areas. However, missing out on the extra dimension of height can mean missing out on context. Robert Simmon for Planet Labs shows off some work in getting the less abstract perspective at a large scale.

Woo. Woo. Alex Hern reporting for The Guardian:

In at least one previous version of the Messenger app, Facebook only told users that the setting would enable them to “send and receive SMS in Messenger”, and presented the option to users without an obvious way to opt out: the prompt offered a big blue button reading “OK”, and a much smaller grey link to “settings”.

Nowhere in the opt-in dialogue was it made clear that text histories would be uploaded to Facebook’s servers and stored indefinitely.

This was only on Android devices, as iOS devices don’t provide developers such access.

Facebook and Google (among other companies) know a lot about you through the services you use. Dylan Curran for The Guardian provides a rundown:

This information has millions of nefarious uses. You say you’re not a terrorist. Then how come you were googling Isis? Work at Google and you’re suspicious of your wife? Perfect, just look up her location and search history for the last 10 years. Manage to gain access to someone’s Google account? Perfect, you have a chronological diary of everything that person has done for the last 10 years.

This is one of the craziest things about the modern age. We would never let the government or a corporation put cameras/microphones in our homes or location trackers on us. But we just went ahead and did it ourselves because – to hell with it! – I want to watch cute dog videos.

We knew this, right? But it’s weird that it took a government-related impetus to bring privacy concerns in social media back into the light. It feels different this time.

From 2010, Steve Jobs on data privacy:

[arve url=”https://www.youtube.com/watch?v=39iKLwlUqBo” /]

Building on their previous visualization work on black boys dropping income levels in adulthood, The Upshot adds the option to change demographic groups. See income mobility for different races, genders, and income starting points.

Lisa Charlotte Rost for Datawrapper provides guidance for designing choropleth maps that most fairly represent your data:

Maps are not objective, but a version of reality. When creating them, lots of choices are made: What to map, how to map and whether or not to use a map in the first place. Here we’ll try to find guidelines to all of these questions, for a specific subset of maps: Choropleth maps (the ones in which each region is filled with a color that represents a value).

Rost’s other guides (line charts, area charts, pie charts, and more) are also full of practical advice. Highly recommended.

Emily Baumgaertner reporting for The New York Times:

But critics of the change and experts in the Census Bureau itself have said that, amid a fiery immigration debate, the inclusion of a citizenship question could prompt immigrants who are in the country illegally not to respond. That would result in a severe undercount of the population — and, in turn, faulty data for government agencies and outside groups that rely on the census. The effects would also bleed into the redistricting of the House and state legislatures in the next decade.

Welp.

Justin Elliot reporting for ProPublica:

The full census, however, hasn’t included questions about citizenship since 1950. The Census Bureau has gathered such data in other surveys. The bureau switched the method of those surveys after the 2000 census. Today, it conducts the American Community Survey every year, which includes questions about citizenship, along with many other questions. The survey covers a sample of residents of the United States.

Experts said the Justice Department’s letter was misleading. And they questioned the Justice Department’s explanation in the letter, noting that the American Community Survey produces data on citizenship that has been used in Section 2 cases.

Welp.

WELP.

Vega-Lite is a grammar for interactive graphics primarily used for analysis. Altair is a visualization library in Python that is based on this grammar.

With Altair, you can spend more time understanding your data and its meaning. Altair’s API is simple, friendly and consistent and built on top of the powerful Vega-Lite visualization grammar. This elegant simplicity produces beautiful and effective visualizations with a minimal amount of code.

Jim Vallandingham just put up a useful introduction to the library if you’re looking to get your feet wet.

I do very little visualization-wise with Python since my current toolset typically covers my bases, but this has me curious.

The USA National Phenology Network uses a computer model to estimate heat build-up and the “first leaf” appearance across the country. Jeremy White for The New York Times animated it.

Issie Lapowsky for Wired:

The change that’s already come to Pennsylvania may not have been possible without the research Kennedy and three other expert witnesses brought to light. They took the stand with a range of analyses, some based in complex quantitative theory, others, like Kennedy’s, based in pure cartography. But they all reached the same conclusion: Pennsylvania’s map had been so aggressively gerrymandered for partisan purposes that it silenced the voices of Democratic voters in the state. Here’s how each came to that conclusion—and managed to convince the court.

This is a great story of visualization and data put to use for a greater good. The analyses solidify the points, and the charts drive them home.

As you can imagine, there was plenty of conversation between Earth and Apollo 11 en route to the moon. Nicholas Rougeux visualized the back and forth with an interactive timeline.

During the historic mission to the moon and back, some of the most famous words ever said were transmitted between the Apollo 11 spacecraft and Earth. Between those icons moments was a great deal of chatter—mostly about technical matters but also about how the Earth looked from space, bounding around on the moon, and even sandwiches.

Also available in print.

Research by Raj Chetty, Nathaniel Hendren, Maggie Jones, and Sonya Porter from the Equality of Opportunity Project suggests that black boys who grow up in rich families are still much more likely to fall into lower income levels than white boys who grow up in equally rich families. The shift from low income to higher levels also appears to be a greater challenge, which makes closing the gap that much harder.

Emily Badger, Claire Cain Miller, Adam Pearce, and Kevin Quealy for The Upshot cover the data extensively with a series of charts, driven visually with a particle-based Sankey diagram (above). The individual points provide a vehicle to show variation over aggregates.

Take your time with this one.

You can grab the aggregated data from the Equality of Opportunity Project. Here’s the research paper in full (pdf) if you’re interested. And if you’re interested in the Sankey implementation.

Army ants function without a leader and yet accomplish very organized-looking things, such as building bridges across gaps:

[arve url=”https://www.youtube.com/watch?v=sgDgYqEXN54″ /]

Researchers from the Swarm Lab believe they can break down the bridge-building process into a simple, two-rule system. Rule 1: If fellow ants are walking over you, stay put. Rule 2: If the number of ants walking over you isn’t higher than some rate, get moving again.

Full paper here (pdf).

Frustrated with vehicles blocking bus and bike lanes, Alex Bell applied some statistical methods to estimate the extent.

Sarah Maslin Nir for The New York Times:

Now Mr. Bell is trying another tack — the 30-year-old computer scientist who lives in Harlem has created a prototype of a machine-learning algorithm that studies footage from a traffic camera and tracks precisely how often bike lanes are obstructed by delivery trucks, parked cars and waiting cabs, among other scofflaws. It is a piece of data that transportation advocates said is missing in the largely anecdotal discussion of how well the city’s bus and bike lanes do or do not work.

Michael W. Kearney implemented a classifier for Twitter bots. It’s called botornot:

Uses machine learning to classify Twitter accounts as bots or not bots. The default model is 93.53% accurate when classifying bots and 95.32% accurate when classifying non-bots. The fast model is 91.78% accurate when classifying bots and 92.61% accurate when classifying non-bots.

Overall, the default model is correct 93.8% of the time.

Overall, the fast model is correct 91.9% of the time.

You can enter Twitter accounts to see what the model projects here. It’s barebones, and I’m not sure what the curve represents, but it’s fun to poke at.

The Upshot has used a needle to show shifts in their live election forecasts, because many readers don’t understand probability. Nate Cohn and Josh Katz:

This was evident before the result of the 2016 election, and as a result we tried something new: a jitter, where the needle quivered to reflect the uncertainty around the forecast. Although many readers disliked it, the jitter reflected an earnest attempt to give tangible meaning to abstract probabilities. Nonetheless, we turned the jitter off for all of our 2017 forecasts.

Tonight, readers will have the option to turn the jitter off. We expect that some readers will opt to do so, but remember this: Switching it off only hides the uncertainty — it doesn’t make it go away.

Read the whole thing for why the needle, what the needle means, and how The Upshot is using it.

As much as I hated what the needle showed me the first time I saw it, I’ve grown to appreciate the uncertainty it represents.

Kofi Annan for Nature on the importance of data in ending poverty and hunger:

Such fine-grained insight brings tremendous responsibility to act. It shows governments, international agencies and donors exactly where to direct resources and support. The Sustainable Development Goals — which UN member states endorsed when the Millennium Development Goals expired in 2015 — include the first targets for reducing stunting and wasting. The data indicate that no African country is currently on track to reach all the targets associated with ending hunger, achieving food security and improving nutrition.

This shows how crucial it is to invest in data. Data gaps undermine our ability to target resources, develop policies and track accountability. Without good data, we’re flying blind. If you can’t see it, you can’t solve it.