Mark Hansen for The Upshot describes the search for balance between individual privacy and an accurate 2020 Census count. It turns out to not be that difficult to reconstruct person-level data from publicly available aggregates:

On the face of it, finding a reconstruction that satisfies all of the constraints from all the tables the bureau produces seems impossible. But Mr. Abowd says the problem gets easier when you notice that these tables are full of zeros. Each zero indicates a combination of variables — values for one or more of block, sex, age, race and ethnicity — for which no one exists in the census. We might find, for example, that there is no one below voting age living on a particular block. We can then ignore any reconstructions that include people under 18 living there. This greatly reduces the set of viable reconstructions and makes the problem solvable with off-the-shelf software.

To combat this, the Census is looking into injecting more uncertainty into their published data. The challenge is figuring out how much uncertainty is too much and what level of privacy is enough.

A couple of famous directors were defending animated films as a medium rather than a genre of film meant for kids. I got to thinking about the parallels to visualization.

This interactive heatmap by Jonas Schöley shows mortality rates by age. Just use the dropdown menu to see the data for various countries. You can also compare male and female populations and countries.

As you might expect, you can see mortality rates decrease steadily, especially in the younger ages. Spikes or abrupt color changes might indicate war or disease. [via @maartenzam]

Multiple people can look at the same dataset and come out the other end with very different interpretations. [via @SteveStuWill]

Shannon Mattern for The Atlantic on how blockchain might be useful in mapping and as a replacement for GPS:

Crypto-cartographers hope to use it for spatial verification—confirming that things are where they say they are, when they claim to be there. How might this be useful? Well, you could know precisely when an Amazon delivery drone drops a package on your doorstep, at which point the charge would post to your account. No more unscrupulous delivery drivers, and no more contested charges for packages lost in transit. Or when opening a new bank account, you could virtually confirm your permanent address by physically being there during a particular verification period, rather than providing copies of your utility bills. You could also submit a photo of your flooded basement or smashed windshield to your insurance company, supplementing your claim with time- and location-verified documentation. Or, as you pass by your local family-owned coffee shop, the owner could “airdrop” some Bitcoin coupons to your phone, and you could stop by to cash in before the offer expires a half hour later.

Oh good.

RJ Andrews has a visualization design book coming out in January called Info We Trust. He hand-drew about 300 graphics for the book. One of the reasons:

I decided very early that Info We Trust would not use any existing images, mine or others. Found examples work fine for certain books. They are also convenient, the work is already done! But found images bring baggage too. You might choose an existing work to highlight one aspect of its design. But the reader will see other facets too.

I’m intrigued. On the upside, you get a consistent visual flow. A focused point of view. On the downside: the possibility of a view that is too focused. I have a feeling there will be more upside than downside. [Amazon pre-order]

Michael Correll on the use of “visualization literacy” in research:

If people (and, by some definitions, many or even most people) are chart illiterates, then we may feel tempted to write those groups off. We may prioritize the design of visualizations to help the creators of, say, machine learning models, from whom we can presume a sufficient level of visual and statistical literacy, rather than the populations who may be impacted by these models (sometimes unjustly). If what we mean by “visualization literacy” is narrow enough, or rare enough, then we’re already setting ourselves mental upper bounds for the number of people we’ll impact with our work.

This was an interesting perspective. I’m used to listening to or reading from people on the presentation side of visualization, in which case it’s your job to raise literacy. You should imagine what others are thinking and explain any points of possible confusion with annotation and intuitive visual encodings.

Don’t ever use “people won’t understand it” as a crutch.

Jonathan Schwabish gave his fourth-grade son’s class a lesson on data visualization. He wrote about his experience:

I’d love to see a way to make data visualization education a broader part of the curriculum, both on its own and linked with their math and other classes. Imagine adding different shapes to maps in their Social Studies classes to encode data or using waterfall charts in their math classes to visually demonstrate a simple mathematical equation or developing simple network diagrams in science class. The combination of the scientific approach to data visualization and the creativity it sparks could serve as a great way to help students learn.

Maybe I should introduce Schwabish’s Match It Game to the Yau household. My five-year-old has been asking why I keep “doing data.”

Throughout the month I collect new tools for data and visualization and additional resources on designing data graphics. Here’s the new stuff for November.

Brian Brettschneider made a joke map randomly designating the favorite pies of certain areas. While intended as a joke and a parody of past “favorite” maps, some people took it too seriously — like Senator Ted Cruz. Brettschneider describes the lessons he learned:

Maps hold a special standing among the public. We tend to place very high value in maps as holders of accurate information. If it’s in a map, it must be true, right? If I had tweeted a joke list of favorite pies by region, it would be very quickly ignored. Since it was in map form, it had an air of authenticity. This matters. As cartographers, we need to be cognizant of the power that maps have.

I had a similar experience years ago with Data Underload. These days I use the project as a catch-all for my own analyses, but I started it as a comic-ish chart series that I used to communicate anecdotes, opinions, and random stuff based on no actual data.

The form confused people at times, especially for those unfamiliar with the series, because it looked so much like charts based on real data. It was like reading a sarcastic comment online from someone you don’t know and trying to guess if it’s serious or not.

So now I keep the real-looking charts for real data unless it’s super obvious. Sometimes your choices on form can lead to unintended interpretations — like a joke taken as serious commentary.

The maps that we imagine as we think about locations around the world often don’t match up with reality. Betsy Mason for National Geographic explains the discrepancy. On the misalignment of Europe:

Europe is also often placed much farther south on mental maps than it really is, appearing directly across the Atlantic from the contiguous United States. But it actually lines up better with Canada: Paris is further north than Montreal, Barcelona is at a similar latitude as Chicago, and Venice lines up with Portland, Oregon.

…and the world as we knew it was never the same again.

Researchers at the University of Chicago’s Energy Policy Institute estimated the number of years lost and the number of people affected due to particulate matter in the air. They estimated per country. The Washington Post used a mosaic plot, aka a Marimekko chart, to show the differences.

The width of each column represents total population for a country. The sections in each columns represent the number of people who will lose a certain number of years. Color represents average years of life lost.

These charts are often a bit confusing at first glance, but the scrolling format used here provides some guidance.

In the spirit of the holidays, here are the tools I am most thankful for. Without them, work would be much more tedious and painful.

Michelle Chandra uses street data as a base for solvable mazes:

I draw each maze map by hand using the real street data of cities. In keeping with the fun nature of my art, I choose iconic city landmarks for the start and end of each maze – landmarks like the Golden Gate Bridge, Coney Island, or the Santa Monica Pier. All my maze maps are tested with friends and family to make sure they are, well, challenging to solve!

Grab a screen print here. Each one comes with an extra sheet to solve for yourself.

In news graphics, blue typically represents Democrat and red represents Republican. However, the definition isn’t so clear-cut by actual party usage. Chris Alcantara for The Washington Post broke it down in 900 campaign logos used during the recent midterms. Each strip represents a logo.

Kyle McDonald describes some of the history and current research on using algorithms to generate music. On how David Cope incorporated Markov chains to aid in his work:

In 1981 David Cope began working with algorithmic composition to solve his writers block. He combined Markov chains and other techniques (musical grammars and combinatorics) into a semi-automatic system he calls Experiments in Musical Intelligence, or Emmy. David cites Iannis Xenakis and Lejaren Hiller (Illiac Suite 1955, Experimental Music 1959) as early inspirations, and he describes Emmy in papers, patents, and even source code on GitHub. Emmy is most famous for learning from and imitating other composers.

I expected samples to sound robotic and unnatural, and some are, but some are quite pleasant to listen to.

When you go skiing or snowboarding, you get a map of the mountain that shows the terrain and where you can go. James Niehues is the man behind many of these hand-painted ski maps around the world, and he has a kickstarter to catalog his life’s work.

This is kind of amazing. I went skiing a lot as a kid, and I have distinct memories of these maps. I would stand at the top of the mountain, rip off one of my gloves with my teeth, and then pull out a folded map from a zipped pocket. I never knew they were by the same man, but in retrospect, it makes sense.

Newsy, Reveal and ProPublica look into rape cases in the U.S. and law enforcement’s use of exceptional clearance.

The designation allows police to clear cases when they have enough evidence to make an arrest and know who and where the suspect is, but can’t make an arrest for reasons outside their control. Experts say it’s supposed to be used sparingly.

Culled data from various police departments shows the designation is used more often that one would expect.