While we’re on the topic of Statistics books for the general public, David Spieglhalter’s The Art of Statistics: How to Learn from Data is also on my reading list.

In The Art of Statistics, world-renowned statistician David Spiegelhalter shows readers how to derive knowledge from raw data by focusing on the concepts and connections behind the math. Drawing on real world examples to introduce complex issues, he shows us how statistics can help us determine the luckiest passenger on the Titanic, whether a notorious serial killer could have been caught earlier, and if screening for ovarian cancer is beneficial. The Art of Statistics not only shows us how mathematicians have used statistical science to solve these problems — it teaches us how we too can think like statisticians. We learn how to clarify our questions, assumptions, and expectations when approaching a problem, and — perhaps even more importantly — we learn how to responsibly interpret the answers we receive.

I was waiting for the book to come to North America, and apparently it did in 2019. I’m so behind in my reading, but I declare 2021 as the year I take my attention span back.

Tim Harford has a new book coming out tomorrow called The Data Detective: Ten Easy Rules to Make Sense of Statistics.

Today we think statistics are the enemy, numbers used to mislead and confuse us. That’s a mistake, Tim Harford says in The Data Detective. We shouldn’t be suspicious of statistics—we need to understand what they mean and how they can improve our lives: they are, at heart, human behavior seen through the prism of numbers and are often “the only way of grasping much of what is going on around us.” If we can toss aside our fears and learn to approach them clearly—understanding how our own preconceptions lead us astray—statistics can point to ways we can live better and work smarter.

Added to the list.

If you’re outside North America, look for How To Make The World Add Up. They’re the same book.

For Reuters, Feilding Cage, Chris Canipe and Prasanta Dutta made an interactive that lets you adjust dose rate and state in a simulation to get an estimate for when we might reach herd immunity.

As with any simulation, there are assumptions and simplifications. In this case, the dose rate stays uniform and total population is used, even though there are no vaccines available to children yet. But it’s something.

My main takeaway is that we’re gonna have to be patient (still).

Just speaking to the chart, I like the sketch-ish dashed lines and gradient to show herd immunity ranges. They communicate that things are still uncertain.

Vivian Wu made a snowflake generator. Adjust parameters such as growth, kaleidoscoping, and density, and you dear friend, can make yourself a unique snowflake of your very own.

I think I’ll just zone out and let the animation play out for a few minutes every day. Breathe in. Breathe out.

Here’s the good stuff for January.

The United States passed 425,000 coronavirus deaths this week. For The New York Times, Lazaro Gamio and Lauren Leatherby used dot density over time to show how we got to this point.

Each dark pixel represents a death, and each tick mark represents a day. So the strip starts light with sparsely placed dots, and then it gets darker and darker. Get to present day, and there’s hardly any white space.

Twitter released a small JavaScript library to make density plots — for when you have a lot of overlapping points and could use some granular binning. Feed a method an array of thousands of x-y coordinates, and the library takes care of the rest.

With vaccines, we might be tempted to jump back into “normal” life before it’s really safe. The New York Times reports on why waiting until March instead of February might be the way to. This is based on estimates from Columbia University researchers, and you can read the preprint here (pdf) by Jeffrey Shaman et al.

We’ve come this far already…

Maybe you remember the SimCity-like views through satellite imagery from a few of years ago. Robert Simmon from Planet Labs returns to the topic discussing practical use cases and advantages over a top-down view:

Satellite imagery surrounds us — from Google Maps and daily weather forecasts to the graphics illustrating news stories — but almost all of it is from a map-like, top-down perspective. This view allows satellite data to be analyzed over time and compared with other sources of data. Unfortunately, it’s also a distorted perspective. Lacking many of the cues we use to interpret the world around us, top-down satellite imagery (often called nadir imagery in remote sensing jargon) appears unnaturally flat. It’s a view that is disconnected from our everyday experience.

Thomas Mock explains how to extract and parse data tables in image files via ImageMagick and R:

There are many times where someone shares data as an image, whether intentionally due to software constraints (ie Twitter) or as a result of not understanding the implications (image inside a PDF or in a Word Doc). xkcd.com jokingly refers to this as .norm or as the Normal File Format. While it’s far from ideal or a real file format, it’s all too common to see data as images in the “wild”. I’ll be using some examples from Twitter images and extracting the raw data from these. There are multiple levels of difficulty, namely that screenshots on Twitter are not uniform, often of relatively low quality (ie DPI), and contain additional “decoration” like colors or grid-lines. We’ll do our best to make it work!

You can never have too many tools to grab data from various, inconvenient file formats.

[arve url=”https://www.youtube.com/watch?v=gFFj22kjlZk&feature=emb_title” loop=”no” muted=”no” /]

Jon Schwabish has a new book coming out: Better Data Visualizations. To kick things off, he’s running a video series on the many different chart types. There will be 50 videos released daily, each with an invited practitioner to briefly talk about what the chart is and how it’s used. They’re already 10 videos into it.

Should be informative.

I make most of my static charts in R, but I always bring them into Illustrator for polish, layout, and readability. Here’s my process.

The New York Times labeled all of the people sitting behind Joe Biden during the inauguration. It’s a straightforward but slick interactive that lets you pan and zoom the photograph. Click on a name for more details or use the list of names in a sidebar.

Based on estimates from the MIT Trancik Lab, The New York Times plotted average carbon dioxide emissions against average cost per month for electric, hybrid, and gas vehicles. Each dot represents a vehicle type. While electric vehicles cost more upfront, the lower maintenance and electric costs make up the difference in the long run.

The chart above only shows vehicles that retail for $55,000 or less, but you can see more vehicles in the original version.

For Bloomberg, Jeremy C.F. Lin and Rachael Dottle show what Joe Biden’s inauguration will look like, given all of the recent events and 2020. No public access and 25,000 National Guard personnel.

For NYT’s The Upshot, Kevin Quealy has been cataloging all of the insults Trump tweeted over the past five years. The project is complete:

As a political figure, Donald J. Trump used Twitter to praise, to cajole, to entertain, to lobby, to establish his version of events — and, perhaps most notably, to amplify his scorn. This list documents the verbal attacks Mr. Trump posted on Twitter, from when he declared his candidacy in June 2015 to Jan. 8, when Twitter permanently barred him.

48,000 words.

As you probably know, there was a big Parler data scrape before the app and site went down. ProPublica spliced Parler video posts, sorting them by time and location. The result is basically a TikTok-style video feed of what happened.

[arve url=”https://www.youtube.com/watch?v=1cUUfMeOijg” loop=”no” muted=”no” /]

Tom Scott explains how Cloudflare uses a wall of lava lamps to generate random numbers. A video camera is pointed at the wall, and the movement in the lamps plus noise from the video provides randomness, which is used to secure websites.

Even though computers can do many things on their own, they still need help from the physical world for true unpredictability. The robot overlords aren’t here yet. [via kottke]

The best way to make charts more interesting is to provide lots of details, which requires more work on your end but pays off in the end result.