Speaking of outliers, it’s not always obvious when and why a data point is an outlier. The Overview of Outliers package in R by Antony Unwin lets you compare methods.

Articles on outlier methods use a mixture of theory and practice. Theory is all very well, but outliers are outliers because they don’t follow theory. Practice involves testing methods on data, sometimes with data simulated based on theory, better with `real’ datasets. A method can be considered successful if it finds the outliers we all agree on, but do we all agree on which cases are outliers?

See also Unwin’s talk from 2017 for more about the thinking behind the package.

Neural networks can feel like a black box, because, well, for most people they are. Supply input and a computer spits out results. The trouble with not understanding what goes on under the hood is that it’s hard to improve on what we know. It’s also a problem when someone uses the tech for malicious purposes, as people are prone to do.

So, folks from Google Brain break down the structures of what makes these things work.

I always love a good lottery hacking story. Jason Fagone for The Huffington Post chronicles the winnings of Gerald and Marge Selbee, a retired couple from a small town in Michigan. It is a story of probabilities, expected values, and arduously buying a lot of tickets to maximize profits.

That’s when it hit him. Right there, in the numbers on the page, he noticed a flaw—a strange and surprising pattern, like the cereal-box code, written into the fundamental machinery of the game. A loophole that would eventually make Jerry and Marge millionaires, spark an investigation by a Boston Globe Spotlight reporter, unleash a statewide political scandal and expose more than a few hypocrisies at the heart of America’s favorite form of legalized gambling.

I think it’s every statistician’s fantasy to crack open a lottery’s flaw using the numbers. No? Just me? Okay, whatever.

The most interesting part though is that the loophole didn’t seem to be that obscure. Selbee just needed a bit of knowledge about big numbers, a pencil, and a napkin to crunch on. Are there more games out there like this? Do I need to start playing the lottery?

See also the statistician who cracked a scratch lottery code and the other statistician who won the lottery four times.

In a project he calls Sentence Space, Robin Sloan implemented a neural network so that you can enter two sentences and get a gradient of the sentences in between.

I’d never even bothered to imagine an interpolation between sentences before encountering the idea in a recent academic paper. But as soon as I did, I found it captivating, both for the thing itself—a sentence… gradient?—and for the larger artifact it suggested: a dense cloud of sentences, all related; a space you might navigate and explore.

The project is open source on GitHub if you want to have at it.

Speaking of surveillance cities, Ali Winston for The Verge reports on the relationship between Palantir and New Orleans Police Department. They used predictive policing, which is loaded with social and statistical considerations, under the guise of philanthropy. Palantir gained access to personal records:

In January 2013, New Orleans would also allow Palantir to use its law enforcement account for LexisNexis’ Accurint product, which is comprised of millions of searchable public records, court filings, licenses, addresses, phone numbers, and social media data. The firm also got free access to city criminal and non-criminal data in order to train its software for crime forecasting. Neither the residents of New Orleans nor key city council members whose job it is to oversee the use of municipal data were aware of Palantir’s access to reams of their data.

False positives. Over-policing. Bias from the source data driving the algorithms. This isn’t stuff you just mess around with.

Smart home. Smart city. They have a positive ring to it, as if the place or thing will know what we want right when we need it and adjust accordingly. It’s all very grand. That’s assuming the new technologies are all used for good things.

Geoff Manaugh for The Atlantic considers what might happen when the sensors and new data streams are used against individuals:

As the city becomes a forensic tool for recording its residents, an obvious question looms: How might people opt out of the smart city? What does privacy even mean, for example, when body temperature is now subject to capture at thermal screening stations, when whispered conversations can be isolated by audio algorithms, or even when the unique seismic imprint of a gait can reveal who has just entered a room? Does the modern city need a privacy bill of rights for shielding people, and their data, from ubiquitous capture?

Yes.

556 people have gone to space. In an article on their changed perspectives, Jason Treat for National Geographic shows when these select few went on their travels.

Birds migrate to areas more hospitable, but where do they go? It depends on the bird. It depends on the time of year. It depends on other various factors. Drawing from several data sources, National Geographic maps how birds migrate thousands of miles. View it on your desktop of maximum animated pleasure.

A car moving at 70 miles per hour has to stop suddenly. Another car going 100 miles per hour also has to stop suddenly. Your intuition might say that the former requires 30% less energy to stop, but the energy required is actually proportional to the square of the velocity. Ben Sparks for Numberphile explains:

[arve url=”https://www.youtube.com/watch?time_continue=364&v=i3D7XYQExt0″ /]

Okay. Now what are the energy gains and losses for the guy trying to speed by weaving in and out of slow traffic?

Mikhail Popov, a data scientist at the Wikimedia Foundation, led a workshop on visualization literacy recently. A short guide from that workshop is now freely available online.

Every year, we look at the medal counts of each country. Who’s winning? It depends on how much value you place on each medal. Do you only count the golds and disregard silver and bronze? Do you just treat all medals the same? Josh Katz for The Upshot lets you test all the possibilities with this interactive.

Apply different values to each medal type by mousing over the x-y coordinate plane and see how the country rankings shift.

Every time we book a flight, a Passenger Name Record is generated and saved by an outdated system, which links to private travel data. Paz Pena, Leil-Zahra Mortada and Rose Regina Lawrence for the Tactical Technology Collective outline that data and describe the consequences of the system failing to keep data private.

But although the PNR system was originally designed to facilitate the sharing of information rather than the protection of it, in the current digital environment and with the cyber-threats facing our data online, this system needs to be updated to keep up with the existing risks. PNRs are information-rich files are not only of interest for governments; they are also valuable to third parties – whether corporations or adversaries. Potential uses of the data could include anything from marketing research to hacks aimed at obtaining our personal information for financial scams or even doxxing or inflicting harm on activists.

Maybe be more careful next time you post your travel pictures online.

Mikaela Shiffrin won her first gold medal in PyeongChang with a fraction of a second lead. In events where athletes race side-by-side, it’s easier to see how close such a lead is. But with alpine skiing, it feels more like a race against a clock. So to capture some of the dramatics of the former, Derek Watkins and Denise Lu for The New York Times imagined the results had all skiers raced down at the same time.

It reminds of The Times’ coverage of Usain Bolt in the 2012 Summer Olympics.

This is fun. It’s a fantasy map generator with the following rules:

Project goal is a procedurally generated map for my Medieval Dynasty simulator. Map should be interactive, scalable, fast and plausible. There should be enought space to place at least 500 manors within 7 regions. The imagined area is about 200.000 km2.

Just click and there’s a new map generated on the fly.

Martin O’Leary’s generator is still my favorite, but I think there is plenty of room in the world for procedurally generated fantasy maps.

[arve url=”https://www.youtube.com/watch?v=spUNpyF58BY” /]

One of my least favorite electrical engineering courses in college was on signals and communications. I remember there being a lot of Fourier Transforms. I also remember falling asleep a lot, because it was a two-hour lecture with the lights turned off. Maybe if the demos were more visual like this, I would’ve stayed awake. (Probably not.)