• Is Jeff Bridges most likely to win best actor?

    March 7, 2010  |  Statistics

    oscar-time

    There's this article on CNN, from The Frisky, that has this little theory about who is most likely to win the Oscar for best actor:

    [T]he Oscar generally goes to the dude who has the most best actor and best supporting nominations under his belt already.

    That seemed like a curious statement. Didn't Forest Whitaker, Philip Seymour Hoffman, and Jaimie Foxx recently win on their first nominations for the coveted award? Okay, so Hoffman was actually up against a bunch of other newbies, but what about the rest?

    Only 10 out of the past 29 winners, or just over a third, had the most nominations their year. Take a look at the data since 1980. Is the theory valid? You decide.
    Continue Reading

  • Think like a statistician – without the math

    March 4, 2010  |  Design, Statistics

    Think like a Statistican

    I call myself a statistician, because, well, I'm a statistics graduate student. However, ask me specific questions about hypothesis tests or required sampling size, and my answer probably won't be very good.

    The other day I was trying to think of the last time I did an actual hypothesis test or formal analysis. I couldn't remember. I actually had to dig up old course listings to figure out when it was. It was four years ago during my first year of graduate school. I did well in those courses, and I'm confident I could do that stuff with a quick refresher, but it's a no go off the cuff. It's just not something I do regularly.

    Instead, the most important things I've learned are less formal, but have proven extremely useful when working/playing with data. Here they are in no particular order.
    Continue Reading

  • Spirit of graph and dance is alive

    February 24, 2010  |  Statistics

    A good portion of my time in high school was spent trying to get into college. The rest of the time I was trying to look cool while doing it. Now of course I know better and fully embrace the inner geek. I'll never know what life would've been like had I thrown caution to the wind back then, but I'm guessing it would've been something like this.
    Continue Reading

  • Get a Date With Your Online Profile Pic – Myths Debunked

    February 10, 2010  |  Statistics

    The online dating world can be a confusing place. How do you interact with others? Who should contact? What should you say about yourself? There are a lot of decisions to make, but it all starts with your profile picture when it comes to grabbing the attention of potential dates. Online dating site, OkCupid, analyzed over 7,000 profile pictures, debunking four myths:

    1. It's better to smile
    2. You shouldn’t take your picture with your phone or webcam
    3. Guys should keep their shirts on
    4. Make sure your face is showing

    Some of the results are pretty surprising. For example, men's photos were most effective when they weren't looking at the camera and not smiling:

    It was the opposite for women. A flirty face or smiling while looking at the camera showed most effective:

    Catch the full analysis here.

    [Thanks, Tom]

  • Data.gov.uk Homepage

    Data.gov.uk versus Data.gov – Which wins?

    Back in May last year, the US government launched Data.gov as a statement of transparency, and the Internet rejoiced. After the launch, excitement kind of…
  • Understanding risk – play it safe or eat a bacon sandwich?

    January 27, 2010  |  Statistics

    David Spiegelhalter is a Professor of the Public Understanding of Risk at Cambridge University. He studies the choices we make, and how those choices can have an effect later on.
    Continue Reading

  • Data.gov.uk Gearing Up For Launch, er, Does Launch

    January 20, 2010  |  Data Sources, Mapping

    Update: I had scheduled this post for next week, but apparently, Data.gov.uk launched today. The site isn't loading for me right now though. I guess they weren't prepared for traffic.

    Data.gov, a catalog of US data, launched last year. Now it's the UK's turn. Well, not yet. But soon. Data.gov.uk is still under lock and key, but it has granted access to some developers. Ito Labs, the group behind mapping a year of OpenStreetMap edits posted screenshots of their maps that show vehicle counts (above).

    Here are some comparison maps between 2001 and 2008, by vehicle type.
    Continue Reading

  • Virtual Slot Machine Teaches the Logic of Loss

    December 18, 2009  |  Infographics, Statistics

    This interactive by Las Vegas Sun describes how in the long run, you're going to lose every single penny when you throw your hard-earned money into a slot machine. In the short-term though, it is possible to win. It's all probability. It's also why statisticians don't gamble. Nobody plays a game that he's practically guaranteed to lose, unless you're a masochist - or you're Al Pacino in that one horrible sports gambling movie with Matthew McConaughey.

    One clarification on the snippet about payout percentage.

    Here's what the graphic reads:

    This is the ratio of money a player will get back to the amount of money he bets, which is programmed into the slot machine. If a machine has payout percentage of 90 percent, that means 90 percent of the money someone bets should statistically be won back. It means a player is not likely to lose 10 percent of the amount initially put into the machine, but rather 10 percent, on average, over time.

    The wording is kind of confusing. To be more clear - over time, on average, you'd lose 10% of the money you put in per bet. This is an important note, because it's how casinos make money. For example, when you play Blackjack perfectly (sans card-counting), you'll lose on average 2% (or something like that) per hand, so play long enough, and you're going to lose all your money.

    Imagine you have two buckets. One is filled with water. The other is empty. Transfer the water back and forth between the two buckets. Some of the water drips out during some of the transfers. Eventually, all the water is on the ground.

    Ah yes, intro probability is fun. Play the virtual slot machine and do some learning for yourself.

    [Thanks, Tyson]

  • Fox News Makes the Best Pie Chart. Ever.

    November 26, 2009  |  Mistaken Data, Ugly Charts

    Fox News pie chart

    What? I don't see anything wrong with it.
    Continue Reading

  • Choose Your Own Adventure – Watch the Stories Unfold (Updated)

    November 19, 2009  |  Infographics, Statistics

    Interaction designer Christian Swinehart takes a careful look at the popular Choose Your Own Adventure books from the 1980s. We saw something like this before, but Swinehart takes it a step further.
    Continue Reading

  • Class Size and SAT Scores By State

    November 10, 2009  |  Statistics

    Are there any differences in student performance between schools with small classes (as in students per teacher) and those with large classes?

    The natural response is yeah, of course, because if there are less students per teacher, each student gets more individual attention from the teacher. Then again, I went to pretty big elementary and high schools where some classes were in the high thirties. It didn't seem all that bad.
    Continue Reading

  • Unemployment, 2004 to Present – The Country is Bleeding

    November 4, 2009  |  Data Sources, Mapping

    The Bureau of Labor Statistics released the most recent unemployment numbers last week. Things aren't looking good for the unemployed, I'm afraid.

    I showed my younger sister the maps. Her response: "It looks like the country is bleeding."
    Continue Reading

  • Target Store Openings Since the First in 1962 – Data Now Available

    October 22, 2009  |  Data Sources

    FlowingData readers who have been around for a while will remember I made a map early this year that showed the growth of Target stores across America. It starts with the first one in 1962 and then goes from there. It was a follow-up to the Walmart map, which I shared the code and data for.
    Continue Reading

  • NYC BigApps Competition – $20k In Prize Money

    October 6, 2009  |  Data Sources

    It's exciting times for data heads. The launch of Data.gov back in May got things jump started; San Francisco recently announced DataSF; and now New York is getting in on the party with the announcement of their own Data Mine (live at 1pm EST today) and the NYC Big Apps competition.
    Continue Reading

  • 30 Resources to Find the Data You Need

    October 1, 2009  |  Data Sources

    Let's say you have this idea for a visualization or application, or you're just curious about some trend. But you have a problem. You can't find the data, and without the data, you can't even start. This is a guide and a list of sources for where you can find that data you're looking for. There's a lot out there.

    Universities

    Being a graduate student, I always look to the library for books and resources. Many libraries are amping up their technology and have some expansive data archives. Many statistics departments also tend to keep a list of data somewhere. Continue Reading

  • Share and Sell Data with Infochimps (100 Invites)

    September 25, 2009  |  Data Sources

    infochimpsThere's a lot of data on the Web, but it's all very scattered. At the same time, there's a lot of data sitting on people's hard drives that we don't have access to. There are various reasons why people don't share, but mainly, they just don't see the point.

    Infochimps tries to solve both of these problems with an open data marketplace.
    Continue Reading

  • Online Dating Service Analyzes Intro Messages – How to Get a Response

    September 24, 2009  |  Statistics

    netspeak-chart

    Online dating can be tricky. What do you say? How do you reply to people? What should you put in your profile? Should you use that profile picture from 15 years ago?

    Well, fret no more, because OkCupid, an online dating service, analyzed over 500,000 introduction messages and whether or not they got a response from the message receiver. For example, the above graphs shows reply rates for intro messages that used netspeak. Here's a tip: don't use it, probably because it makes you sound like an idiot or you take writing advice from the comments on YouTube.

    Other fine tips include: avoid compliments on physical appearance (because it's the inside that counts) and don't try to bring the conversation outside the service (because that's creepy).

    [via Waxy]

  • What Cell Phone Provider is Best For You?

    September 15, 2009  |  Statistics

    Picking a cell phone plan is confusing, but it doesn't have to be.

    Providers purposely make it that way, so you don't see all that you're forking over per month until you're locked into a horrible 2-year plan. It doesn't have to be like this though. Let's look at the data to find what cell phone provider has the best price.
    Continue Reading

  • Low Income Hinders College Attendance, Even for Top Students

    September 1, 2009  |  Statistics

    snap20051012a

    What if you were a good student but knew you weren't going to be able to go to college?

    I was fortunate enough for most of my life to know that if I wanted to get a higher education, I would be able to. Thanks, Mom and Dad. It's hard for me to imagine working hard in middle school and high school if I didn't have that goal in mind, but that's the path that many grow up with.

    The above graph are the results of a study by the Department of Education started in 1988. It shows that low-income students are most likely not to complete college - despite doing well in 8th grade. It's a much different story for high-income students.

    The Department tracked student progress in 8th grade and through high school and college over the next 12 years. Only 3% of students, from low income families, with low 8th grade math performance, completed college. Compare that to students with the same math performance but from high income families. Thirty percent finished college. That's ten times more than the former.

    What's worse is that many low-income students who had high math performance still didn't complete college. The percentage of college completion for low-income, high math students was still lower than high-income, low math students.

    [via @golan]

  • Data is the New Hot, Drop-dead Gorgeous Field

    August 7, 2009  |  Statistics

    We all know this already, but it's nice to get some backing from The New York Times every now and then. In this NYT article, that I'm sure has spread to every statistician's email inbox by now, Steve Lohr describes the dead sexy that is statistics:

    The rising stature of statisticians, who can earn $125,000 at top companies in their first year after getting a doctorate, is a byproduct of the recent explosion of digital data. In field after field, computing and the Web are creating new realms of data to explore sensor signals, surveillance tapes, social network chatter, public records and more. And the digital data surge only promises to accelerate, rising fivefold by 2012, according to a projection by IDC, a research firm.

    I've got about one more year (hopefully) until I finish graduate school. Hmm, things are looking up, yeah? Of course, it's never been about the money. The profession of statistician didn't nearly seem so hot when I started school. The best news here is that us data folk are going to get paid for doing what we enjoy, and as time goes on there's only going to be more data to play with, and we're going to be in high demand:

    Yet data is merely the raw material of knowledge. "We're rapidly entering a world where everything can be monitored and measured," said Erik Brynjolfsson, an economist and director of the Massachusetts Institute of Technology's Center for Digital Business. "But the big problem is going to be the ability of humans to use, analyze and make sense of the data."

    Wait, but it's not just statisticians who can interpret data:

    Though at the fore, statisticians are only a small part of an army of experts using modern statistical techniques for data analysis. Computing and numerical skills, experts say, matter far more than degrees. So the new data sleuths come from backgrounds like economics, computer science and mathematics.

    Like a... data scientist? Excellent.

Unless otherwise noted, graphics and words by me are licensed under Creative Commons BY-NC. Contact original authors for everything else.