• IBM data propaganda – babies and old guys with glasses

    March 27, 2010  |  Statistics

    IBM has been spreading the whole "smarter planet" spiel for a while now, but in the past few days, they've revealed the punchline. It's data. The key to a smarter planet is learning how to process and extract information from the 15 petabytes of data we generate per day.

    Surprising? No, not at all. Data's the hot thing right now, and that's where the money's at. Learn how to process all of it and you're gold.
    Continue Reading

  • Buy and sell data at Data Marketplace

    March 22, 2010  |  Data Sources

    Add another site to the list of places to find the data you need. Data Marketplace connects people who want data to people who can find, scrape, and cull data.

    Here's how it works. If you want data, you put in a request and optionally, a deadline and budget. A provider can then go find that data for you, maybe through scraping a difficult-to-parse website, and then post it online. You then have the option to purchase the tabular data.

    There are three big humps to get over though for Data Marketplace to work.
    Continue Reading

  • Tim Berners-Lee with an update on open data

    March 15, 2010  |  Data Sharing

    If people put data on the Web - government data, scientific data, community data - whatever it is, it will be used by other people to do wonderful things in ways they never could have imagined.

    — Tim Berners-Lee, TED, February 2010

    Tim Berners-Lee, credited with inventing the World Wide Web, comes back to TED a year after his call for open, structured data with a quick update. Spoiler alert: things are looking good - and they're only going to get a lot better. But you already knew that, right?
    Continue Reading

  • Is Jeff Bridges most likely to win best actor?

    March 7, 2010  |  Statistics

    oscar-time

    There's this article on CNN, from The Frisky, that has this little theory about who is most likely to win the Oscar for best actor:

    [T]he Oscar generally goes to the dude who has the most best actor and best supporting nominations under his belt already.

    That seemed like a curious statement. Didn't Forest Whitaker, Philip Seymour Hoffman, and Jaimie Foxx recently win on their first nominations for the coveted award? Okay, so Hoffman was actually up against a bunch of other newbies, but what about the rest?

    Only 10 out of the past 29 winners, or just over a third, had the most nominations their year. Take a look at the data since 1980. Is the theory valid? You decide.
    Continue Reading

  • Think like a statistician – without the math

    March 4, 2010  |  Design, Statistics

    Think like a Statistican

    I call myself a statistician, because, well, I'm a statistics graduate student. However, ask me specific questions about hypothesis tests or required sampling size, and my answer probably won't be very good.

    The other day I was trying to think of the last time I did an actual hypothesis test or formal analysis. I couldn't remember. I actually had to dig up old course listings to figure out when it was. It was four years ago during my first year of graduate school. I did well in those courses, and I'm confident I could do that stuff with a quick refresher, but it's a no go off the cuff. It's just not something I do regularly.

    Instead, the most important things I've learned are less formal, but have proven extremely useful when working/playing with data. Here they are in no particular order.
    Continue Reading

  • Spirit of graph and dance is alive

    February 24, 2010  |  Statistics

    A good portion of my time in high school was spent trying to get into college. The rest of the time I was trying to look cool while doing it. Now of course I know better and fully embrace the inner geek. I'll never know what life would've been like had I thrown caution to the wind back then, but I'm guessing it would've been something like this.
    Continue Reading

  • Get a Date With Your Online Profile Pic – Myths Debunked

    February 10, 2010  |  Statistics

    The online dating world can be a confusing place. How do you interact with others? Who should contact? What should you say about yourself? There are a lot of decisions to make, but it all starts with your profile picture when it comes to grabbing the attention of potential dates. Online dating site, OkCupid, analyzed over 7,000 profile pictures, debunking four myths:

    1. It's better to smile
    2. You shouldn’t take your picture with your phone or webcam
    3. Guys should keep their shirts on
    4. Make sure your face is showing

    Some of the results are pretty surprising. For example, men's photos were most effective when they weren't looking at the camera and not smiling:

    It was the opposite for women. A flirty face or smiling while looking at the camera showed most effective:

    Catch the full analysis here.

    [Thanks, Tom]

  • Data.gov.uk Homepage

    Data.gov.uk versus Data.gov – Which wins?

    Back in May last year, the US government launched Data.gov as a statement of transparency, and the Internet rejoiced. After the launch, excitement kind of…
  • Understanding risk – play it safe or eat a bacon sandwich?

    January 27, 2010  |  Statistics

    David Spiegelhalter is a Professor of the Public Understanding of Risk at Cambridge University. He studies the choices we make, and how those choices can have an effect later on.
    Continue Reading

  • Data.gov.uk Gearing Up For Launch, er, Does Launch

    January 20, 2010  |  Data Sources, Mapping

    Update: I had scheduled this post for next week, but apparently, Data.gov.uk launched today. The site isn't loading for me right now though. I guess they weren't prepared for traffic.

    Data.gov, a catalog of US data, launched last year. Now it's the UK's turn. Well, not yet. But soon. Data.gov.uk is still under lock and key, but it has granted access to some developers. Ito Labs, the group behind mapping a year of OpenStreetMap edits posted screenshots of their maps that show vehicle counts (above).

    Here are some comparison maps between 2001 and 2008, by vehicle type.
    Continue Reading

  • Virtual Slot Machine Teaches the Logic of Loss

    December 18, 2009  |  Infographics, Statistics

    This interactive by Las Vegas Sun describes how in the long run, you're going to lose every single penny when you throw your hard-earned money into a slot machine. In the short-term though, it is possible to win. It's all probability. It's also why statisticians don't gamble. Nobody plays a game that he's practically guaranteed to lose, unless you're a masochist - or you're Al Pacino in that one horrible sports gambling movie with Matthew McConaughey.

    One clarification on the snippet about payout percentage.

    Here's what the graphic reads:

    This is the ratio of money a player will get back to the amount of money he bets, which is programmed into the slot machine. If a machine has payout percentage of 90 percent, that means 90 percent of the money someone bets should statistically be won back. It means a player is not likely to lose 10 percent of the amount initially put into the machine, but rather 10 percent, on average, over time.

    The wording is kind of confusing. To be more clear - over time, on average, you'd lose 10% of the money you put in per bet. This is an important note, because it's how casinos make money. For example, when you play Blackjack perfectly (sans card-counting), you'll lose on average 2% (or something like that) per hand, so play long enough, and you're going to lose all your money.

    Imagine you have two buckets. One is filled with water. The other is empty. Transfer the water back and forth between the two buckets. Some of the water drips out during some of the transfers. Eventually, all the water is on the ground.

    Ah yes, intro probability is fun. Play the virtual slot machine and do some learning for yourself.

    [Thanks, Tyson]

  • Fox News Makes the Best Pie Chart. Ever.

    November 26, 2009  |  Mistaken Data, Ugly Charts

    Fox News pie chart

    What? I don't see anything wrong with it.
    Continue Reading

  • Choose Your Own Adventure – Watch the Stories Unfold (Updated)

    November 19, 2009  |  Infographics, Statistics

    Interaction designer Christian Swinehart takes a careful look at the popular Choose Your Own Adventure books from the 1980s. We saw something like this before, but Swinehart takes it a step further.
    Continue Reading

  • Class Size and SAT Scores By State

    November 10, 2009  |  Statistics

    Are there any differences in student performance between schools with small classes (as in students per teacher) and those with large classes?

    The natural response is yeah, of course, because if there are less students per teacher, each student gets more individual attention from the teacher. Then again, I went to pretty big elementary and high schools where some classes were in the high thirties. It didn't seem all that bad.
    Continue Reading

  • Unemployment, 2004 to Present – The Country is Bleeding

    November 4, 2009  |  Data Sources, Mapping

    The Bureau of Labor Statistics released the most recent unemployment numbers last week. Things aren't looking good for the unemployed, I'm afraid.

    I showed my younger sister the maps. Her response: "It looks like the country is bleeding."
    Continue Reading

  • Target Store Openings Since the First in 1962 – Data Now Available

    October 22, 2009  |  Data Sources

    FlowingData readers who have been around for a while will remember I made a map early this year that showed the growth of Target stores across America. It starts with the first one in 1962 and then goes from there. It was a follow-up to the Walmart map, which I shared the code and data for.
    Continue Reading

  • NYC BigApps Competition – $20k In Prize Money

    October 6, 2009  |  Data Sources

    It's exciting times for data heads. The launch of Data.gov back in May got things jump started; San Francisco recently announced DataSF; and now New York is getting in on the party with the announcement of their own Data Mine (live at 1pm EST today) and the NYC Big Apps competition.
    Continue Reading

  • 30 Resources to Find the Data You Need

    October 1, 2009  |  Data Sources

    Let's say you have this idea for a visualization or application, or you're just curious about some trend. But you have a problem. You can't find the data, and without the data, you can't even start. This is a guide and a list of sources for where you can find that data you're looking for. There's a lot out there.

    Universities

    Being a graduate student, I always look to the library for books and resources. Many libraries are amping up their technology and have some expansive data archives. Many statistics departments also tend to keep a list of data somewhere. Continue Reading

  • Share and Sell Data with Infochimps (100 Invites)

    September 25, 2009  |  Data Sources

    infochimpsThere's a lot of data on the Web, but it's all very scattered. At the same time, there's a lot of data sitting on people's hard drives that we don't have access to. There are various reasons why people don't share, but mainly, they just don't see the point.

    Infochimps tries to solve both of these problems with an open data marketplace.
    Continue Reading

  • Online Dating Service Analyzes Intro Messages – How to Get a Response

    September 24, 2009  |  Statistics

    netspeak-chart

    Online dating can be tricky. What do you say? How do you reply to people? What should you put in your profile? Should you use that profile picture from 15 years ago?

    Well, fret no more, because OkCupid, an online dating service, analyzed over 500,000 introduction messages and whether or not they got a response from the message receiver. For example, the above graphs shows reply rates for intro messages that used netspeak. Here's a tip: don't use it, probably because it makes you sound like an idiot or you take writing advice from the comments on YouTube.

    Other fine tips include: avoid compliments on physical appearance (because it's the inside that counts) and don't try to bring the conversation outside the service (because that's creepy).

    [via Waxy]

Copyright © 2007-2014 FlowingData. All rights reserved. Hosted by Linode.