[T]he Oscar generally goes to the dude who has the most best actor and best supporting nominations under his belt already.

That seemed like a curious statement. Didn't Forest Whitaker, Philip Seymour Hoffman, and Jaimie Foxx recently win on their first nominations for the coveted award? Okay, so Hoffman was actually up against a bunch of other newbies, but what about the rest?

Only 10 out of the past 29 winners, or just over a third, had the most nominations their year. Take a look at the data since 1980. Is the theory valid? You decide.

Of course when Jeff Bridges wins tonight, the theory authors will declare victory, but oh well.

Just for fun let's take a poll:

World Progress Report available - 6 days left

I call myself a statistician, because, well, I'm a statistics graduate student. However, ask me specific questions about hypothesis tests or required sampling size, and my answer probably won't be very good.

The other day I was trying to think of the last time I did an actual hypothesis test or formal analysis. I couldn't remember. I actually had to dig up old course listings to figure out when it was. It was four years ago during my first year of graduate school. I did well in those courses, and I'm confident I could do that stuff with a quick refresher, but it's a no go off the cuff. It's just not something I do regularly.

Instead, the most important things I've learned are less formal, but have proven extremely useful when working/playing with data. Here they are in no particular order.

Attention to Detail

Oftentimes it's the little things that end up being the most important. There was this one time in class when my professor put up a graph on the projector. It was a bunch of data points with a smooth fitted line. He asked what we saw. Well, there was an increase in the beginning, a leveling off in the middle, and then another increase. However, what I missed was the little blip in the curve in the first increase. That was what we were after.

The point is that trends and patterns are important, but so are outliers, missing data points, and inconsistencies.

See the Big Picture

With that said, it's important not to get too caught up with individual data points or a tiny section in a really big dataset. We saw this in the recent recovery graph. Like some pointed out, if we took a step back and looked at a larger time frame, the Obama/Bush contrast doesn't look so shocking.

No Agendas

This should go without saying, but approach data as objectively as possible. I'm not saying you shouldn't have a hunch about what you're looking for, but don't let your preconceived ideas influence the results. Because if you go to length looking for some specific pattern, you're probably going to find it. It'll just be at the sacrifice of accurate results.

Look Outside the Data

Context, context, context. Sometimes this will come in the form of metadata. Other times it'll come from more data.

The more you know about how the data was collected, where it came from, when it happened, and what was going on at the time, the more informative your results and the more confident you can be about your findings.

Ask Why

Finally, and this is the most important thing I've learned, always ask why. When you see a blip in a graph, you should wonder why it's there. If you find some correlation, you should think about whether or not it makes any sense. If it does make sense, then cool, but if not, dig deeper. Numbers are great, but you have to remember that when humans are involved, errors are always a possibility.

*Photo by misterbisson

World Progress Report available - 6 days left

Amelia Downs, the girl in the video above, sent this in with her college application to Tufts University. The university encouraged applicants to submit short YouTube videos.

Hopefully Amelia gets in, but if she doesn't, at least she got her 15 minutes: New York Times, Boston Globe, and ABC. Also, she can have the satisfaction in knowing she started a worldwide phenomenon called the scatter plot.

[via infosthetics]

World Progress Report available - 6 days left

The online dating world can be a confusing place. How do you interact with others? Who should contact? What should you say about yourself? There are a lot of decisions to make, but it all starts with your profile picture when it comes to grabbing the attention of potential dates. Online dating site, OkCupid, analyzed over 7,000 profile pictures, debunking four myths:

1. It's better to smile
2. You shouldn’t take your picture with your phone or webcam
3. Guys should keep their shirts on
4. Make sure your face is showing

Some of the results are pretty surprising. For example, men's photos were most effective when they weren't looking at the camera and not smiling:

It was the opposite for women. A flirty face or smiling while looking at the camera showed most effective:

Catch the full analysis here.

[Thanks, Tom]

World Progress Report available - 6 days left

Back in May last year, the US government launched Data.gov as a statement of transparency, and the Internet rejoiced. After the launch, excitement kind of fizzled with the actual Data.gov site, but big cities like San Francisco, New York, and Toronto got in on the open data party.

Then just a couple of weeks ago, Data.gov.uk launched, which brought me back to the US counterpart. How do the two compare? Here's my take.

Behind the Application

The two applications are very similar on the surface. They catalog government data. Look a little closer though, and you'll see that they're actually really different in purpose, design, and end results. These key differences stem directly from those who were involved in the creation of each.

Data.gov: According to FAQ, it was developed by the Federal CIO Council, as in Chief Information Officers Council. Here's their main role in government, which sounds a lot like information science:

The CIO Council serves as the principal interagency forum for improving practices in the design, modernization, use, sharing, and performance of Federal Government agency information resources.

Data.gov.uk: It was a much more tech-oriented operation from the UK side with Tim Berners-Lee and Nigel Shadbolt advising. Shadbolt is a professor of artificial intelligence at the University of Southampton and Berners-Lee is credited with inventing the World Wide Web.

Winner: Toss up. While the computer science backgrounds can lead to good implementation, the information science crew took on organizing the many many US agencies. However the UK had the Web inventor. Yeah, gotta give the edge to the UK.

Data

Now let's get into the actual data. When Data.gov first launched there were only forty something datasets, but the collection has since grown to several hundred. Data.gov.uk, on the other hand, started with a few hundred.

Data.gov: It links directly to data files in various formats including CSV, XML, Excel, and KML. A lot seems to be lacking though. For example, there's no basic demographic data like population from the Census Bureau. You think that's where they'd start. Maybe the Open Government Directive, which instructs Executive departments to publish three "high-value" data sets, might help this along.

Data.gov.uk: Instead of hosting the data, the UK took a link catalog approach. To the end user this doesn't make a huge difference. As long as you get the data, you're good, but from a developer standpoint, it's a lot easier to catalog links than files.

Winner: Data.gov.uk. A quick browse through the available data sets on both sites will show you the wider range of topics on Data.gov.uk.

Design and Usability

One looks like a government website. The other sports a more modern design.

Data.gov: At launch, Data.gov didn't look all that bad to me, but then I tried to use the site and find some data. The main flaw I saw was in the data browser. I wasn't looking for a particular data set. I just wanted to browse, so I'd select an agency and search. The problem is that a lot of the listed agencies don't have any data cataloged.

Data.gov.uk: Navigation through the site was very familiar using common netspeak like tags and apps. I felt more engaged.

Winner: Data.gov.uk. Both could use a data preview though.

Projects

Data.gov: Other than the "join the dialogue" section on a separate domain and the pitch on the homepage, Data.gov makes little effort to highlight or promote any projects that use the data from the site. The focus is on a repository. What you do with the data doesn't seem to matter much. At least Sunlight Labs is making an effort.

Data.gov.uk: Lists recent apps and has an idea submission section. They clearly want you to use the data with a developer-centric approach.

Winner: Data.gov.uk. It's all about engaging with data over creating a catalog.

Bottom Line

While Data.gov.uk was just recently launched publicly, it has many advantages over Data.gov. It's easier to use and geared towards developers, who, let's face it, are the only ones who are going to do more with the data than open it up in Excel. Data.gov has some catching up to do. Both still have a long way to go. Both are good steps in the right direction.

World Progress Report available - 6 days left

David Spiegelhalter is a Professor of the Public Understanding of Risk at Cambridge University. He studies the choices we make, and how those choices can have an effect later on.

As he explains in the video below, we have a risky side (the part of you that says it's okay to eat a pound of bacon every day) and a cautious side (the part that says to eat some oatmeal in place of bacon).

Things get really interesting when you start to look at not the individual choices, but how one choice affects the next. Is knowledge of the outcome enough to make you change your path? How do you find that balance between risky and cautious?

I don't know much about the subject, but it's easy to see how it could be really interesting to learn. Now if you'll excuse me - my bacon is ready.

[via @hadleywickam]

World Progress Report available - 6 days left

Update: I had scheduled this post for next week, but apparently, Data.gov.uk launched today. The site isn't loading for me right now though. I guess they weren't prepared for traffic.

Data.gov, a catalog of US data, launched last year. Now it's the UK's turn. Well, not yet. But soon. Data.gov.uk is still under lock and key, but it has granted access to some developers. Ito Labs, the group behind mapping a year of OpenStreetMap edits posted screenshots of their maps that show vehicle counts (above).

Here are some comparison maps between 2001 and 2008, by vehicle type.

Once Data.gov.uk is up, it'll be interesting to see how it compares to its US counterpart. Even more interesting will be the projects that come out of it.

Despite all the broohaha over Data.gov, not many useful projects (or datasets) come to mind. Can you think of any? There's still a long way to go from both sides of government and developer.

[Thanks, Oliver]

World Progress Report available - 6 days left

This interactive by Las Vegas Sun describes how in the long run, you're going to lose every single penny when you throw your hard-earned money into a slot machine. In the short-term though, it is possible to win. It's all probability. It's also why statisticians don't gamble. Nobody plays a game that he's practically guaranteed to lose, unless you're a masochist - or you're Al Pacino in that one horrible sports gambling movie with Matthew McConaughey.

One clarification on the snippet about payout percentage.

Here's what the graphic reads:

This is the ratio of money a player will get back to the amount of money he bets, which is programmed into the slot machine. If a machine has payout percentage of 90 percent, that means 90 percent of the money someone bets should statistically be won back. It means a player is not likely to lose 10 percent of the amount initially put into the machine, but rather 10 percent, on average, over time.

The wording is kind of confusing. To be more clear - over time, on average, you'd lose 10% of the money you put in per bet. This is an important note, because it's how casinos make money. For example, when you play Blackjack perfectly (sans card-counting), you'll lose on average 2% (or something like that) per hand, so play long enough, and you're going to lose all your money.

Imagine you have two buckets. One is filled with water. The other is empty. Transfer the water back and forth between the two buckets. Some of the water drips out during some of the transfers. Eventually, all the water is on the ground.

Ah yes, intro probability is fun. Play the virtual slot machine and do some learning for yourself.

[Thanks, Tyson]

World Progress Report available - 6 days left

Some stuff hasn't changed all that much. Entertainment for example, is up less than a percentage point for the average American, and that change is just recently.

Apparel, on the other hand is a different story. Spending portion wasn't that high to begin with, so the 2+ percentage point drop is significant:

Take a look at the interactive to explore for yourself.

World Progress Report available - 6 days left

What? I don't see anything wrong with it.

Alright, alright, so it's local news, but still, come on. I wonder if the newscaster even bats an eye as he's reading the numbers off the teleprompter:

[via @kevinthepang & @eagereyes & Wonkette | Thanks, all]

World Progress Report available - 6 days left