• US Oil Doesn’t Come From Where You Think it Does

    November 21, 2008  |  Data Sources, Mapping

    Where do you think the US imports the most oil from? Most of us would probably say somewhere in the Middle East, but Jon Udell does some number crunching and shows that misconception is false. Canada supplies us with the most oil (according to the US Department of Energy).

    This realization however, isn't the post's punchline. It's how easy it was for Jon to figure this stuff out. With some help from Dabble DB (an app that lets you easily use a database without too much technical fuss), Jon was able to parse the data and map it by region with a few swift clicks.

    We’re really close to the point where non-specialists will be able to find data online, ask questions of it, produce answers that bear on public policy issues, and share those answers online for review and discussion. A few more turns of the crank, and we’ll be there. And not a moment too soon.

    We're gettin' there.

    [Thanks, Tim]

  • New York Times Visualization Lab – Collaboration with Many Eyes

    October 28, 2008  |  Data Sources

    It was just a little over a week ago that The New York Times announced their Developer Network i.e. Campaign Finance API. Yesterday, they announced something more - the Visualization Lab. In collaboration with the Many Eyes group, the Times has rolled out a Many Eyes for data used by Times writers. You can visualize, explore, and comment on data posted at the Visualization Lab in the same way that you can at Many Eyes.

    Today, we’re taking the next step in reader involvement with the launch of The New York Times Visualization Lab, which allows readers to create compelling interactive charts, graphs, maps and other types of graphical presentations from data made available by Times editors. NYTimes.com readers can comment on the visualizations, share them with others in the form of widgets and images, and create topic hubs where people can collect visualizations and discuss specific subjects.

    A Few More Steps

    I said the API was a good step forward. The Visualization Lab is more than a step. No doubt The Times heard what I said about their API and decided to roll with it since I am the head authority on everything. Yes, I'm totally kidding, in case that didn't come across as a joke. Come on now.

    I'm looking forward to seeing how well Times readers take to this new way of interacting.

    [Thanks, William]

  • Playboy Playmate Curves and the State of the Economy

    October 24, 2008  |  Data Sources, Economics

    Terry Pettijohn and Brian Jungeberg of Mercyhurst College took a very close look at the curves, um, measurements of past Playboy Playmates of the Year in relation to the state of the economy.
    Continue Reading

  • Lexical Analysis of Presidential Debates and the Windbag Index

    October 23, 2008  |  Statistics

    lexical

    Martin Krzywinski, whose previous work includes Circos, digs deep into the presidential debate transcripts with tedious manual (or was it automatic?) annotation of words (noun/verb/adjective/adverb), Wordle, and his custom metric called the Windbag index that measures speech complexity.
    Continue Reading

  • Who’s Leading Whom? Predictive Markets Versus Polls

    October 22, 2008  |  Statistical Visualization, Statistics

    This is a guest post from Michael Drumheller, Dirk Karis, Raif Majeed and Robert Morton of Tableau Software. They use Tableau to explore the relationship between polls and predictive markets.

    Predictive markets such as Intrade and the Iowa Electronic Markets have attracted more attention this year than in past Presidential elections. Some political observers such as ElectoralMap.net look to these markets as indicators of who's winning or losing.
    Continue Reading

  • New York Times Rolls Out Campaign Finance API

    October 16, 2008  |  Data Sources

    The New York Times announced the opening of their Developer Network a couple of days ago. It's their "API clearinghouse and community." It might seem kind of weird that a newspaper company has an API, but as many FlowingData readers know, the Times prides itself on innovation.

    The Campaign Finance API is currently available:

    With the Campaign Finance API, you can retrieve contribution and expenditure data based on United States Federal Election Commission filings. Campaign finance data is public and is therefore available from a variety of sources, but the developers of the Times API have distilled the data into aggregates that answer most campaign finance questions. Instead of poring over monthly filings or searching a disclosure database, you can use the Times Campaign Finance API to quickly retrieve totals for a particular candidate, see aggregates by ZIP code or state, or get details on a particular donor.

    For anyone who has tried to play with FEC data, myself included, knows that this API is cool. You could get the data directly from the FEC, but it's a bit of a painstaking process. Now you don't have to sift through a bunch of reports or an awkward user interface.

    The Movie Review API is next in line. After that, who knows, but it's a good step forward for The Times.

    [via serial consign]

  • 3 Applications that Tap Into the Wisdom of Crowds

    September 30, 2008  |  Social Data Analysis

    crowd

    James Surowiecki writes in The Wisdom of Crowds that the group is smarter than the individual (under four conditions). Essentially, the premise is that if you get enough different people to work on a single problem independently, you're going to get as good or better results than that of a small group of experts working together. Think of it as advanced crowdsourcing.

    These three applications tap into the wisdom of crowds. It's clearly election season.
    Continue Reading

  • OneGeology Wants to Be Geological Equivalent of Google Maps

    September 11, 2008  |  Data Sources, Mapping

    There's lots of free geographical data about what's going on at the surface of our planet. It's a different story for what going on underneath though. OneGeology aims to be the solution to that problem.

    OneGeology is an international initiative of the geological surveys of the world and a flagship project of the 'International Year of Planet Earth'. Its aim is to create dynamic geological map data of the world available via the web. This will create a focus for accessing geological information for everyone.

    I've never been one for the geology, but if the data (and interactive maps) were easily accessible, there certainly would be a peak in interest.

    [via msnbc | Thanks, Samantha]

  • If You’re a Criminal on the Run, Don’t Use GPS

    July 11, 2008  |  Statistics

    With all the new technologies we've come to rely on, it's easy to forget just how much data we're automatically logging on our own devices or some central server in the boonies.

    GPS is one such example. Some of us can't imagine going out of town without it. What you might not know is that while that GPS device tells you where to turn left, it is also storing where you go in its memory. Scotland Yard has started using this data to solve crimes:

    Scotland Yard analysis of the [GPS] devices has helped solve dozens of investigations into kidnappings, grooming of children, murder and terrorism. Information about a suspect's whereabouts at particular times, their journeys and addresses of associates can all be discovered - if they have been using a GPS. The devices retain hundreds of records of locations and routes in their memory.

    So all you criminals out there, make sure you use GPS whenever possible. We all know your actions are a desperate cry for attention.

    [Thanks, Tim]

  • FlowingData Cited in Forbes Magazine?

    June 28, 2008  |  Data Sources

    Whaaa? Cool beans.

  • What Do People Want to Do With Their Lives?

    June 17, 2008  |  Data Sources, Projects, Visualization

    43things-viz

    43 Things is a goal-setting community where people set goals, cheer each other on, and connect with others who are trying to achieve the same thing. Even if you're not setting goals yourself, it's still interesting and often amusing to see what others have set out to do e.g. go skinny dipping, have a one night stand, and be myself.
    Continue Reading

  • Our Non-ability to Misunderstand Statistics of Rare Events

    June 4, 2008  |  Statistics

    The DiceCory Doctorow from The Guardian writes about our inability to understand the statistics of rare events. We obsess so much over the near-impossible probability that something could happen that it clouds our vision of more probable events.

    The rare - and the lurid - loom large in our imagination, and it's to our great detriment when it comes to our safety and security. As a new father, I'm understandably worried about the idea of my child falling victim to some nefarious predator Out There, waiting to break in and take my child away. There's a part of me who understands the panicked parent who rings 999 when he sees some street photographer aiming a lens at a kids' playground.

    But the fact is that attacks by strangers are so rare as to be practically nonexistent. If your child is assaulted, the perpetrator is almost certainly a relative (most likely a parent). If not a relative, then a close family friend. If not a close family friend, then a trusted authority figure.

    Says Doctorow, such misunderstanding is why we gamble in casinos and why we have to wait in long security lines at the airport. We see piles of money and terrorist attacks when ultimately, the chances that you'll win a jackpot or pass over violence is much less likely - near impossible - compared to losing all of your money and losing valuables to a curious luggage handler.

    If there's one thing the government and our educational institutions could do to keep us safer, it's this: teach us how statistics works.

    Amen to that.

    [Thanks, Jan]

  • Statistics is a Diverse Field With Different Paths of Study

    May 26, 2008  |  Statistics

    Rows in a Field
    Photo by Duncan H

    One of the huge factors that drew me in to statistics is that you can apply it to so many different areas of study. When someone asks me what the job market is like for someone in statistics, I always tell them, "Wherever there's data, there's a job to fill by a statistician. Marketing, biology, traffic, finance, crime..."

    It's also my way of answering, "What are you going to do when you graduate?" In other words, I'm not sure yet. I keep running into more and more fun stuff I can do with my degree so it's hard to decide right now. But hey, it's better to have too many paths to choose from that not enough, right?

    Interdisciplinary Statistics

    In the most recent Amstat News is a short article - Statistics as an Interdisciplinary Science:

    An issue touched on briefly is statistics as an interdisciplinary science. I think there is a general agreement that (almost) all other scientific disciplines need statistics (and statisticians).

    Speaking to people outside of the field, there's this idea that statistics is very focused (which it is in some ways, I guess) and very narrow, but it's pretty much whatever you want it to be. You can focus completely on say, crime, or you can be more broad and examine issues in social science, for example.

    It's like design or computer science. You might use your skills for very specific areas like page layout or web programming, but just as easily, you could use that know how on a broad range of projects.

    In summary, statistics is awesome. What have you used statistics for lately?

  • U.S. Census Bureau’s 2008 Statistical Abstract – Looking at America’s Data

    May 21, 2008  |  Data Sources

    The U.S. Census Bureau released their 2008 Statistical Abstract, the National Data Book, not too long ago (um, like in January). There are state rankings and data in 30 categories and many more sub-categories. All this data is in the form of PDFs and Excel spreadsheets, which doesn't lend much to readability, but still, it's nice to have access to all the information.

    Maybe FlowingData readers can put together a giant statistical abstract all conveyed through graphics. That would be cool. Above are six data sets that I picked from the billion or so available.

  • The Safest Seat to Sit In On a Plane is…

    May 20, 2008  |  Statistics

    Safest seat to sit on a plane

    Popular Mechanics did a study on where it was safest to sit on an airplane based on all commercial jet crashes since 1971. Contrary to expert statements that "one seat is safe as the other," the study found that it is safer to sit in the back.

    The funny thing about all those expert opinions: They're not really based on hard data about actual airline accidents. A look at real-world crash stats, however, suggests that the farther back you sit, the better your odds of survival. Passengers near the tail of a plane are about 40 percent more likely to survive a crash than those in the first few rows up front.

    The percentages in the above graphic are survival rates.

    [Thanks, Tim]

  • Why Did Andy Dufresne Escape from Shawshank?

    May 8, 2008  |  Statistics

    If I were to skip straight to the part in The Shawshank Redemption when Andy Durfesne climbs out of the pipe of poo (and put it on mute), someone who never saw the movie might see an escaped convict who steals money from a warden and fleas to some random place in Mexico called Zihuatanejo. Out of grief, the warden kills himself and Ellis Boyd "Red" Redding eventually teams up with Andy to commit more crimes.

    Those of us who have seen the movie though know this isn't the case. Why? Because we saw the whole movie and have context.

    Context Matters

    As Andrew, a FlowingData reader, put it, "For statistics to be useful, it needs to be explained in a context." When I get my hands on some data, whether I'm analyzing or visualizing, I want to know the context of data first. I want to know who collected the data, how it was collected, when it was collected, and what was done to it before it arrived in my hands. Without that meta-information, I could easily make an incorrect assumption about the data or misrepresent it somehow in a visualization - which is very bad.

    Simply put, we use visualization and statistics to tell stories with data. If we don't have all the information, then we can't tell a complete story.

  • Data and Statistics For Human Rights

    April 27, 2008  |  Statistics

    Patrick BallPatrick Ball, a human rights statistician, finds truth in numbers while analyzing and consulting to find patterns and uncover scale in crimes against humanity.

    The tension started in the witness room. "You could feel the stress rolling off the walls in there," Patrick Ball remembers. "I can remember realizing that this is why lawyers wear sport coats – you can't see all the sweat on their arms and back." He was, you could say, a little nervous to be cross-examined by Slobodan Milosevic.

    Mr. Ball was the first expert witness called in the case against the former Serbian president, who was representing himself against mass atrocity charges at the International Criminal Tribunal for Yugoslavia. Ball had spent 10 months crunching numbers about migration patterns in the former Yugoslav province of Kosovo; his findings suggested that hundreds of thousands of refugees who fled to Albania were spurred by the violence of Mr. Milosevic's army. By the time Ball entered the tribunal chamber, in March 2002, the ousted leader had a reputation for grand orations rather than direct questions; when Milosevic veered off track, the judge would interrupt. "Milosevic would say, 'Dobro,' and go on...." Ball remembers. "It means, 'OK, very well,' but it was clearly a, 'Very well, we'll have you shot later.' I hear [that] in my dreams periodically."

    Ball is a statistician – not exactly a profession usually associated with human rights defense. But the Human Rights Data Analysis Group that he heads at Benetech, a technology company with a social justice focus, is bringing the power of quantitative analysis to a field otherwise full of anecdote.

    That's right. Statistics is awesome. I dare you to disagree.

    [via Statistical Modeling]

  • Atheist Statistics For 2008 – Do You Believe These?

    April 16, 2008  |  Mistaken Data

    This video shows statistics centered around atheism, claiming that atheism is correlated with a healthy society. I don't want to turn this into a religious debate, but I really don't like these types of videos, slide shows, etc. It's not the ideas that bother me, but because some people think it's a great idea to rattle off a bunch of numbers to "prove" a point. Nevermind the biases, invalid studies, poor analysis, cruddy data, and "results" taken out of context.

    What do you think? Do you buy this stuff?

  • Reflecting on Life After Statistics – R.I.P. Minghui Yu

    April 12, 2008  |  Statistics

    Rachel, one of the organizers of Columbia's Life After Statistics, reflects on lessons learned from the conference and gives respects to a fellow statistician who was lost the night of.

    As one of the organizers of the event, Life After a Statistics Doctoral Program (a conference organized by the doctoral students in Columbia's Statistics Department), I was excited to be invited to guest post on Nathan's blog but then realized that my perception of the event would be so different than that of an attendee that perhaps I shouldn't. Two post-docs from Columbia's Statistics department, Matt and Kenny, agreed that they would post and they did -- once on Andrew Gelman's blog and once on Nathan's.
    Continue Reading

  • What Can You Do With a Degree In Statistics? – A Follow Up

    April 9, 2008  |  Statistics

    This past Friday, Columbia University stat graduate students hosted a symposium on careers for students in statistics. Kenneth Shirley, a stat post doc, was nice enough to write this guest post about the conference so that we can all learn from it. There were two panels - academic and industry - including representation from Google, AT & T, and Pfizer.

    Yesterday's conference at Columbia about career opportunities for Statistics Ph.D. graduates was a great success. It was organized by the graduate students in Columbia’s Stats department and advertised on the web here:

    http://www.stat.columbia.edu/career_conf08/

    Andrew Gelman made some opening remarks, and then there were two panel discussions, each with five professional statisticians. The first panel consisted of academic statisticians, and the second panel consisted of industry statisticians. Here are some comments I found interesting.
    Continue Reading

Copyright © 2007-2014 FlowingData. All rights reserved. Hosted by Linode.