Category: Social Data Analysis

  • The Current State of Social Data

    Posted Jun 16, 2009 to Social Data Analysis / 7 comments

    Check out my guest post on The Guardian's Data Blog on the current state of social data applications. There are what seems like a ton of them but none of them have really taken off (yet).

    While the post is more of an overview of what's available, I'd like to start a little discussion here on why these data apps haven't gained more popularlity. There always seems be a lot of buzz around launch time, but then it fizzles.

    Are people just not interested in interacting with data or do we need to approach the whole social data puzzle from a different angle?

    Subscribe to the RSS feed or follow on Twitter to stay updated on what's new in data visualization. All the cool people are doing it.

  • 3 Applications that Tap Into the Wisdom of Crowds

    Posted Sep 30, 2008 to Social Data Analysis / 3 comments

    James Surowiecki writes in The Wisdom of Crowds that the group is smarter than the individual (under four conditions). Essentially, the premise is that if you get enough different people to work on a single problem independently, you're going to get as good or better results than that of a small group of experts working together. Think of it as advanced crowdsourcing.

    These three applications tap into the wisdom of crowds. It's clearly election season.

    Prediction Markets

    Intrade works like a stock market, but instead of buying stocks, you buy shares in your opinion. For example, if you think Barack Obama is going to win the election, invest some money in him. The price of the contract changes with demand in the same way stock prices change for Google. Below is a prediction of election results based on what is currently invested by Intrade users.

    [Thanks, Max]

    Predict the News

    Predictify pays users for correct predictions of the news. I've earned a whopping 4 bucks. Topics vary from current events, to pop culture, to sports. You can also put in your own questions to gain some of insight from the crowd.

    Social Data Analysis

    FlowingData readers are familiar with Many Eyes by now. Users upload datasets and evaluate what they see via a mix of visualizations. Users are also encouraged to evaluate others' datasets and discuss what they see - hence the phrase of the day, social data analysis.

    Do They Succeed?

    Whether these three applications successfully tap into the wisdom of the crowd is up for discussion. I already know what I think. More importantly, what do you think?

  • Tap Into the Wisdom of Crowds, Make Money by Predicting Future Events

    Posted Feb 5, 2008 to Social Data Analysis / 1 comment

    Predictify LogoPredictify takes James Surowiecki's The Wisdom of Crowds to heart. Surowiecki argues that when certain factors are present (for example, group diversity), then the group is always smarter than the individual. Predictify has turned this "principle" into a money-making platform.

    Predictify provides a simple, fun way to engage in current and future newsworthy topics. You can research, discuss and predict the outcomes of real-world events, challenge your friends to private prediction contests, build a reputation based on your accuracy, and even get paid real money when you're right.

    There are two roles at Predictify -- those who answer and those who ask -- and you can do both.

    Contribute to the Wisdom

    Once the questions get posted, you predict the outcomes. If you're correct, you'll get a share of the "pot" and your reputation will improve. The better your reputation and the sooner you make your predictions, the more of the pot you will earn. There's also a social aspect to it with a discussion section and a chance to provide reasons for your answer.

    Request the Wisdom of Crowds

    If you want to ask a question, you can get 200 responses with "basic" results for free or you can get the premium plan for $1 per response which includes "enhanced" results and a "rich data set." Here are some of the questions being asked now:

    • Who will win the California 2008 primary election?
    • Which conference will win the Pro Bowl?
    • How many Grand Slam tournament wins will Maria Sharapova have in 2008?
    • What will be the total box office gross for the opening weekend of the upcoming movie "Jumper"?

    Implications for Social Data Analysis

    The questions getting asked on Predictify are fairly straightforward, but for those interested in social data analysis, it'll be a site worth watching. With a monetary incentive, people are eager to answer, which in turn provides for a lovely case study. Since every posted question comes with a discussion section, it'll be interesting to see the reasons why people give the answers they do.

    Do people usually answer correctly? How well do they answer questions with no pot? Do low reputation and high reputation users answer differently? What would happen if we were able to ask more complicated questions or request an analysis of a large data set? Should I post these questions on Predictify?

    UPDATE: I wrote this before the Super Bowl, but now that it's over, check out the predictify results for the game. Spot on.

  • Going Beyond Collaborative Visual Analytics with Statistics

    Posted Jan 15, 2008 to Featured, Social Data Analysis / 4 comments

    Going Beyond Collaborative Visual Analytics with Statistics

    Jeffrey Heer et al. writes in Design Considerations for Collaborative Visual Analytics about a couple of models for social visualization -- information visualization reference model and the sensemaking model. The former is a simpler, more straightforward model starting with raw data -> processed data -> visual structures -> actual visualization; while the latter is a bit more complicated with similar stages but with feedback loops. My main reflections weren't so much with the ideas proposed by the paper. Rather, I'm more interested in what was not mentioned -- not only in this paper but in other social data analysis papers.

    Continue Reading

  • Sharing Personal Data to Push Social Data Analysis

    Posted Nov 19, 2007 to Social Data Analysis / 1 comment

    I'm staying in a hostel here in Madrid and am currently in the "Internet Room." I'm on my laptop, but there are six desktop computers in front of me, all of which are occupied. Three of the six people have Facebook open plus myself. It's come to the point that Facebook has so many ways to share information, that almost everyone can find some use for it. Is there some way to share data in some similar social way?

    I know there's some data blogging available and a few social data sites, but they don't have the same feel as Facebook. I think the main reason people like Facebook (other than an entertaining way to waste a few hours) is because they personally relate to the information displayed and there's some kind of connection between friends and strangers.

    Something Social Data Analysis Needs

    For social data analysis to work, designers will have to take a few pointers from the popular social network. The current approach seems to be throw as many data sets at the user and hope that something sticks. It doesn't seem to be working. It's been written in some papers that people are greater intrigued when there's a personal connection (Jeffrey Heer's paper is the first that comes to mind right now...Socializing Visualization...I think that's it). I can see how data can be completely disconnected from the user if s/he hasn't actually collected it or is directly related to someone s/he knows. I, myself, feel that disconnection all the time. It always take some effort (or a good shove) to get into it.

    Personalizing Data

    It's really easy to say, "Oh, improve social data analysis results just by making the data more personal." In practice, it's hard. In my efforts to collect data about myself, I fell pretty short. I always forgot to record and would lose numbers; things just seemed to always get in the way, and that was with me -- someone who actually cared.

    However, with the rise in mobile technology (who wants to buy me an iPhone?) and sites like Twitter, there seems to be some light. People are posting extremely frequently on whatever is going on at the time, what they're feeling, thinking, etc. I can imagine some kind of data collection or recording with a Twitter feel to it.

    The forever continuing project in the UK, Mass Observation, gives me some hope. If people are willing to write about daily life in Britain or count odd things like the number of people who wear hats, then clearly there has to be a small, medium, or large group of people who could do something special with data sharing. It could be a neighborhood, a group of friends, a family, a class, or a university. There's a niche for everything.

    I'm not sure what avenue would allow and inspire people to engage in that type of data sharing. Is it something like Twitter, Facebook, some combination of the two, or something that hasn't been created yet? Nobody knows for sure yet.

    However, once that outlet for personal data sharing becomes available, I think social data analysis will develop into something really successful. People will be analyzing data that they care about. Until then, it looks like real, in-depth results won't result unless a group of statistically-minded individuals are involved, because nobody else cares.

  • Social Data Analysis by the Swarm

    Posted Jul 4, 2007 to Social Data Analysis / Add your comment

    Swarm Theory

    Swarm Theory, by Peter Miller, talks about how some animals, as individuals, aren't smart, but as a group or a swarm, they can do amazing things. The above is a flock of starlings that can change shapes even though no single bird is the leader.

    Can we apply swarm theory to social data analysis? As individuals, we might not be able to hold onto or understand a dataset, but as a group, we can come at a dataset from different perspectives, look at very small parts, and then as an end result -- extract real, worthwhile meaning.

    That's how swarm intelligence works: simple creatures following simple rules, each one acting on local information. No ant sees the big picture. No ant tells any other ant what to do. Some ant species may go about this with more sophistication than others. (Temnothorax albipennis, for example, can rate the quality of a potential nest site using multiple criteria.) But the bottom line, says Iain Couzin, a biologist at Oxford and Princeton Universities, is that no leadership is required. "Even complex behavior may be coordinated by relatively simple interactions," he says.

    It reminds me of that common saying, or maybe it's a quote, about how if you put a bunch of monkeys in a room with typewriters, you'll eventually get the works of Shakespeare via the magic of probability. While the whole monkey thing is a bit far-fetched, swarm theory is certainly worth my attention.