Category: Coding

  • How to Make Your Own Twitter Bot – Python Implementation

    Posted Nov 5, 2008 to Coding, Self-surveillance, Tutorials / 24 comments

    Following up on my post last week about using Twitter to track eating and weight, some of you voiced some interest in creating your own Twitter bot. This post covers how you can do that.

    The Gist of It

    Creating my own Twitter bot was pretty straightforward (much more than I thought it'd be), mostly because Twitter provides an API and the resources to make it that way.

    I wanted something really simple that I could play around with. I just wanted to be able to send a direct message to my Twitter bot, and from there, it would store my data. OK, so here are the basic steps I took:

    1. Create Twitter account for bot
    2. Turn on email notification for direct messages only
    3. Check email periodically for new direct messages
    4. Parse direct messages and store in database

    Continue Reading

  • Code For Walmart Growth Visualization Now Available

    Posted Oct 21, 2008 to Coding, Data Sources, Mapping, Projects / 30 comments

    Code For Walmart Growth Visualization Now Available

    It took me three months to do it, but the code to visualize the growth of Walmart is now available under a BSD license (that means free and open like a leaf in the wind):

    Download Walmarts.tar.gz

    I've included the Actionscript and the Walmart openings data, which should be all you need to create your own Walmart growth visualization, or if you're more industrious, some other type of growth in the world. Let me know if you're able to improve upon my code as there's definitely a few areas that wouldn't mind some improvement.

    So go wild, have fun with it, and let me know if you apply the code to another dataset. (I also wouldn't mind if someone wrote some documentation.)

    UPDATE: I am no longer supporting this code.

  • Grabbing Weather Underground Data with BeautifulSoup

    Posted Jul 9, 2007 to Coding, Data Sources, Tutorials / 3 comments

    Weather Underground is a useful site and a fun place for weather enthusiasts. WU has a bunch of weather data (current and historical) from established weather stations, like at airports, and home stations setup by hobbyists. One problem: most of the data is in HTML tables instead of the CSV format that we like. I say most because you can download hourly data from a single day in CSV, but if you want say, temperature data over the past 5 years, you're kind of at a loss.

    But wait, there's a solution. That solution is BeautifulSoup, an XML/HTML parser written in Python. Um, parse... what does that mean? Basically, the Python script will go through, or look at, a document extracting certain information from that document.

    Back to WU. Like I said, there's historical data in HTML tables like this. I just want the actual mean temperature in Fahrenheit for the past five years or so. I could go to every single page manually and record the temperature in Excel, but why do that when I can make the computer do it for me? Continue Reading