your.flowingdata (YFD), a Twitter application that lets you collect data about yourself, is now LIVE!
I feel like I’ve been working on this project forever, but it’s finally at a place where I think it’s ready for human consumption. And unlike the previous version, what you track is completely up to you.
How to Get Started
You can start collecting data about yourself in just a few easy steps:
- First off, follow @yfd on Twitter.
- Second, sign in to your.flowingdata with Twitter.
- Once you’re logged in to YFD, you’ll see a link to a quick start guide. Follow the step-by-step directions and you’ll be tweeting data in no time.
Once you’ve started tweeting data, it’ll take about two minutes (usually less) for your data to appear on YFD.
Patterns Over Time
The real strength of your.flowingdata is exploration of personal trends over time. Like most data, your data is going to get more interesting the longer you collect.
This is why you use YFD the same way that you use Twitter. YFD was designed to be unobtrusive. You simply send direct messages about what you’re doing, what you have done, or what’s going on around you.
Collect regularly, and you’re going to see something interesting.
If you just log a few data points about yourself, you’re not going to get much out of it; however, if you log a few data points per day, then you’re on to something. Imagine if Al Gore only showed one temperature value in his now-famous global warming presentation. It’s the cycles that mattered – not the single data point.
Remember – you want a whole story (about you), not just a few random words.
Explore Your Data
YFD helps you tease out the story in your data. It’s all about interaction and exploration, largely inspired by IBM’s Many Eyes, except it’s all about you.
Maybe you’re interested in categories, like the range of your daily emotions. YFD provides a searchable treemap that can be browsed by time:
You could be interested in daily, weekly, or monthly patterns. Use the searchable and interactive stacked time series chart, much like the popular Baby Name Wizard:
You can also browse via a calendar interface like the one at the beginning of this post.
How about a word cloud? That’s searchable too:
Again, all of these visualizations are browseable by time. Track your life in data long enough, and you’re going to spot something interesting, and hopefully, have some fun in the process.
An Intertwined Stream Without a Dashboard
With that said, some of you might be wondering where the dashboard is. The previous version did after all have a drag and drop dashboard-y feel to it. But I decided to get away from that on this go around.
I just feel like when I put my data in widgets and modules and boxes, it seems disjoint, while in reality, all the choices we make are intertwined.
Your blood sugar, for example, is directly affected by what you eat, but is there anything that changes your eating habits? Maybe mood. Maybe exercise. All of it? It’s an intertwined chain of events and sort of flows like river. There might be a boulder way upstream that steered the water to where it is now.
Of course I could be completely wrong and maybe we really do need a dashboard for this type of data, but for now, the focus is on exploration.
Your Private Data Stream
In the early beginnings of YFD, all data was public. I changed that. Now all data is private. The only person who can see your data is you. I did this for two reasons.
The first reason is that I didn’t want to add more noise to the stream. There’s enough as it is and there’s only so much you can take. I mean, I have enough of my own data. I don’t need to keep track of others’. It’s kind of interesting to peep in on others’ lives, but in the end, it doesn’t do you much good. I think your Twitter public timeline is a good threshold for what to share and what not to.
The second reason is that there are a lot of aspects of your life that you don’t want to share with the world but are worth keeping track of. You don’t have to look very far for examples.
I saw this immediately after I started tweeting data about myself. Once you log one thing, a lot of other stuff tends to pop up in your head.
So yeah, I take data privacy seriously. I’ve been toying around with the idea of the choice to make some data public and some not, but that’s further down the timeline.
Much More to Come
This project is far from final, but I think YFD is pretty useful as-is. Some of the major plans I have for YFD include customizable pages, integration of more statistical analysis techniques, and alerts and and pattern recognition.
Finding YFD Help
Please post all of your YFD questions, comments, or bug reports to the new your.flowingdata Forum. I’ll be checking it regularly, and knowing me, probably multiple times a day.
If you don’t know what OAuth is or don’t know a lick (or care) about programming or implementation, feel free to skip the next section or get right to tweeting data on your.flowingdata.
For the Nerds
For the rest of you, here are a few quick facts about how I implemented YFD.
The application itself was built with Django, a Web framework in Python. The prototype was first conceived with a few PHP scripts, so I did try CakePHP, but after a few frustrations, I figured it was a good of a time as any to finally learn Python. I’m glad I did.
Several of the YFD visualizations make use of Jeffrey Heer’s Flare visualization toolkit. It’s in Actionscript 3 and works pretty well. Because Flare has so much built-in functionality though, it did take a little time sifting through all the documentation to figure out what I wanted to do. However it saved me a lot of time in the end.
I also used Garreth Watts’ jQuery sparklines plugin. It was a piece of cake to use. By the way, if you’re doing javascript without jQuery, you’re seriously missing out on some javascript goodness.
Finally – OAuth. A lot of you were pretty vocal about this one. YFD now uses OAuth, so that you don’t have to give your Twitter password to a third-party application.
Disclaimer
Please keep in mind that I am only one graduate student with a thesis to finish. YFD is part of my research in self-surveillance. While I’ll do my best to make sure YFD runs smoothly, I will almost without a doubt run into some kinks. Please help me out by reporting any bugs or problems in the your.flowingdata forum.
Also, in the case that @yfd doesn’t follow you back very promptly, usually within a few minutes, it’s likely that @yfd has hit a daily limit for the number of follows in a day. Check back the next day. If it’s been more than 24 hours, please let me know in the forums. I’ve already talked to Twitter about this, and they’re working on a solution, but it won’t be for a while.
Have fun capturing your life in data!
I’m interested in using this, but utterly uninterested in tweeting things directly to it. Can you not let it pick up all public tweets, and analyse that?
One requires constant ongoing effort, the other just needs a quick setup and it’s done (and would work with historical data to boot).
Not saying DM’ing is bad, it may be suitable in some instances, but I very much doubt lots of people are going to want to specifically DM this app all the time. Most of what they’d write they’re happy to put live on Twitter anyway. The few more confidential things they might be happy to DM.
that is something i’m thinking about as well as bringing in existing data streams, but as you might understand that requires completely different mechanisms, namely natural language processing.
i’ll admit sending frequent DM’s won’t feel natural at first, especially if you don’t use Twitter, but honestly, you used to it pretty quickly. at east i did :)
I don’t see it as a question of getting used to it but rather a statement Ian is making. To put it simply, he’d like to see what he’s socializing in public about. You’ve got to admit, it’d be interesting to learn that all you tweet about is “drinking” for example, wouldn’t it? Honestly, I’m surprised Twitter hasn’t done this already.
I think I’d quit following people who all of a sudden started tweeting their jelly bean intake or cigarette breaks. So for a few of us the DM method is actually preferable.
basically what ian is after is analytics for twitter, which i think we can agree is a completely different app. your.flowingdata just happens to use twitter as the data mechanism. it could’ve just as easily been email, text messages, or aim.
Actually, there are enough irrelvant/boring things going into my tweets as it is, and since I have my Twitter updates posted as FB status updates, I’d REALLY hate to have the default be “public”. I’m very glad Nathan did it the way he did.
And geez, be a grown-up. It’s all of typing FIVE ADDITIONAL KEYSTROKES to DM to yfd instead of public. You don’t have to go anywhere special to do it, even using TwitterFox I just typed d yfd drank 16oz coffee (I’ve been wanting to track my caffeine intake for the past several months so THRILLED to have this now). Those 5 keystrokes are less than 10% of what you’re going to be typing in a typical tweet.
Pingback: .: sermad :. » Blog Archive » Capture your life in data
this is brilliant Nate!
I don’t know if you knew, but there are already several embedded applications out there where machines use twitter as a means of logging remotely the states of any measurement device; and uses twitter also as a means of notification if there are needs of maintenance or requiring attention. Your little tool now allows the data to be viewed from a historical perspective. Very neat!
Kudos!
Will
This is really beautiful, Nathan, congratulations.
This is a big part of what I envisioned for lifemetric, but you’ve pulled it off well! Can’t wait to get started using it.
Nathan, you are my hero of the day. Just got my grad school acceptance letter yesterday, I’m a mom of two little boys (ages 5 and 2) and constantly wishing I had a way to “catch” life data in a mobile fashion – this is a very elegant solution since a Twitter DM can be sent via SMS. Thank you so much!!
excellent, congrats! welcome to the club :)
and the SMS via Twitter is one of the primary reasons I chose it as the data reporting mechanism
Good idea, I have thought for a while I would want an app like this. Question though, can you track the time doing X, Y, or Z; say exercise, or doing task A, B, or C?
I think this is completely sensational! Life streaming + Open Twitter Framework + Visual Data….
Thanks, I’m eager to get going.
The twitter folks have to take note of this one – I think this is the type of application that shows the giant potential for twitter as a platform.
Looks like someone’s already dug it on digg.com. …if you-all wanted to take an active role in making it go viral; dig it…and put it on the feed for FARK, etc…
http://digg.com/business_finance/Take_steps_to_control_the_flow_of_your_personal_data
The more I think of this, the more I think there’s a lot of different aspects you can gear the data towards specific areas of interest depending on the verb used.
Pingback: Your Life Visualized | Jordan Salvit
Pingback: Prove you’re the most interesting person – ever! « Lactose-Free Thoughts
Nice, I am looking forward to messing around with this.
BTW, I have built a couple simple apps with CakePHP, _ it is so frustrating,_ there really is a major lack of good documentation around it.
i’ve made a couple of cakePHP apps too, and it wasn’t too bad for the simpler stuff, but i ran into troubles when i needed something more.
i really like Django because I get to do a lot of server-side stuff too with the API and make use of Python modules.
This is fantastic and very close to an application I have wanted for a long time. If you added ONE option, we’d be there. Send reminder tweets at random intervals. I reply with what I am doing (or was doing at the time of the text). Over a fairly short time I get a statistically valid picture of my activity because you have randomized the sampling!
PLEASE DO THIS!
Alerts are on the way, and smart ones at that
:-)
If I may ask, what is the reasoning behind the decision to use/rely on twitter? I’d love to try out/use the app, but I don’t have a twitter account and don’t really feel like getting one.
I do see the fun of twitter but personally I don’t want to get yet another account some where and I also don’t see the point of sharing my moments of “drinking coffee right now” with the world. Would it be possible to make the app such that there’s an access point independent of twitter?
Matthijs: folks use Twitter because cell phones these days are ubiquitous. Where else are you going to have tools/software that permits you to update remotely wherever you are. I suppose one can set it up on facebook app or text messaging app; but Twitter is already done. Plus, there’s nothing that says you’d have to share anything on twitter. I know of plenty of people who use twitter more of as an instant news center then a self-absorbed “look at me” instant stat center. As I understand it, all of the info is merely collected at yfd; not shared
Thanks, Will, you’re dead-on.
I wanted people to log data from anywhere and when it happened instead of waiting for the availability of a computer. With Twitter, people can record data with their cell phone, from their computer, or with automated scripts.
As for sharing, all of YFD data is private.
I’m planning to let people enter data directly from the your.flowingdata website, but that’s further down on the to-do list.
Of course I can see the convenience of using a cell phone to post something. But does twitter have the trademark/copyright for the technology to use a cell phone to post something to the internet?
If it’s just for now because that was the easiest to build, then I can understand that.
No, of course Twitter isn’t the only way to submit via mobile, but there are millions using the service already, so there’s a smaller hump to jump over.
And it’s not just that Twitter makes things mobile. It’s that it’s mobile, plus desktop, but programmatic via the API. It’s a one-stop service – as opposed to users learning how to log data with multiple methods. No matter what method you use, you’re going to send a direct message in the exact same format every time.
Pingback: » Visualization sweep CQ2
I’ve loved YFD since I first started using it and have used it to track wines that I drink for later reference, but just started an eight week paleo diet challenge today and was going to manually write it down until I saw this post and checked out the new interface…the new release is so awesome, great job!
I also wanted to add that I really like being able to edit the actions log – not sure if it is more obvious to other people, but I only just now saw this URL http://your.flowingdata.com/actions/log/2009/ which comes in handy if you accidentally post something after midnight and it appears for the next day, so that you can go back and edit the timestamp.
Pingback: links for 2009-07-30 « Through the Looking Glass
Pingback: links for 2009-07-30 « The Future of Self-Knowledge
I’ve been intrigued about YFD since accidentally finding it a week ago- mostly that it is a largely open ended data system where us users can develop a language/taxonomy for reporting data.
Blogged it here
http://cogdogblog.com/2009/07/29/yfd/
I’ve been tracking my blood glucose levels and dream of a device that can DM my numbers from my meter directly to YFD!
thanks, alan. i couldn’t have described it better myself. by the way, individual posts on cogdogblog are throwing errors at the moment.
Pingback: Academic Productivity » Testing the general model of productivity
Pingback: Jacob Toft Pedersen (pedersen) 's status on Friday, 07-Aug-09 06:27:50 UTC - Identi.ca
Pingback: Jacob Toft Pedersen (pedersen) 's status on Friday, 07-Aug-09 06:27:30 UTC - Identi.ca
Really looking forward to seeing how my Twitter looks! Fine work guys :)
Pingback: links for 2009-08-15 « Blarney Fellow
Hi Nathan,
I just started using your.flowingdata – thanks for the work you have put into this project, I am looking forward to tracking my progress.