Back in May last year, the US government launched Data.gov as a statement of transparency, and the Internet rejoiced. After the launch, excitement kind of fizzled with the actual Data.gov site, but big cities like San Francisco, New York, and Toronto got in on the open data party.
Then just a couple of weeks ago, Data.gov.uk launched, which brought me back to the US counterpart. How do the two compare? Here’s my take.
Behind the Application
The two applications are very similar on the surface. They catalog government data. Look a little closer though, and you’ll see that they’re actually really different in purpose, design, and end results. These key differences stem directly from those who were involved in the creation of each.
Data.gov: According to FAQ, it was developed by the Federal CIO Council, as in Chief Information Officers Council. Here’s their main role in government, which sounds a lot like information science:
The CIO Council serves as the principal interagency forum for improving practices in the design, modernization, use, sharing, and performance of Federal Government agency information resources.
Data.gov.uk: It was a much more tech-oriented operation from the UK side with Tim Berners-Lee and Nigel Shadbolt advising. Shadbolt is a professor of artificial intelligence at the University of Southampton and Berners-Lee is credited with inventing the World Wide Web.
Winner: Toss up. While the computer science backgrounds can lead to good implementation, the information science crew took on organizing the many many US agencies. However the UK had the Web inventor. Yeah, gotta give the edge to the UK.
Data
Now let’s get into the actual data. When Data.gov first launched there were only forty something datasets, but the collection has since grown to several hundred. Data.gov.uk, on the other hand, started with a few hundred.
Data.gov: It links directly to data files in various formats including CSV, XML, Excel, and KML. A lot seems to be lacking though. For example, there’s no basic demographic data like population from the Census Bureau. You think that’s where they’d start. Maybe the Open Government Directive, which instructs Executive departments to publish three “high-value” data sets, might help this along.
Data.gov.uk: Instead of hosting the data, the UK took a link catalog approach. To the end user this doesn’t make a huge difference. As long as you get the data, you’re good, but from a developer standpoint, it’s a lot easier to catalog links than files.
Winner: Data.gov.uk. A quick browse through the available data sets on both sites will show you the wider range of topics on Data.gov.uk.
Design and Usability
One looks like a government website. The other sports a more modern design.
Data.gov: At launch, Data.gov didn’t look all that bad to me, but then I tried to use the site and find some data. The main flaw I saw was in the data browser. I wasn’t looking for a particular data set. I just wanted to browse, so I’d select an agency and search. The problem is that a lot of the listed agencies don’t have any data cataloged.
Data.gov.uk: Navigation through the site was very familiar using common netspeak like tags and apps. I felt more engaged.
Winner: Data.gov.uk. Both could use a data preview though.
Projects
Data.gov: Other than the “join the dialogue” section on a separate domain and the pitch on the homepage, Data.gov makes little effort to highlight or promote any projects that use the data from the site. The focus is on a repository. What you do with the data doesn’t seem to matter much. At least Sunlight Labs is making an effort.
Data.gov.uk: Lists recent apps and has an idea submission section. They clearly want you to use the data with a developer-centric approach.
Winner: Data.gov.uk. It’s all about engaging with data over creating a catalog.
Bottom Line
While Data.gov.uk was just recently launched publicly, it has many advantages over Data.gov. It’s easier to use and geared towards developers, who, let’s face it, are the only ones who are going to do more with the data than open it up in Excel. Data.gov has some catching up to do. Both still have a long way to go. Both are good steps in the right direction.
While Eurostat is statistics and not data, I’d love a comparison with that as well, http://ec.europa.eu/eurostat. The same goes for UN’s data, http://data.un.org/. Are there any other national or regional data sets out there that we should know about?
Data.gov doesn’t host the data either; it is also a link catalog.
right… i guess i meant they link to actual data files, which leads to its own set of problems.
Hi Nathan,
I’ve put together a Data.gov browser using Tableau Public: Data.gov Browser. The data sources covered include XLS, CSV and XML files, and it gives you a nice overview of where most of the data sets originate.
-Robert
DATA.GOV.UK REPRESENTS ONLY CERTAIN DATA, CONTRIVED BY THE GOVERNMENT, AND DELETES AND COVERS UP OTHER METHODS. IT’S CORRUPT, THE GOVERNMENT USES “BOOK-COOK 5.1” .
data.gov.uk SPARQLs, therefore wins (well, if they get some meaningful data in RDF format that is).
It’s really not a government’s place to take part in web2.0 shiny mashuppery, but to get the data out, as transparent as possible. UK’s Semantic Web approach is the way to go: pure data, and documentation about it.
The current link catalogue really isn’t what we’ve been promised or what we wanted when we talked about the machine-readable state
Nathan,
Coincidentally, last night I visited data.gov for the first time since it launched, hoping more datasets were in the collection. My feelings are the same about usability–it is set up like a government website and while I commend the attempt toward “transparency”, the website buries the data to the point of frustration. Have you considered submitting your suggestions for improvements? http://datagov.ideascale.com/
If you haven’t seen it yet, you might find this article by Nat Torkington interesting. He identifies the various issues and stakeholders that are interested in open government data, and how they might not want the same outcome. http://blog.okfn.org/2010/02/04/rethinking-open-data-lessons-learned-from-the-open-data-front-lines/.
I do find it really puzzling that Data.gov doesn’t even include all the data that’s already been publicly available on agency websites for awhile, including major data sets like the CDC’s National Health and Nutrition Survey. (Though these agency sites have their own problems, which I wrote about here http://bit.ly/6jSbAp and here http://bit.ly/7xAJ6y.) Obviously, those data sets are already out there, but I imagined part of the function of Data.gov was to provide a centralized place to locate them. Perhaps not? It seems like there’s a lot of sorting out to do, in terms of what we want, why, and what kind of financial and political will there needs to be to make it happen.
Could the differences in web design and data access be related to the relative sizes of the governments involved? I.e., is data.gov set up to be more scaleable then data.gov.uk? or is it less scaleable?
For example, there at 316 million people in the USA vs. 61 million in the U.K. I’m guessing the ratio of data sets to be made public would be far greater in the USA than in the UK, and the web site design would have to be set up to scale to much larger number of data sets and sizes.
[Btw, you may want to check out this very hard to read visualization of the UK population: http://bit.ly/bWQWns ]
Pingback: SearchCap: The Day In Search, February 4, 2010
Craig Thomler, a gov2.0 leader in Australia, wrote an interesting blog post on the launch of the data.gov.uk site, and how in his opinion it leapfrogged the US site. He then went on to put together his own wishlist of what could be done to make the data.australia.gov.au site leapfrog them both and has asked for feedback on what else could be done. It’s worth taking a look at his blog post and contributing. http://bit.ly/aRwXgn
Don’t know where you got the figure of “several hundred” data sets on Data.gov. It reached 100,000 last July: http://j.mp/BXtFS
i was going off this table on data.gov for “raw datasets’:
http://www.data.gov/reports/agencyparticipants
but yes, now i see they’ve split up geodata and raw data. why they don’t have some kind of universal search is beyond me.
Data.gov definitely not user-friendly, and I do think, unless there are some legal issues I’m unaware of (or if they’re concerned about having some interest group hijack the site) that it would be better if they hosted threaded discussions on visualizations and/or data, a la Swivel or Many Eyes. Hey, all of these sites are works in progress, and I suspect that there will be rapid progress as citizens force their government to follow innovations others have introduced. Virtuous competition.
Pingback: links for 2010-02-05
Pingback: Things I learned this week – #6 | dougbelshaw.com/blog
Pingback: KoopTech » Innovation » Einige Links zu Linked wie Open Data
Pingback: Digest #23 : ::: Think Macro :::
Pingback: Open Data: hoe nu verder? — Ambtenaar 2.0
Nathan,
Very interesting post from our perspective as Im aware of 3 sites already listed by our customers on data.gov.uk and more are adding there projects.
David
Pingback: Données publiques en France: Girondins vs Montagnards ? «
Pingback: Données publiques en France: Girondins vs Montagnards ? | Owni.fr
Pingback: links for 2010-02-04 « Vorsprung durch Burokratie
Pingback: tecosystems » The Future of Open Data Looks Like…Github?