Dependencies

In my current employment my job is to manage the flow of large amounts of data for quite a well-known university. It sounds quite dull but it's actually really exciting because I don't just get to maintain data I get to write cool stuff that uses the data too. For example, every five minutes I get fed a list of all the PCs in the university and whether there's anyone using them, so while I was going through the process of making sure this data was being stored and managed correctly, I took a few hours to write a little web app for students which draws the uni's workstations on a map and tells them where their nearest available one is. I've had loads of positive feedback and the site gets hundreds of hits per day, which is why my job's so rewarding.

My predecessor and mentor had similar experiences - he used live information fed to the university from the council's transport department to produce a website which gives up-to-the-minute bus information, and because we re-publish the data in a sensible format, anyone who can program can write a website or smartphone app that does the same or similar things. Just recently the council changed their data provider and I've spent large amounts of the last month (and probably the coming month too) hacking the code on our server to ensure that no external apps break during this transition phase. OK, some downtime is unavoidable, but in a month's time if there are apps written six months ago that no longer work, I've done something wrong. This is really important to me, as I feel that by publishing this data and having people rely on it, we have a duty to those who trust our data. I really don't want to have to track down everyone who's using our data and tell them they need to re-write their code because I've changed the format, and I certainly don't expect any developers who've written apps to keep checking our site to make sure we haven't just changed the format without telling anyone.

But this is why it pisses me off when others don't do the same. I've had so many troubles writing Facebook apps that I just don't bother any more - if I need data from Facebook I screen-scrape it. You think the front-facing parts of Facebook change all the time, well you should try using the API. I've written so many things that worked for months and then simply broke without warning because Facebook decided to change something. The most recent example is their sudden removal of user RSS feeds, which I've been using for years, as I tend to use Google Reader rather than logging into Facebook. I write quite a lot of Twitter-related scripts too, and I noticed while checking something in their docs today that they indent to deprecate version 1 of their API "within the coming months" in favour of version 1.1. It's not a simple transition either, things that before required no authentication now require OAuth, which is a real pig for a programmer because things I used to be able to do in one line of code (eg getting the public tweets of a particular user) now requires me to implement an entire authentication pipeline, which will probably take me hours.

The biggest bane of my life in recent years was in my previous job as a computer science researcher. I worked on a pervasive device for sufferers of memory loss, and a part of its functionality - recognising the faces of friends and colleagues - was provided by an external service known as Face.com. We used the service for about a year before the company was bought by Facebook, and as soon as this takeover happened they shut down their API, effectively making our system useless overnight. All the training data we'd provided to Face.com was lost, and even if we had found another service, we would have had to start again from scratch. Cheers, guys.

Google seem to be doing the right thing - I've written many Google Maps applications, the first of which was the places part of this very website. The Google Maps API is now up to version 3, which I use when writing anything new, but the places page still uses version 1, and it still works. I've not had to re-write anything. Although v1 of the API is officially deprecated and unsupported, Google have kept it live so that apps written using it don't break. It amazes me that organisations such as Facebook have so much contempt for developers that they can't keep old APIs active, or at least consistent, despite being worth billions of US dollars. If you can't support an API, you really shouldn't provide one.

I doubt anything relies on the data made available by Madhouse Beyond, but I promise you now I take data dependency very seriously. The URL structure of Madhouse Beyond changed considerably in the last year, but I made absolutely sure that with a few exceptions (the text speak translator has gone because it's not funny any more) all the old URLs redirect properly. I did this myself, because I care, and it's sad that I seem to be one of a very small number of people who do. I just hope that as the world wakes up to the possibilities of linked open data, app developers gravitate towards data sources that actually bother to keep their formats consistent. This will force less competant providers to improve their practices or simply fade away.