geek

A geeky rant that non-geeks really should read

Nice to see that this is finally getting some press...

Tesco web security 'flaw' probed by UK data watchdog, BBC News
Tesco face enquiry over 'lousy' website security, Telegraph

So what's happened? Basically, Troy Hunt, a software architect, discovered a flaw or two in Tesco Online's security a few weeks back. Geeks can read the whole thing here but for the non-technical, if you use Tesco's website your password is being stored on their server in a decryptable way. This is actually provable - go to any website you have to log into, and use the password recovery function. If the function resets your password to something random or allows you to change it to something you can remember, that's good. If it emails you your password, then that's a broken system. Tesco does the latter. If a website stores passwords on a server (which Tesco must do, in order to email it to you) then all it takes is one hacker to get in and all the passwords are compromised. The story has been picked up by numerous IT professionals - including the CTO of Sophos, Graham Cluley - all of whom criticise Tesco's security.

So all Tesco have to do is start encrypting their passwords server-side, and this whole problem will go away. Instead they came out with this tweet:"Passwords are stored in a secure way. They’re only copied into plain text when pasted automatically into a password reminder mail."

This might calm the layman but everyone with even the basic knowledge of computer security will read that sentence and scream at the insanity of it. It's physically impossible to copy a password to plain text if it's actually stored securely. Secure password storage means one-way irreversable encryption (known as 'hashing').

If that didn't annoy me enough, this tweet was the nail in the coffin..."We know how important internet security is to customers and the measures we have are robust." Which is basically the Twitter equivalent of sticking their fingers in their ears and going "la la la I'm not listening."

OK, so Tesco hasn't been hacked. But that doesn't matter - the world now knows that their security is crap, so 10 to 1 there are already malicious hackers targeting them. And when they get in, because Tesco don't hash their passwords, your security as a customer is at stake, and Tesco will only have themselves to blame for sticking their heads in the sand. Letting a massive security flaw like this lie is like not locking your front door when you go out. Sure, you may not get robbed for years but the one day that the burglars do come, they'll get away with everything with very little effort.

There is no such thing as too much security... and no amount of security is ever enough. Especially when you're trusted with the details of thousands of innocent customers.

As it's ranting season on Facebook

Facebook have recently added a bunch of new features. They've implemented 'friends lists' (also known as the 'circles' feature on Google Plus) and re-vamped the news feed again. That's not what I want to rant about.

In their ongoing quest to become more like their superior cousin, Twitter, Facebook have created (sorry - stolen) the ability to add people without being friends with them. I've always maintained that Twitter's success is due to the fact that you can add people without the add having to be reciprocal, thus not creating this illusion of friendship that exists on Facebook (and Livejournal before it). You can follow strangers, celebrities and friends alike, and they can choose whether or not they're interested in what you have to say. Facebook have now got something similar in the subscription idea - instead of being friends with someone you can subscribe to them. They don't have to do anything, but anything that they post (publically) will show up in your feed, just like Twitter. Presumably it's to entice the hordes of celebrities who are happy to post on Twitter but are aware of becoming 'friends' with random fans they've never met. That's also not what I want to rant about.

What I do want to rant about is that icon. It's the RSS icon. When I click a link on the web that has that icon next to it, I expect my browser to add it to my feed reader. Not so on Facebook, it simply adds their posts to your Facebook news feed, and increases their subscriber count by one, no RSS in sight. Heck, I wouldn't be surprised if it posted something in their news feed saying "Ash is now subscribed to you!" That's not supposed to happen when you click a link that has that icon.

What's even more annoying is that the one place on Facebook from where you still can get an RSS feed now doesn't have the icon, whereas it previously did. Go to your friends' notes page and look at the bottom of the left hand column where it says "Friends' notes"... that's an RSS feed, and until Facebook introduced their subscription feature it had an RSS logo next to it indicating as such. The logo is now gone. Talk about intentionally misleading.

The sad thing is that RSS is an open standard - nobody really controls or owns it. So if Facebook really have intentionally stolen the logo on purpose then there's not really anything anyone can do about it. Hopefully the RSS logo and standard are widespread enough that Facebook will realise it's confusing people and come up with their own icon for subscriptions.

Beware of Link Previews

Hilarity ensued on Twitter this week - an image went round showing a story on the Guardian website with a rather unexpected headline. Thing is, the article didn't actually say that, and although it could have been a clever photoshop, most people seem to think that some clever techie at the Guardian's website modified the page's meta-tags in order to make social media links to the article say something sweary while the actual article is clean as a whistle.

pic.twitter.com/3BgucCVqTl

Whatever the reason, there is a very good point here that lots of people have missed, and that's that it's a piece of cake to fake links on Facebook or Twitter. Even if we ignore the incredibly dangerous practice of link-shortening that Twitter kinda forces you to use in order to keep within the character limit, it appears that modern link-sharing sites try to be clever by showing the user a preview of what they're going to see if they click the link.

But this is really, really easy to abuse. When a web server responds to a web request (eg you, clicking on a link) it will normally respond with the page requested, but it doesn't have to. It can send what it likes. In this case it's really easy to program a web server to respond to Facebook with one thing and everyone else with something else. A while back I did a proof-of-concept of this in action on this very site...

http://www.madhousebeyond.com/cuteandfluffy

It works by sending Facebook the cute and fluffy picture promised, but everyone else gets the scary picture of the bear and the skeleton from Look Around You. The upshot is that if you share any of the links on that page on Facebook, the auto-generated preview will show that the page contains something completely different to what you'd actually see if you clicked the link. Feel free to fool your friends!

Obviously this is just harmless fun and I'm not an actual scammer, but this is actually the method a lot of scams use in order to work. A recent scam on Facebook shows up in your news feed as a link to a 'shocking' video of an horrific rollercoaster accident (which didn't actually happen). The preview makes it look like a link to an actual news site, but clicking on the link takes you to the permissions page for a malicious Facebook app with the same name as a popular news site, attempting to con you into granting access to your account to the scammers. This scam almost certainly works the same way, as there's clearly no photo of a rollercoaster or news story on the page linked to, yet we're so used to seeing 'previews' of links that we don't really notice.

To summarise: beware of Facebook and Twitter links. If you click a link and get something you didn't expect, there's a very real risk that someone's trying to screw with you, you should restart your web browser and return to the page you were originally looking at.

Blaming the Victim

There's a lot of talk at the minute, regarding the recent theft of millions of customers' personal details from Sony's online services, Playstation Network (PSN) and Qriocity. Obviously it's yet another plus point for us paranoid technophobes who don't use the same password for anything, have a separate email address for every service we use and never give out credit card numbers unless we're 100% sure we can trust the security being used, but me saying "told you so" is hardly helpful, and certainly doesn't change the fact that 77 million people are now living in the knowledge that their name, address, phone number and possibly credit card number and password is currently in the hands of a malicious hacker.

But there is a moral dilemma... who to blame? I immediately began badmouthing Sony for this obvious lapse in security, but this morning a colleague of mine pointed out to me that you should never blame the victim; the fault lies with the hacker. This is a very good point, and one echoed by many, some even go so far as to suggest that blaming Sony for this hack is like blaming a shopkeeper for a burglary, or telling a rape victim she was asking for it. I would never blame a rape victim for being raped, nor would I blame a shopkeeper for being burgled. But let's say the shopkeeper were to go home for the night, trusting the locking up to his absent-minded apprentice. Then, let's say the apprentice gets drunk, staggers home leaving the door of the shop wide open, and insults a local gang on the way home before drunkenly daring them to burgle the shop. Would it then be OK to blame the burglary on the apprentice?

For those who don't know, this hack has a history. The hackers almost certainly got in by discovering some weakness in the protocol used to access the Playstation Network from a Playstation 3. This time last year, such a task would have been impossible, but, at the 27th Chaos Communication Congress meeting in Berlin in December 2010, a group of hackers known as fail0verflow presented their work [YouTube] in hacking the PS3. During this presentation, they pointed out that the PS3's security model is fundamentally broken because although Sony uses a pretty damn bullet-proof elliptic curve cryptography method to sign its code, rather than use a different random number each time, they use the same number, which effectively means that anyone with a basic understanding of maths can reverse engineer Sony's private key, effectively rendering the PS3's entire code-signing functionality completely useless. So who do we blame for this... fail0verflow for pointing out Sony's mistake, or Sony for making such a stupid, rookie mistake in the first place?

Soon after fail0verflow gave their presentation, George "geohot" Hotz, the hacker previously known for his work in breaking the security of the iPhone, used fail0verflow's methods to reverse-engineer the master private key of the Playstation 3. Anyone who has this number can write and run any code they damn well like and run it on any PS3 console in the world. It was a godsend to homebrew coders, and I know people who have done some really cool things with it, including one person who wrote some code to use an Xbox Kinect to control a PS3. But in blowing the PS3's security wide open in this way, it's very likely that geohot inadvertently allowed malicious hackers to write code that interfered with the Playstation Network, leading to the theft of 77 million peoples' personal details. So should we be blaming geohot for this mess? Many do.

For my part, we need to go back to fail0verflow's presentation in Berlin. Early in the presentation, the group make a very good point about the PS3's security. The PS3 remained unhacked for 4 years after its release. Many owners of the console wrongly assume that this means the PS3 is very secure, unlike the Wii which was hacked in under a week. But, as fail0verflow point out, when it first came out the PS3 didn't need to be hacked, because it ran OtherOS. This was a piece of software built into the console that effectively allowed homebrew coders to do almost what they wanted with it. This was a happy co-existance for over three years until Sony, for one reason or another, decided to kill OtherOS on existing consoles via a firmware update. At the time I argued that this was a bait-and-switch and that Sony should really be in court for breach of the Trade Descriptions Act... people bought the PS3 knowing they could use it for homebrew and now they've parted with cash they're being told they can't any more. I'm not a lawyer, but regardless, Sony pissed off thousands of hackers with this rather odd decision. This led to the hacking and subsequent discovery of the master private key. The PS3 didn't take four years to hack, it took four years for a hack to become necessary, and then less than a month to hack.

I'm not defending the yet-unnamed person or people who broke into PSN and stole all the customer details, they're clearly bad people. And no, I'd never blame the victim for a crime. But in this case, there are 77 million victims and Sony aren't one of them. Sony, instead, is the incompetant apprentice and a victim only to karma. Perhaps one day they'll learn that people in glass houses shouldn't throw stones... and people who suck at security shouldn't piss off hackers.

BondPotterWho

Quite a long time ago, I noticed that quite a lot of actors in the Harry Potter movies have also been in Doctor Who, and there are also several actors in Harry Potter that have also been in one of the more recent Bond movies. This led to a conversation with some like-minded friends, during which we decided that, like a Bacon number, british actors should have a ranking system based on how many of the UK's national institutions in which they have appeared, and that we should use Harry Potter, James Bond and Doctor Who as our three. We also decided that John Cleese is the only one we could think of who's been in all three.

In fact, there are four. The obvious one, John Cleese, had a cameo role as an art critic in the Doctor Who episode "City of Death", a slightly larger role as Nearly Headless Nick in Harry Potter, and a role as Q's replacement in two Bond movies.

Julian Glover has also been in all three. He's one of those actors who has been in practically everything but nobody really notices him. He's been in Star Wars and Indiana Jones, and two roles in Doctor Who, most memorably Scaroth in "City of Death". He played the villan Kristatos in For Your Eyes Only, and voiced Aragog the spider in Harry Potter and the Chamber of Secrets. Yes, we are counting voice roles.

Helen McCrory is best known as Narcissa Malfoy in the Harry Potter movies, but has appeared in Doctor Who as head vampire Rosanna in "Vampires of Venice", and as the MP Clair Dowar in Skyfall.

Last, but by no means least, we have Greg Bennett. Judging from his IMDB profile he's a background actor for hire, he's been in pretty much everything. He's had multiple roles in Doctor Who, and has also appeared in the other two, albeit in uncredited minor roles.

Interestingly, only three lead actors have "crossed over" as it were. Two Doctors have appeared in Harry Potter; David Tennant and John Hurt (because he counts as a Doctor in my book.) and only one Bond, Timothy Dalton, has appeared in Doctor Who, as the corrupt Time Lord leader, Rassilon.

A few other honourable mentions before the complete list. BBC News presenter Huw Edwards has managed to play himself in both Doctor Who and James Bond. He appears on the television in the Doctor Who episode "Fear Her", and also in Skyfall. BAFTA and Golden Globe winner Bill Nighy has had walk-on roles in both the Doctor Who episode "Vincent and the Doctor", and in the first part of Harry Potter and the Deathly Hallows, and was rumoured to be in the running to play the ninth Doctor back in 2005. Finally, veteran actor Warwick Davis has yet to appear in a Bond movie, but that's not to say he won't one day, seeing as he's been in pretty much everything else!

On the subject of people who have been in virtually everything, Martin Freeman doesn't appear to have been in Doctor Who, James Bond or Harry Potter. I'm sure this will be rectified soon, especially as he clearly knows Steven Moffat.

I'm sure this list will be revised in the future, especially as Downton Abbey and Coronation Street both seem to be melting pots of well-known actors these days. In the meantime, here's a complete list of everyone who's been in more than one of Doctor Who, James Bond and Harry Potter, plus links to IMDB so you can see who they played. Enjoy, stats fans.

| Greg Bennett | (James Bond, Harry Potter, Doctor Who) | | --- | --- | | Julian Glover | (James Bond, Harry Potter, Doctor Who) | | John Cleese | (James Bond, Harry Potter, Doctor Who) | | Helen McCrory | (James Bond, Harry Potter, Doctor Who) | | | | | Roy Stewart | (James Bond, Doctor Who) | | Ingrid Pitt | (James Bond, Doctor Who) | | John Hurt | (Harry Potter, Doctor Who) | | Peter Roy | (James Bond, Doctor Who) | | Imelda Staunton | (Harry Potter, Doctor Who) | | Lesley Hill | (James Bond, Doctor Who) | | John Atterbury | (Harry Potter, Doctor Who) | | Jeremy Bulloch | (James Bond, Doctor Who) | | Ralph Fiennes | (James Bond, Harry Potter) | | Shirley Henderson | (Harry Potter, Doctor Who) | | George Pravda | (James Bond, Doctor Who) | | John Hollis | (James Bond, Doctor Who) | | James Bree | (James Bond, Doctor Who) | | Noel Johnson | (James Bond, Doctor Who) | | Anthony Carrick | (James Bond, Doctor Who) | | Michael Percival | (James Bond, Doctor Who) | | George Roubicek | (James Bond, Doctor Who) | | Colin Salmon | (James Bond, Doctor Who) | | Christopher Bowen | (James Bond, Doctor Who) | | Jim Conway | (James Bond, Doctor Who) | | Leonard Sachs | (James Bond, Doctor Who) | | Peter Cartwright | (Harry Potter, Doctor Who) | | Simon Fisher-Becker | (Harry Potter, Doctor Who) | | Ellie Darcey-Alden | (Harry Potter, Doctor Who) | | Jessica Hynes | (Harry Potter, Doctor Who) | | Bill Hutchinson | (James Bond, Doctor Who) | | Joseph Frst | (James Bond, Doctor Who) | | Graham Crowden | (James Bond, Doctor Who) | | Carl McCrystal | (James Bond, Doctor Who) | | David de Keyser | (James Bond, Doctor Who) | | Edward de Souza | (James Bond, Doctor Who) | | Cyril Shaps | (James Bond, Doctor Who) | | Zoë Wanamaker | (Harry Potter, Doctor Who) | | Christina Cole | (James Bond, Doctor Who) | | Philip Voss | (James Bond, Doctor Who) | | Mark Williams | (Harry Potter, Doctor Who) | | Marc Lawrence | (James Bond, Doctor Who) | | Peter Brooke | (James Bond, Doctor Who) | | Graham Cole | (James Bond, Doctor Who) | | Ron Tarr | (James Bond, Doctor Who) | | Bill Nighy | (Harry Potter, Doctor Who) | | Alan Talbot | (James Bond, Doctor Who) | | Terrance Denville | (James Bond, Doctor Who) | | Patrick Barr | (James Bond, Doctor Who) | | Dennis Edwards | (James Bond, Doctor Who) | | Barry Andrews | (James Bond, Doctor Who) | | R.J. Bell | (James Bond, Doctor Who) | | Tim Pigott-Smith | (James Bond, Doctor Who) | | Jimmy Vee | (Harry Potter, Doctor Who) | | Jeffry Wickham | (James Bond, Doctor Who) | | Burnell Tucker | (James Bond, Doctor Who) | | Graham Duff | (Harry Potter, Doctor Who) | | Glen Stanway | (James Bond, Harry Potter) | | Neil Hallett | (James Bond, Doctor Who) | | Tony Sibbald | (James Bond, Doctor Who) | | David Yip | (James Bond, Doctor Who) | | Paul Ritter | (James Bond, Harry Potter) | | Jeff Rawle | (Harry Potter, Doctor Who) | | Kristopher Kum | (James Bond, Doctor Who) | | Neville Jason | (James Bond, Doctor Who) | | Geoffrey Palmer | (James Bond, Doctor Who) | | Max Faulkner | (James Bond, Doctor Who) | | Rocky Taylor | (James Bond, Doctor Who) | | Derek Deadman | (Harry Potter, Doctor Who) | | Selva Rasalingam | (James Bond, Doctor Who) | | Warwick Davis | (Harry Potter, Doctor Who) | | Jeremy Wilkin | (James Bond, Doctor Who) | | Michael Byrne | (James Bond, Harry Potter) | | Paul Carson | (James Bond, Doctor Who) | | Hans De Vries | (James Bond, Doctor Who) | | Burt Kwouk | (James Bond, Doctor Who) | | Bill Mitchell | (James Bond, Doctor Who) | | Pip Torrens | (James Bond, Doctor Who) | | Robbie Coltrane | (James Bond, Harry Potter) | | Kerry Shale | (James Bond, Doctor Who) | | David Bradley | (Harry Potter, Doctor Who) | | John Sarbutt | (James Bond, Doctor Who) | | Elizabeth Spriggs | (Harry Potter, Doctor Who) | | Timothy Bateson | (Harry Potter, Doctor Who) | | George Baker | (James Bond, Doctor Who) | | Hugh Bonneville | (James Bond, Doctor Who) | | Norman Jones | (James Bond, Doctor Who) | | Chris Webb | (James Bond, Doctor Who) | | Greg Powell | (James Bond, Doctor Who) | | Jimmy Gardner | (Harry Potter, Doctor Who) | | John Moreno | (James Bond, Doctor Who) | | Eva Alexander | (Harry Potter, Doctor Who) | | Vernon Dobtcheff | (James Bond, Doctor Who) | | Michael Moor | (James Bond, Doctor Who) | | Richard Trinder | (Harry Potter, Doctor Who) | | Lisa Osmond | (Harry Potter, Doctor Who) | | Diana Rigg | (James Bond, Doctor Who) | | David Decio | (James Bond, Harry Potter) | | Catherine Schell | (James Bond, Doctor Who) | | Earl Cameron | (James Bond, Doctor Who) | | Paul Heasman | (James Bond, Doctor Who) | | Garrick Hagon | (James Bond, Doctor Who) | | Honor Blackman | (James Bond, Doctor Who) | | Timothy Dalton | (James Bond, Doctor Who) | | Stephen Hubay | (James Bond, Doctor Who) | | Francis De Wolff | (James Bond, Doctor Who) | | Daisy Haggard | (Harry Potter, Doctor Who) | | Bill Weston | (James Bond, Doctor Who) | | Christopher Whittingham | (Harry Potter, Doctor Who) | | Roger Lloyd Pack | (Harry Potter, Doctor Who) | | Jason Watkins | (James Bond, Doctor Who) | | Toby Jones | (Harry Potter, Doctor Who) | | Robert Jezek | (James Bond, Doctor Who) | | David Tennant | (Harry Potter, Doctor Who) | | Elize du Toit | (James Bond, Doctor Who) | | Aileen Lewis | (James Bond, Doctor Who) | | Dennis Matsuki | (James Bond, Doctor Who) | | Anthony Ainley | (James Bond, Doctor Who) | | Bhasker Patel | (James Bond, Doctor Who) | | Jim McManus | (Harry Potter, Doctor Who) | | Tom Chadbon | (James Bond, Doctor Who) | | Gbor Baraker | (James Bond, Doctor Who) | | Shane Rimmer | (James Bond, Doctor Who) | | Geoffrey Cheshire | (James Bond, Doctor Who) | | Albert Moses | (James Bond, Doctor Who) | | Huw Edwards | (James Bond, Doctor Who) | | Steven Berkoff | (James Bond, Doctor Who) | | Clifford Earl | (James Bond, Doctor Who) | | Jan Williams | (James Bond, Doctor Who) | | Edward Underdown | (James Bond, Doctor Who) | | Nichola McAuliffe | (James Bond, Doctor Who) | | Clive Cazes | (James Bond, Doctor Who) | | Marc Boyle | (James Bond, Doctor Who) | | Kevin McNally | (James Bond, Doctor Who) | | Nina Young | (James Bond, Harry Potter) | | Peter Brace | (James Bond, Doctor Who) | | Philip Locke | (James Bond, Doctor Who) | | Colin Stinton | (James Bond, Doctor Who) | | Dinny Powell | (James Bond, Doctor Who) | | Leslie French | (James Bond, Doctor Who) | | Carl Rigg | (James Bond, Doctor Who) | | Nick Hobbs | (James Bond, Doctor Who) | | Ronald Rich | (James Bond, Doctor Who) | | Alan Bond | (James Bond, Doctor Who) | | George Pastell | (James Bond, Doctor Who) | | Paul Darrow | (James Bond, Doctor Who) | | Michael Osborne | (James Bond, Doctor Who) | | Brian Grellis | (James Bond, Doctor Who) | | Ralph Morse | (James Bond, Doctor Who) | | Steve Plytas | (James Bond, Doctor Who) | | Alf Joint | (James Bond, Doctor Who) | | Bernard Horsfall | (James Bond, Doctor Who) | | Lasco Atkins | (James Bond, Harry Potter) | | Tobias Menzies | (James Bond, Doctor Who) | | Ken Norris | (James Bond, Doctor Who) | | Shaun Lucas | (James Bond, Doctor Who) | | John Abineri | (James Bond, Doctor Who) | | Paul Marc Davis | (Harry Potter, Doctor Who) | | Julian Seager | (James Bond, Doctor Who) | | Laurie Goode | (James Bond, Doctor Who) | | Tim Condren | (James Bond, Doctor Who) | | Sonny Caldinez | (James Bond, Doctor Who) | | Jennifer Hill | (James Bond, Doctor Who) | | Gertan Klauber | (James Bond, Doctor Who) | | Terence Brook | (James Bond, Doctor Who) | | Vincent Wong | (James Bond, Doctor Who) | | Richard Cubison | (James Bond, Harry Potter) | | Dudley Jones | (James Bond, Doctor Who) | | Ray Marioni | (James Bond, Doctor Who) | | Andr Maranne | (James Bond, Doctor Who) | | David Hankinson | (Harry Potter, Doctor Who) | | David Ashton | (James Bond, Doctor Who) | | Jo Osmond | (Harry Potter, Doctor Who) | | Philip Rham | (Harry Potter, Doctor Who) | | Sean Cronin | (James Bond, Harry Potter) | | Adrian Rawlins | (Harry Potter, Doctor Who) | | Terence Bayler | (Harry Potter, Doctor Who) |

Dependencies

In my current employment my job is to manage the flow of large amounts of data for quite a well-known university. It sounds quite dull but it's actually really exciting because I don't just get to maintain data I get to write cool stuff that uses the data too. For example, every five minutes I get fed a list of all the PCs in the university and whether there's anyone using them, so while I was going through the process of making sure this data was being stored and managed correctly, I took a few hours to write a little web app for students which draws the uni's workstations on a map and tells them where their nearest available one is. I've had loads of positive feedback and the site gets hundreds of hits per day, which is why my job's so rewarding.

My predecessor and mentor had similar experiences - he used live information fed to the university from the council's transport department to produce a website which gives up-to-the-minute bus information, and because we re-publish the data in a sensible format, anyone who can program can write a website or smartphone app that does the same or similar things. Just recently the council changed their data provider and I've spent large amounts of the last month (and probably the coming month too) hacking the code on our server to ensure that no external apps break during this transition phase. OK, some downtime is unavoidable, but in a month's time if there are apps written six months ago that no longer work, I've done something wrong. This is really important to me, as I feel that by publishing this data and having people rely on it, we have a duty to those who trust our data. I really don't want to have to track down everyone who's using our data and tell them they need to re-write their code because I've changed the format, and I certainly don't expect any developers who've written apps to keep checking our site to make sure we haven't just changed the format without telling anyone.

But this is why it pisses me off when others don't do the same. I've had so many troubles writing Facebook apps that I just don't bother any more - if I need data from Facebook I screen-scrape it. You think the front-facing parts of Facebook change all the time, well you should try using the API. I've written so many things that worked for months and then simply broke without warning because Facebook decided to change something. The most recent example is their sudden removal of user RSS feeds, which I've been using for years, as I tend to use Google Reader rather than logging into Facebook. I write quite a lot of Twitter-related scripts too, and I noticed while checking something in their docs today that they indent to deprecate version 1 of their API "within the coming months" in favour of version 1.1. It's not a simple transition either, things that before required no authentication now require OAuth, which is a real pig for a programmer because things I used to be able to do in one line of code (eg getting the public tweets of a particular user) now requires me to implement an entire authentication pipeline, which will probably take me hours.

The biggest bane of my life in recent years was in my previous job as a computer science researcher. I worked on a pervasive device for sufferers of memory loss, and a part of its functionality - recognising the faces of friends and colleagues - was provided by an external service known as Face.com. We used the service for about a year before the company was bought by Facebook, and as soon as this takeover happened they shut down their API, effectively making our system useless overnight. All the training data we'd provided to Face.com was lost, and even if we had found another service, we would have had to start again from scratch. Cheers, guys.

Google seem to be doing the right thing - I've written many Google Maps applications, the first of which was the places part of this very website. The Google Maps API is now up to version 3, which I use when writing anything new, but the places page still uses version 1, and it still works. I've not had to re-write anything. Although v1 of the API is officially deprecated and unsupported, Google have kept it live so that apps written using it don't break. It amazes me that organisations such as Facebook have so much contempt for developers that they can't keep old APIs active, or at least consistent, despite being worth billions of US dollars. If you can't support an API, you really shouldn't provide one.

I doubt anything relies on the data made available by Madhouse Beyond, but I promise you now I take data dependency very seriously. The URL structure of Madhouse Beyond changed considerably in the last year, but I made absolutely sure that with a few exceptions (the text speak translator has gone because it's not funny any more) all the old URLs redirect properly. I did this myself, because I care, and it's sad that I seem to be one of a very small number of people who do. I just hope that as the world wakes up to the possibilities of linked open data, app developers gravitate towards data sources that actually bother to keep their formats consistent. This will force less competant providers to improve their practices or simply fade away.

Draw This

"You should play Draw Something, it's really good" said a good friend of mine the other day. I called it up on Android Market and checked the different versions available.

Firstly, I'm all for paying. I hate adverts. If I get the choice of a free version with adverts or a no-ad version for under a quid, it's a no-brainer for me. So I compared the two versions of the app, the premium and the free, and noticed something interesting... they both require access to the GPS hardware.

| | | | --- | --- |

Now, I'm aware that sometimes the free version of an app will require access to the user's location mainly to decide which adverts to show (pointless showing an ad for a service only available in the US if the user is in the UK for example) It's only fair - you still pay for the app, just not in financial terms; you pay either with your time (reading ads) or your privacy (app collects lots of info on you) but in this case the game wants to know the location of the user even if they opt to pay, which isn't right. More worryingly, it asks for the fine location - the actual GPS position (potentially accurate to within a metre) of the user, rather than just the network location. Fair enough, if a developer wants to know I'm in Southampton I've no real issue with that - but I don't want them knowing my exact address. I decided not to install Draw Something.

What worried me the most is that when I mentioned to another good friend of mine why I decided not to install the app, they replied "all apps want to know your location", dismissing it as not worth worrying about. So do I worry too much? I decided to find out.

One of my favourite games on Android is Cut The Rope. I had a look at the permissions required for the different versions. There are three versions - Free, paid and HD. As a tablet user I have the HD edition (which isn't free). It requires the coarse location permission. This allows the game to determine the approximate location of the user without giving it access to the GPS. This is perfectly reasonable in my opinion. After all, any app with access to the internet can gain this information without needing the location permission anyway. As to why it needs it, I've no idea, but as I said, I've no issue with it knowing roughly where I am. As a developer I like to know stuff about my users so I can tailor my future stuff to them, I'd be surprised if others didn't think the same way. A little worrying is the fact that it wants to know the device's phone number, but as my device is a tablet with no phone function I guess this doesn't really apply to me. I'd hesitate before installing the app on my phone though. But look - the free edition requires the fine location permission, which allows the app access to the GPS. This runs true with my earlier assertion that you pay for free apps with your privacy. The app only wants to know exactly where you are if you don't pay up.

| | | | --- | --- |

But let's look at another example - the ubiquitous Angry Birds. As far as I can tell, Android has no paid version, the only version is the free one (not counting the sequels, Rio and Space). Even so, the game doesn't want to know exactly where you are, just roughly. And as it's a free app, with ads all over the place, I think this is fair enough. In fact, Rovio are so intent on people not getting the wrong end of the stick that they went public [talkandroid.com] with the reasons behind all the permissions required to run Angry Birds (or, to be more precise, one of its sequels) on an Android device. They specifically mention that they didn't feel the need to request a precise GPS-based location for the user in order to target their adverts. So when one of the market leaders says things like that, OMGPOP, developers of Draw Something, had better follow suit or at least give a damn good reason for not doing so. Sadly, there doesn't seem to be any kind of privacy information for the app on their website. Their privacy policy only refers to the website itself, and the one page on the site relating to Draw Something has nothing other than a promotional video and links to the various app stores.

In this age of identity theft and privacy infringements I think we all need to be a little more cynical. When someone asks for information, there's always a reason. If they refuse to divulge that reason, a smart person should really be wondering what they have to hide. But for developers, it really does make good commercial sense to keep users informed - OMGPOP have certainly lost one potential customer and I'm certainly not the most privacy-concious person I know. Perhaps it's a Google problem. I applaud them for making app permissions so transparent, but the Android Market should maybe force developers to provide a reason for each permission they request. And let's face it, the only apps that should need to know your exact position on the earth are navigation or local interest applications - if we sleepwalk into a world in which we unquestioningly expect apps to want to know where we are, then we've taken a massive wrong turning.

Fighting Fire with Water

There's a lot of controversy in the press at the minute (rightly so) about the fact that a bill is about to be passed through parliament. All the main party leaders (plus Nick Clegg too) have come out in support of it despite it not even being read in parliament yet. It's being called "emergency" legislation, meaning it'll be passed through much quicker than any other law. In fact, had Tom Watson (the only MP for whom I have any respect whatsoever) not brought it to the attention of the press, we probably wouldn't have even heard about it until after it became law. The big-shot politicians are all saying this is necessary to prevent terrorism (textbook excuse #1) and paedophiles (textbook excuse #2) but a lot of people are beginning to think that, in the wake of the actions of Edward Snowdon and Chelsea Manning, maybe a government isn't really fit to wield this kind of power. Whether you agree with that or not (I'm torn, I must admit) it's not exactly a healthy sign of government when bills get rushed through parliament lickety split before anyone can read them, even when you consider this bill will probably have no effect whatsoever on the powers available to, say, GCHQ.

So you'd think that I'd be outraged at this bill. Well, I'm not. And the reason is simple: technology is a much more effective weapon against law than law is.

Law is slow. I mean, really slow. It's also dumb. I kinda lost my last shred of respect for the law during the infamous Twitter Joke trial when they started arguing over grammatical technicalities, when anyone with more than two brain cells could see the guy had no intention of blowing up an airport. The fact is that law doesn't understand technology, so usually when laws are made to restrict technology there is an awful lot of collateral damage. This is partially because law makers like to cover all bases, but mostly because politicians, judges and lawyers don't know enough about the technology they're trying to legislate. Basically, law is really bad at solving problems, particularly modern ones.

Compare this to technology, which is very good at solving problems. The problem of your ISP being required by law to store logs on your browsing habits, solved by Tor and VPNs. The problem of not being able to smoke legally indoors, solved by e-cigarettes. Heck, when vehicle clamping became illegal suddenly every private car park in Southampton had ANPR cameras installed. Look at any kind of obstacle, good or bad, legal, technical or otherwise, and there's more than likely a form of technology that can circumvent it.

I don't fear a surveillance state because I know that for every wall there is a higher ladder. And as I'm not a lawyer, or anyone with any kind of political influence whatsoever, my ladder is technology. When (not if) the new bill is passed, I will simply continue using Tor to encrypt my traffic. I will continue to use VPNs to mask my IP address, and to fool my ISP's traffic shaping procedures. And the next time a law is passed that I'm not happy about, I'll come up with a technical solution to that too, rather than waste my time lobbying politicians who don't listen.

Going Backwards

Those who weren't familiar with the Blackberry Messenger service (BBM) before the recent civil unrest are probably familiar with it now. It's basically a text message service exclusively for Blackberry phones, but it's free to use. Which, I guess, is a plan that's working for RIM, the company behind the Blackberry brand, because I'm seeing people swapping BBM numbers on Facebook just as much, if not more than their mobile numbers these days.

A less successful 'brand exclusive' communication medium is Apple's Facetime. It's a method of making video calls between iPhone 4 devices. It's not quite as popular as BBM, firstly because of the restrictions (you need an iPhone 4, the other person needs and iPhone 4, and you both need to be in range of a wifi access point because it refuses to work over 3G) but also because people don't like video calls. Proof of this is in the fact that every halfways decent phone for the last five years (except the iPhone) has had video calling functionality, and people simply don't use it - even though the functionality of the universal service works between different makes of handset and also over cellular.

But Facetime and BBM both do something which I consider quite dangerous... they replace a universally accessible service with an alternative that's restricted to one make of handset. Facetime replaces video calling with a system only available to iPhone users, and BBM replaces universal text messages with a system only available to Blackberry users. It doesn't stop there... Facebook have announced an application for mobile messaging, and there are rumours flying around that Google are about to release their own closed messaging system to tie in with Google Plus.

Now, make no mistake, I strongly dislike text messages. People's attitudes towards them are wrong - if you send a message via SMS and get no reply it's far more likely that the message hasn't got to its destination yet, but many assume it's the recipient being rude. Also the cost is outrageous. It's around 10p for a single message, 140 bytes, depending on your network. That works out at £714.29 per megabyte, and to the phone company that's almost pure profit. I get 500MB per month for a tenner on my current data plan, and the phone network are making a profit out of that. The same amount of data would cost me over £357,000 to send via SMS. If you have a contract with 500 text messages, that's only actually 70K of data, so compared to your data allowance the text messages should be pretty much free. Personally I'd love to see a world in which everyone drops text messaging and starts using mobile email instead - it's cheaper by far, even if you're on a flat rate contract, it's easier to filter for spam and unwanted communications, and people know not to expect an immediate reply.

But all that said, I'd rather have text messages as they are today than go back to the bad old days. Remember when text messaging was a new thing? You could only send messages to people on the same network as you, so you had to make sure you bought a phone on the network most of your friends were on, even if that network wasn't exactly what you wanted... it was a pain in the arse. By segregating messaging systems by OS, by handset manufacturer or by social network, we're basically going back to the way things were in the bad old days of mobile messaging. And that's not a good thing.

Google are evil, but everyone else is OK

So here it begins... the Wall Street Journal report that Google are bypassing security settings on certain versions of Safari, specifically the iPhone version. Cue the shitstorm as hundreds of "privacy advocates" start bleating about how Google are 'evil'. Well I'm not going to make excuses, nor am I going to claim two wrongs make a right, but there are a few points that need to be addressed and nobody seems to be doing so.

Firstly, an analysis of what Google are actually doing. In order to make their 'Google Plus' code work, they need to be able to drop what's known as third party cookies on peoples' web browsers. You don't need to know what these are or how they work, but the default security model on lots of browsers these days is to disallow this, as it's a common method that advertising sites use to track you round the web. Maybe Google are doing this, maybe they aren't. Truth be told, they probably are, seeing as how advertising is how they make all their money. But the fact is that Google used this exploit to drop cookies on versions of Safari for which they had been disabled. You'll notice that the exploit was is over a year old, and since then it's become common in Facebook applications, which also rely on passing cookies between IFRAME elements.

So my first point: are Google really doing anything wrong? It's not hacking, it's computer science. They hit a problem, they solve it. The problem in this case is that they can't drop cookies on some browsers. They learn that it's possible to do so using a clever form hack as described in the previous link, and implement it. Problem sorted, they can now drop the cookie they needed, let's move on to the next problem without even batting an eyelid. By the same logic, Google Maps is 'evil' as it uses clever hacks to generate dynamic scrolling maps in an otherwise static web page.

My second point: even if the practice is slightly shady, why is everyone having a go at Google when the exploit has clearly been working on Facebook for over a year? If it really is such a problem, why have Apple not patched the hole? They've had a year to do it. Even if you do consider this frankly quite clever workaround to a programming problem to be wrong, let's bash Facebook as much as Google, and certainly let's bash Apple for not patching a one-year-old vulnarability in their web browser. It's certainly a genuine shame to see Google getting so much stick rather when openly privacy-apathetic organisations like Facebook and companies with a piss-poor reputation for fixing security vulnerabilities like Apple seem to be able to get away with anything these days.

In which I do give a damn about inaccurate metadata

I'm going to do a tech rant now, so you can turn off if you're sick of them. Apple fans, however, may be pleased to know that for once the rant isn't about something they own. It is in fact about the HTC Desire Z.

Firstly, it's not a bad phone overall. It's certainly not the most technically advanced phone in the world, but it has a keyboard which is a must for my fat fingers. It does everything an Android 2.3 device should do. One of the things it does is take photos. Sadly the camera app on it is pathetic.

Again, I must clarify: I appreciate that all phone cameras are shit. The lenses are tiny and despite manufacturer claims of massive numbers of megapixels and "high definition" cameras, there isn't a phone camera in the world that is good enough for anything more than taking photos for sharing on Twitter. This I can forgive. What I have a problem with is the actual camera software, specifically the metadata it stores with each photo.

Every digital camera since the dawn of time has a realtime clock. When you take a photo, the time and date is stored with it. This allows photo management software like Picasa and iPhoto to organise photos in chronological order. Over time, geo-tagging has become common, mainly because of smartphones. The idea here is that the geo-coordinates of the image are also stored in the digital file, so you can now sort by location as well. The Desire Z does both of these things badly.

The timestamp stored is always the local time, with no timezone information. This means that if you take a photo in Paris, then fly to London and take a photo there within an hour, the London photo will appear to have been taken first, and there is nothing you can do about it other than manually edit the time in the image. This is also a problem during summer time when DST is in place, if you have lots of photos taken at the same time each day, you need to adjust the time for summer. The correct thing to do would have been to store the timezone information along with the date, or even better, simply store the UTC timestamp which is the same worldwide, like my standalone camera does.

It also buggers up the geo-information. When you open the camera app it begins searching for GPS satellites. If you take a photo before it gets a fix, rather than not adding a location, it will add the last location it found, even if it was hours ago. Frequently I take photos miles away from my home and they're tagged as being at my house because that was the last place I used the GPS. No problem, I hear you say, simply turn geo-tagging off, right? Well, no, because if you turn geotagging off it will still store a location, but it will store latitude and longitude co-ordinates 0,0, which, as any geography expert will tell you, is in the Atlantic just off the west coast of central Africa.

I've no idea if it's just the Desire Z, all HTC phones, or indeed all Android 2.3 phones that have these two problems, but for christ sake someone sort it out. If I were to buy a camera phone and take a photo of my hamster and get a photo of a goat I'd consider the phone to be faulty - EXIF data is no different.

It's all about the pixels

At some point over the last few years it became the 'de-facto' standard for 'high definition' to refer to a picture size of 1920 by 1080 pixels. Allow me to rant on why I think this is dumb and wrong.

Firstly, 'high'. When DVDs came out, they were capable of producing (in PAL territories anyway) a progressive picture of 576 lines high. This is now known as 'standard definition', despite the fact that before DVDs pretty much all video was interlaced, meaning there were only really 288 lines of visible video at any one time. High def is obviously higher, but it is just that: higher. Not high. High def only came in because TVs are getting bigger. Compare 'high' definition to the definition of, say, a cinema camera and it looks very low indeed. So in 20 years time when everyone has an 80-inch screen in their front room, 'high definition' will start to look really pixelly, and you'll probably find that 'extra high definition' and 'super extra amazingly high definition' will need to supercede high def. They really should have called it 'digital video generation 2' or something like that, so they can go for 3, 4, 5, etc next time.

But secondly, and more importantly, people say 'definition' when they actually mean 'resolution'. By only taking into account the image resolution (the frame size in pixels) when defining 'high definition' you end up with some pretty shocking pictures that are, in my opinion, wrongly classed as high definition. Heck, the word 'definition' actually means clarity, so why is it that a blocky, low-bitrate video stream can be classed as high definition just because it's 1080 pixel lines high when a crystal-clear, higher bitrate displayed at 576 lines is considered standard definition, even if it has a clearer picture? The answer is simple: 'high definition' is nothing more than a marketing term. It has about as much meaning as the 'V' in 'DVD'.

A brief analogy: go into any decent camera shop and the salespeople will (correctly) tell you that megapixels are pointless, it's the lens that's important. More and more cameras, and even phone cameras, are being sold with 8, 10, even 12 megapixel definition... but if you don't have a decent lens and CMOS sensor then it's only producing 12 megapixels of rubbish. Video is exactly the same... you can have a high def camcorder, but if it's storing hours of video on a poxy 2GB SD card then you may as well be recording in standard def and upscale it later, it will look just as bad. I'm labelling home-video types with HD camcorders here, but professionals aren't flawless; try watching some low popularity digital TV channel (ie Sky 3, ITV4, etc) on a full-HD setup and you'll see how bad the picture is. There is no trickery or half-truth going on, the picture is indeed 'high definition', at least by its universally recognised definition, but it's a low bit rate and this is why it looks crap and blocky.

I think it's time we stopped thinking purely in terms of pixel resolution and more in terms of bit rate. We also need to redefine the phrase 'high definition' to better reflect the reality of digital video... it's not just about the picture resolution. As for me, I'm going to start saying 'high-res' rather than 'high-def', it's more technically accurate. You're welcome to join me if you like.

Silver Linings

Some of you will notice this site has changed in the last week. Now my tweets, photos and new NP doodles appear alongside the increasingly rare blog posts, the locations bit has been vastly improved and there's lots more music stuff, including an encyclopaedia of my music collection, and a list of upcoming gigs in Hampshire's pubs. Oh, and the poll is back and logins work again, so you can stop moaning at me now.

Most of the cool stuff relies on external content. The music database is all based around open linked data and the textual content comes mostly from Wikipedia. The gigs are screen-scraped from various bands' websites and Facebook, and the tweets on the main page obviously come from Twitter.

So yesterday, as we all know, Twitter went down for a few hours. We all know because the BBC started banging on about it, completely giving away where they get their news from these days :) I kinda passive-tweet, in that I tend to use apps rather than the website and only really check Twitter when I can be bothered. I certainly wouldn't notice if there had been no new tweets for a few hours. I did, however, notice that all the tweets had vanished from the front page of this website. It became apparent to me that my website now has many points of failure rather than just the server on which it's hosted, so I began to write a hotfix.

Now I believe you'll find you can always find my tweets here, even if Twitter is down. Every time the page is reloaded it pulls my new tweets from Twitter and stores them in a local database. If it can't get to Twitter, it simply reads the most recent local copy and seamlessly generates the page from that instead. This should be the case with Last.FM, MusicBrainz and Wikipedia, as well as the BBC's open data, all of which are used by this site to populate its content.

Someone find me a memory tube

Previously...

Now, of course, it's all happening again [wired.com]

The Echo needs to work on its applied psychology

Ooh yes please! Sign me up! I'll turn Javascript on, I always wanted to see more adverts on the web! And ad servers are renowned for their honesty and good practices too, so I've no problem whatsoever in allowing them to run client side code on my machine!

Fucking idiots.

The Information Monopoly

Just recently there have been two new products announced that make it easier to communicate, and keep track of messages by aggregating all your messages from different sources into one place.

The first of these products is the long-awaited Windows Phone (aka Windows Mobile 7) which is a complete re-write of Microsoft's mobile platform to make it less computer and more phone. They seem to have taken a leaf out of Apple's book with a lot of the design choices (ie no cut and paste or multitasking in the initial version) and there are lots of 'silly' things, like an in-built Zune and Xbox 360 integration. However, the main feature that everyone in the tech world is raving about is the fact that it makes messaging so easy, in fact, its simplicity is the main focus of the current TV advert. It does this by turning messaging on its head. You have all these communication methods - SMS, email, Facebook, MSN, etc - on your phone anyway, so why not combine them into an easy 'people' hub where you can sort all your messages by person and subject, rather than have to keep them seperated by delivery method, or in different apps. It also, being written by Microsoft, will probably crash quite a bit.

The other of these products is Facebook Messages. It performs roughly the same task, keeping your email, texts and Facebook messages together. Suddenly all the messages you get sent via Facebook can be read at the same time as your email, you don't have to check them both. And when you send text messages on the go, and then later continue the conversation online via email or chat, you can see the previous text messages there too and refer back to them without having to switch to your phone. It also, being a feature of Facebook, will probably crash quite a bit.

These two services have a crucial difference. Facebook Messages works server-side and Windows Phone works client-side. I will explain. If you have a smartphone, you probably already manage your text messages, email, etc all on your phone anyway, albeit in different apps for each communication type. The only additional functionality that Windows Phone gives you is the ability to view them all in one place. They're still all delivered to and stored on your phone the same way, and as far as anyone but you is concerned, nothing has changed. You don't have to change providers, your email provider doesn't have access to your Facebook posts, Facebook doesn't have access to your MSN, Microsoft don't have access to your text messages, etc etc. The only place where everything comes together is right in your hand, where it belongs. Facebook's alternative is different in that everything now happens through Facebook. You just have one connection - to Facebook - and all your email, texts, chat, etc all have to go through them before they get to you. Of course, to be fair, Windows Phone means buying a new phone, so it's the more expensive option as Facebook's service doesn't cost you a penny. But you do still have to pay for it, in the traditional Facebook currency that is your privacy - and by switching everything over to Facebook you're effectively giving them a monopoly on your personal data.

It's no secret that Facebook is a security nightmare. The fact that it's so easy to view strangers' data makes it an unwise decision to upload anything even remotely private, and many computer security experts suggest that you really shouldn't upload anything to Facebook that you wouldn't put on the public internet... that is, of course, excluding the ones that believe nobody should be using Facebook in the first place. There are many stories of people who have been far too naive on social networks, I personally know of at least three people who have been in trouble with their boss over things posted on Facebook, and there are actually people who have been fired and even killed over things they've put on there. Even if we forgive Facebook's murky reputation, there's always a risk letting one company have control of so much information, as recent news reports about HMRC and ACS:Law show. As I've blogged before, You can never be entirely sure who has access to this information when it's in someone else's care. So why would anyone want to turn their entire communications network over to Facebook, or any other company for that matter?

The answer is, of course, in the question: naivety. People are naive enough to talk about their personal life after 'friending' work colleagues, people are naive enough to post rants about their boss and co-workers online and people are naive enough to talk about everything they've ever done despite the fact that their employer, boyfriend, girlfriend and even the police have access to it. People are naive enough to put their email password into Facebook's 'Friend Finder' and then get surprised when the service starts advertising their presence to their psychotic ex, or some guy they emailed once to exchange insurance details after a prang. People are naive enough to click 'Yes' when an application that claims to be just a silly quiz or gift app asks for permission to access their account even when they're not online. So, when offered a simple method of keeping all their communications together without paying a penny, of course people are going to be naive enough to sign up.

But in reality, nothing is really free and anything that seems too good to be true usually is. People say I'm cynical, negative and paranoid, but can say with 100% certainty that my boss will never see a photo of me drunk.

The Kinect really is awesome

...and not just as a game platform. This article (hat tip to Nik for pointing me to it) describes viSparsh, a system for assisting blind people which is made from a modified Kinect. The Kinect's depth awareness allows the device to determine how far the wearer is from an object and feeds back a series of vibrations, a bit like a car's reverse sensor. Over time the wearer learns to judge distance using the vibrations, allowing them to walk around much more confidently.

The use of a Kinect for good reminds me of the robot built by researchers from the University of Warwick, which uses a Kinect's 3D imaging capabilities to locate trapped survivors in the aftermath of an earthquake.

There are two things to learn from these stories. Firstly, the Kinect is wasted on video games. It's genuinely groundbreaking technology. Secondly, both these stories are examples of the good that can be done when a tech company opens up their hardware to homebrew developers and hobby hackers. Microsoft have very publically announced [eff.org] that they encourage people to use the Kinect in whichever way they see fit, a very different attitude from Sony, Nintendo and even one-time proponents of freedom Apple, who all frown upon the use of their kit for anything other than its intended purpose, and go to great lengths to ensure it doesn't happen.

The Unknown Known

People often ask me why I trust Google more than Facebook. After all, both provide services in return for personal information, both are big US companies based on a clever piece of technology, both were started by university students, and both are worth an awful lot of money. Both have privacy issues, most have been identified and many have been fixed. Both are opt-in, you don't have to use them. The reason I trust one more than the other is simple: the unknown known.

Here's a good example. Check out Google's privacy policy. It states happily that when data is 'deleted' from their services, the data may be retained by Google even if not publically available. Facebook contains no such line, so it's implied that deleting something actually deletes it from Facebook's servers. Yet I had a Facebook profile that I deleted about two years ago, and the photos I uploaded to the account before I deleted it were still accessible to anyone with the JPG URL some eighteen months later. In fact, the only reason I can't access them now is because Facebook changed their URL structure a few months back and all old URLs became invalid; I don't for a minute believe that those images aren't still on Facebook's servers. There are good reasons why the images don't disappear immediately - residual data and backups being the main two. But the fact is that Google announces this up front, and Facebook doesn't. And this is the crux of why I don't trust Facebook as far as I can throw it.

Another issue of contention with Facebook is the Friend Finder feature. You enter your Hotmail or GMail username and password and Facebook logs into your account and hoovers up all the email addresses it can find. It specifically states that it doesn't store your password, but it doesn't mention keeping a login session active and it certainly doesn't say what it does with the emails and contacts that it finds. Someone I know, who has never had a Facebook account, recently had an invite email sent via Facebook saying "[x] wants to be your friend". Contained within the email were suggestions for about a dozen other people she knows, some of whom were family members who had no contact whatsoever with the person who sent the invite. The only way this could have happened is that the family members also used the Friend Finder, and Facebook stored all the connections for future use. Basically, Facebook has a sort of dark network underneath its world-facing one to which you have no access and can't opt out of, Facebook account or not. If you have an account you can delete all your Friend Finder history, but this doesn't really help you if you choose not to have a Facebook account, or if someone who has your email address has previously used the Friend Finder.

Back to photos, you may already know that when a digital camera takes a photo it stores lots of information about the camera as hidden data within the JPG file. The time, the date, the camera settings, make and model. Smartphones with GPS often geo-tag images, meaning that the location in which the photo was taken gets stored as well. When you upload images to Facebook, it processes them to optimise them for web use, and this includes removing meta-data - download a photo from Facebook and load it into an EXIF viewer and you'll see it has no meta-data whatsoever. However, recently Facebook have started trying to encourage people to 'check in' to places they've visited and occasionally you'll get one of your photos shown to you with the message "this photo looks like it was taken in [y]". It gets this information from the geo-tag, which it's been storing, inaccessible to you or other Facebook users, since the photo was first uploaded. It's not that Facebook are trying to do something clever with the geo-tag information, it's the fact that they're clearly storing meta-data and not telling anyone that I have a problem with.

There is a movie called The Social Network, which tells the story of the creation of Facebook. The opening scenes show founder Mark Zuckerberg building a collection of photos of every Harvard student without their knowledge or consent, and hosting it on a public server for everyone to see. Zuckerberg's complete contempt for anyone's privacy is illustrated further in an infamous leaked IM conversation between Zuckerberg and an anonymous friend. He offers his friend personal info from Facebook's database. When asked how he got the data, he simply replied "They 'trust' me. Dumb fucks." Zuckerberg is still running Facebook, and probably has complete access to all sorts of information about you, whether you use his website or not. At least the information Google collects is used in their products and services to their users, and not just hoarded away where only the site admins can see it. Google even has a dashboard feature where you can see exactly what information they have on you and with whom they're sharing it, which gives you the opportunity to delete information if you don't want it shared. Facebook has no such feature.

So, to summarise: Google take your information, are completely transparent about what they're collecting and how, and give you something useful back in return. Facebook take your information, often without your knowledge or consent, fuse it with information they've conned out of your friends and family, and then hide it away, sometimes even denying they have it. It's not really surprising that I trust Google more.

Translucent Privacy

Just recently there's been a lot of hoo-hah about information privacy. I think it started with the almighty cock-up at ACS:Law - effectively they were hacked and all their company email was leaked onto the internet. The emails contained lists of people accused of online piracy (that's accused, not convicted). Now there's legal challenges left right and centre from ISPs against legal firms trying to get customer information so they know who to sue. The farcical Digital Economies Act obliges the ISPs to simply hand over all information requested, but then that's what you get for passing a piece of legislation that most MPs clearly didn't bother to read.

DEA rant aside, I've received a lot of privacy-related stuff from lots of companies recently. My ISP has contacted me with a copy of their privacy policy, and my credit card company has done the same. Any company of which I am a customer I have actually looked for their privacy policy. And although they give lots of information - my credit card company for example gives two A4 pages of text about what information they collect and store, how they collect it, how it's used, etc - none of the privacy policies I've seen so far actually answer the two most important questions I have about my personal data: how it's stored, and who has access to it.

I got a sales call from my phone company the other day, they asked me how much on average I spend a month. I simply answered "if you really are from my phone company then you can tell me", to which I was told that the sales team only have access to names and phone numbers, not to actual customer records. Which actually annoyed me. Firstly, if they had access to this information they could tell I spend very little on my phone bill and am therefore highly unlikely to want to upgrade to a more expensive service. But secondly and more importantly, this implies that the information the phone company actually owns, ie my phone records, is treated with a higher level of security than my personal details. I'd love to know exactly what parts of my personal information are accessible to which parts of the company, but the privacy policy makes no attempt to tell me, and if I perform a freedom of information request I'll only get the data they hold on me, not who has access to it.

Additionally, the ACS:Law cock-up happened because ACS:Law don't encrypt their internal email. Not only that, but BT have since admitted that they've sent customer details across the internet in plain text email without using any encryption or security whatsoever. This genuinely concerns me, and I'll certainly think twice about using BT for anything in the future. But what worries me the most is that BT and ACS:Law clearly both have a very lapse attitude to information security - so how many other companies have the same attitude? You can't tell simply with a freedom of information request or a privacy policy if a company has a competant information security policy... or indeed any security policy at all.

My solution? Simple: along with each privacy policy should be an information security policy. In the same way that a company is not allowed to store personal information unless they provide a privacy policy outlining what information is collected and how it is used, I believe that no company should be allowed to use, collect or store personal information unless they also provide an up-to-date document describing their information storage systems, what level of security is being used, and who has access to it. Perhaps there should even be a requirement that anyone storing personal info should be required to make their security systems available for independent inspection, although I see how certain industries (ie defence) might have a legitimate problem with that.

I think the biggest problem with storing personal information in the digital age has nothing to do with evil intent, it's ignorance that's the biggest problem.

Trolls

This has been annoying me for some time but it's about time I said something about it.

There's been an increase in the traditional media just recently of stories about 'trolls'. Trolls, as anyone who's been on the internet for more than 20 minutes will tell you, are people who engage in the act of trolling; posting comments on online bulletin boards and similar services with the intention of provoking an outraged response. Call it a form of online baiting if you will. Skilled trolls will post seemingly genuine and innocent comments on posts on typically emotionally charged subjects such as religion or politics, and see who bites. The troll never directly instigates any hostility, merely encourages others to do so. I've done it many times, it's actually quite good fun if you like winding up easily aggitated people with not enough things to worry about... Mac users, for example ; ) Trolls normally target entire communities rather than individuals - a good example would be the 4chan users who turned up to launch parties for the final Harry Potter book armed with leaked copies of the book and then proceeded to spoil the ending of the book to everyone in the queue. It's a matter of opinion as to whether this is funny or not, but it doesn't actually hurt anyone, and certainly doesn't target an individual or a small group of people.

Compare this with stories in the press and you'll see no similarity whatsoever. Examples the BBC give of trolls are the guy who sent abusive emails to Louise Mensch, and a guy posting abusive messages on the Facebook page of a dead girl. Neither of which are trolls by the correct definition of the word, they're simply online bullies.

Let's get this straight before the word 'troll' becomes as misunderstood as the word 'hacker' currently is - trolls are harmless. They're just out to have wind people up and have a good laugh at the reaction. They merely post or do things likely to provoke a strong response. Bullies are quite different - their aim is to abuse, hurt and emotionally scar people. They say very hurtful things, often aimed at vulnerable individuals. These people are not trolls. It's unfair to dismiss online bullies with such a tame word as 'troll' and it's certainly unfair to most trolls to tar them with the same brush as these hateful, spiteful bullies. Please, BBC, Guardian, and many other news sources I otherwise respect, please stop using words you clearly don't understand. Confusing bullies with trolls is like confusing Wolfgang Priklopil with Jeremy Beadle.

Tweeting Television

I was going to do a post on tweeting TV a while back, after reading Krishnan Guru-Murthy's blog post on the subject. Now the BBC are on the bandwagon too, I thought I'd put in my thoughts on the matter.

For those who've not experienced a TV show 'augmented' with Twitter, then please do. Even if you don't use Twitter, you can use services like monitter.com to view live feeds of tweets containing a particular term. On twitter we have 'hashtags' to denote subjects, and many TV shows actually display their officially recognised hashtag on the screen at the beginning. For example, Watchdog is #bbcwatchdog and Have I Got News for You is #hignfy. The Apprentice (#bbcapprentice) is particularly entertaining if you are watching the show while at the same time sat on Twitter; some of the comments from various people better than the show itself, and of course Twitter users can get away with saying far more offensive things than can be broadcast on prime time BBC1. I actually had the idea a while back to write a VLC plugin or something that cottons on to what you're watching, determines the appropriate hashtag and displays tweets on-screen alongside the TV show, but that's a "to do" for me and I'm sure someone else will do it quicker and better than I can, if they haven't already done so.

So in all, I think tweeting TV shows is great. But there is a problem - not everyone gives a shit. I follow over 100 people on Twitter, all of whom are interesting and/or funny. But some of them occasionally post streams of drivel about a TV show I'm not watching or in which I have no interest. A top example is the X-Factor (#xfactor). My views on a rigged karaoke contest run for Simon Cowell's personal benefit aside, some TV shows just aren't interesting to everyone. Heck, I'm sure every time I tweet something about Doctor Who I get people thinking of unfollowing me. You can block users, why not hashtags? Twitter could even sell statistics to the TV companies for market research purposes, how cool would that be? You could tell how popular a show is by comparing hashtag uses with hashtag blocks. And at the same time, even the most considerate Twitter user would be more eager to tweet about what they're watching knowing that their followers have the ability to block the hashtag if they don't care about it.

So there we go - my to do list. Implement Twitter client with 'hashtag block' function, and implement some kind of hashtag detector using the BBC's linked data. I may be some time.

Why hacking games consoles is a good thing

I often rant on this blog about how console manufacturers should just allow hackers to do what the hell they want with the hardware they themselves have purchased. Sony and Nintendo are both renowned for producing hardware to keep homebrew programmers out and any time someone does find a way in, the company in question immediately push an 'update' via the online connection that breaks any homebrew code. Microsoft recently became the first company to buck the trend by explicitly encouraging the use of their 'Kinect' hardware for Xbox 360 by hackers and other home hobbyists. Every time I rant about not being able to hack my games consoles a lot of non-geeks ask me why I care and why I don't just use my games consoles the way they were intended, to play games? I never really had a good, non-political answer to that question... until now.

http://www.bbc.co.uk/news/technology-12559231

Step in, a team of mechanical engineers from the University of Warwick. They've built a robot (that looks very much like Johnny Five) whose purpose is to search for earthquake survivors in rubble too hazardous for a live human to enter. Except rather than use the usual expensive laser scanning technology employed by similar robots, their creation's vision is powered by a Kinect, which is available for £100 at your local toy shop. This thing is cheap and can save lives, and is generally ten shades of awesome. And it's only possible because Microsoft don't care if homebrew developers want to write code for their hardware.