thoughts about software and sensible security.

The future won’t be anything like 1984: it could be even worse

T

In George Orwell’s Nineteen Eighy-Four, English Socialism (Ingsoc) has created the mother of all police states and turned proud England into a wretched wasteland. Suppression and retribution are as brutal as the neglect of people’s well-being is callous. Everyone is a slave but for a tiny elite of Inner Party members. The Party exists only to grow more powerful and its sole purpose and expression of this power is to make people suffer. It is personified by Big Brother, whose image stares you in the face at every street corner. Yet he never appears in public and we can’t be certain he even exists. Everywhere you are watched. Every passer-by, every co-worker could be an agent of the Thought Police. Rumour has it they can read the tiniest involuntary twitches in your face that could betray anti-revolutionist thoughts, as if they can look right inside your brain. Fear is constant and daily life an exhausting struggle to toe the line.

John Hurt as Winston Smith and Richard Burton as O’Brien in Michael Radford’s adaptation (1984)

Ever since CCTV cameras started appearing in the High Street and long before Facebook and Google Lens worried parallels have been drawn with Orwell’s dystopia. We shouldn’t do that. It’s a Godwin argument. No regime is or has been as bad as Ingsoc, though North Korea and the Third Reich come close. Orwell did not mean his novel to be a realistic depiction of a possible future. Written in his late forties after a lifetime dedicated to the social democratic cause and shorty after Hitler’s demise, he issued a stern warning in the shape of a pitch-black satire. He knew this extreme form of fascism wasn’t viable: a psychopathic state that kept the populace miserable for its own sadistic enjoyment is bound for suicide. People will be too stressed and depressed to raise children for starters, when everything worth living for is either outlawed, defunct or rationed. Even emperor Nero knew you had to give them bread and circuses.

If you want a perfect surveillance state with a massive army of informants that make sure everybody toes the line it will cost you. There won’t be much money or manpower left to fix potholes and leaking taps. You must also budget liberally for a hierarchy of spies spying on the spies, as any system where a tiny elite polices a country of drones is vulnerable to corruption. Ingsoc’s machinery of oppression was unreasonably thorough and extremely labour-intensive. Heretics were meticulously brainwashed to stamp out every last anti-revolutionist thought. And then after months of needless torture they would still be shot. You see, the Party would not stand for martyrdom at any price. They would make you love Big Brother. Winston’s interrogation and torture was overseen by the veteran Thought Police official O’Brien. To keep the country orthodox he and his comrades must have been very busy indeed, as no time and expenses were spared in Winston’s lengthy rehabilitation. In computer terms, the system didn’t scale. It was too much expensive manual work.

Since the Thought Police couldn’t be everywhere all the time, they operated in a cloak-and-dagger fashion. It was enough to be reminded that they might be watching you at any time, knowing full well they could not possibly watch everybody all the time. It works, because the actual odds don’t matter. Something only has to be sufficiently imaginable in order to feel really urgent. The likelihood of dying in an airline disaster is minuscule – even for frequent flyers. But it doesn’t take much imagination to visualize what a crash looks like, because each one is rare enough to make headlines. This cognitive bias is called the Availability Heuristic. Orwell’s agents of evil managed to appear omnipresent just by staying invisible. Your friends could disappear without a trace, their life history purged from all records and their very existence denied. If that’s not terrifying I don’t know what is.

Surveillance tech no longer has to appear omnipresent. We know it is. We’ve had CCTV with clumsy videotape for decades and soon the old-school spy skills of the fictitious O’Brien will be replaced by clever algorithms that don’t need sleep and aren’t prone to corruption – barring software bugs. What’s more, it has become cheap enough to deploy everywhere, not just around banks, border crossings, power plants and other places where it would make actual sense. You are recorded shopping in the High Street, minding your own business, stored in high definition just in case it could come in handy one day.

There’s another big difference with the analogue days of the 1940s when Orwell was writing: nobody had kept records of what ordinary people were up to before the fascist revolution, i.e. things that could get them in trouble. And even if there were such a file – ‘file’ as in a manila folder -, the state would lack the manpower to abuse it wholesale. Yet make no mistake that right now and in recent years we have all been instrumental in allowing such a file to be collected of us. And there’s no escaping, whether or not you’re on Facebook, whether you buy your shoes online, use Uber or ride a bike. Given enough data and the skill to correlate, there is no such thing as anonymous data. The recent revelation of the privacy breach in the Polar fitness app by Dutch journalists of de Correspondent has proved this. Considering this was all carried out using legal means, imagine what’s possible when all privacy safeguards fall away.

Any society governed by the rule of law will have codified the principle of nulla poena sine lege: no punishment without law. It’s in article 1 of the Dutch penal code, which states that you cannot be prosecuted for something you did unless it was punishable by law at the time you did it. The law cannot be applied retroactively, unless it’s to the advantage of the accused. Imagine there exists a detailed and pretty complete history of your most mundane doings. In fact, don’t imagine it. Just remind yourself it’s already there. Now imagine if you dare that an ill wind of change brings an undemocratic government to power, eager to rewrite the law and unwilling to leave the past alone. They nationalise Big Data and set their algorithms to work on a treasure trove of all your likes, tweets and CCTV appearances. In a split second it will unearth that a certain Jasper Sprengers was a regular blogger on online privacy, visited all manner of sites devoted to the same cause, was a subscriber of the left-wing Guardian and Volkskrant newspapers, all outlawed now. There’s no digital record of his voting history, but given where he lives, what he eats, what music and films he likes, the people he meets, it doesn’t take many processor cycles to work that out.

By then I hope I won’t be considered subversive enough to qualify for O’Brien’s brainwashing and torture program. I’ll probably be too old to do much damage and Ingsoc was too gloomy to have a future anyway. The Brave New World is more likely to resemble the brilliant Black Mirror episode Nosedive. I will just have my social credit curtailed, cursed never to rise over a 1.5. In Europe we don’t seem too worried it will ever come to this, but China’s social credit system spells a future not unlike it.

It’s not about data security that I’m worried most. I’m sure Google and Facebook employ better than average techies to keep their assets safe. It worries me that data which carries a value (to you, commercial parties and law enforcement) will stay put in the Cloud, unchanged and incorruptible, something you can’t confidently say of the governments that govern them. It’s our very democracy we have to defend at all costs to ensure our data is ours and ours alone and used as a force for the good.

Enough with the doom and gloom: I don’t want to leave on a depressing note. So let me warmly recommend all the other superb novels and non-fiction by George Orwell. He is still one of my favourite authors of all time. Nineteen Eighty-Four doesn’t bring out his great sense of humour and satire of bourgeois values the way he did in Keep the Aspidistra Flying and Coming up for Air. His investigative journalism that took him to the coal mining communities of the North is chilling and riveting (The Road to Wigan Pier). His love of humanity and devotion to justice is palpable on every page. He is one of those rare writers you wish you could have met personally, almost seventy years after his death.

Add comment

thoughts about software and sensible security.

Recent Posts

Jasper on twitter

Catching thought criminals in Orwell's analogue dictatorship was time-consuming and ineffective. A.I. will fix all… https://t.co/XpzIDondJe
h J R
I expect Mars to be successfully colonized long before we have flawless PDF to Word conversion.
h J R
Don’t tout #kotlin conciseness as a unique selling point. Concise does not equate understandable and if concise is… https://t.co/Hj7FrbZuI8
h J R
Hilfiger gives ‘smart dress’ a whole new meaning with new tracking chip. https://t.co/OeB4NEbVQI
h J R
ATDD is really different. Think of it as All Tests Drive Development. New blog post. https://t.co/J7uyHmCXFf
h J R