A Review of New Dark Age and Outnumbered

This review was first published in The Weekend Australian.


In his brilliant and beautiful book, New Dark Age, the British artist James Bridle invokes Isaac Asimov’s Three Laws of Robotics. First set out in Asimov’s short story ‘Runaround’ (1942), these laws are usually expressed as follows:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

Like many contemporary commentators, Bridle is under no illusion that these laws are sufficient for the twenty-first century. Writing as Alan Turing and his team were cracking the Enigma codes (and building the first modern computer in the process), Asimov could have had no idea of how quickly, and in what direction, ‘thinking machines’ would evolve. The sci-fi author was thinking principally of androids, not of autonomous vacuum cleaners or social media algorithms, let alone ‘robots’ made from biomolecules that might one day be used to correct gene disorders. Notwithstanding the ethical minefield laid by drones and other military hardware, the incredible advances in machine learning bring with them a different set of dangers.

For Bridle the principal danger of such technologies is that most of us have no idea how they work; we know less and less about more and more. This is the clue to the book’s title, which is taken from another short story – H. P. Lovecraft’s ‘The Call of the Cthulhu’ – and describes, not a lack of information, but an abundance of it so overwhelming that the world has become opaque to its human inhabitants. The ‘counterintuitive premise’ of the age is that enlightenment has brought darkness in its wake. And so Bridle proposes an additional law:

  1. A robot – or any other intelligent machine – must be able to explain itself to humans.

The central argument (or anxiety) of New Dark Age is that we are trapped in a computational fallacy. Not only do we erroneously model our own minds on computers, we also believe that they – the computers – can solve all our problems, if fed enough data. This ‘computational thinking’, or ‘solutionism’, places a blind faith in computers’ problem-solving capacity, and in data as the central component of every problem. The result is that our subjectivities have been transformed. Computation, writes Bridle, is ‘no mere architecture’; it has become ‘the very foundation of our thought’ – even to the degree that it is now impossible to think about the world ‘in terms that are not computable’.

It is a mistake, argues Bridle, to think that computers and artificial intelligence are value-neutral. On the contrary there is a concrete relationship between the complexity of the systems we encounter every day and ‘issues of inequality, violence, populism and fundamentalism’. From ‘dark dealing’ on the world’s stock exchanges to the oppression visited on workers (‘meat algorithms’) in Amazon’s vast warehouses to the inhuman treatment of Uber drivers, computers and computational thinking are marbled into the deep unfairnesses and brutalities of modern life. As Bridle puts it, with characteristic portentousness: ‘To the capitalist ideology of maximum profit has been added the possibilities of technological opacity, with which naked greed can be clothed in the inhuman logic of the machine.’

So convincing is Bridle’s description of this malady that his remedies can seem inadequate. Certainly his suggestion that ‘centaur chess’ – chess played between human-computer teams, as opposed to between humans and computers – figures a new ‘ethics of cooperation’ strikes me as a little hopeful. But his call for a deeper appreciation of the interplay between technology and other, older ways of knowing – of human and nonhuman factors within the ‘agential soup’ – is timely and important. The line of human progress, Bridle reminds us, does not always go ‘up and to the right’. We need, he argues, to embrace ‘unknowing’ and to rediscover ‘cloudy thinking’.

There’s nothing cloudy about David Sumpter’s thinking. Where Bridle offers metaphor, irony and synthesis – not, it should be said, in order to show off, but in order to demonstrate the kind of creativity computational models cannot replicate (yet) – Sumpter offers cool analysis, hard numbers and unembroidered prose. True, Outnumbered does begin with an anecdote about the anonymous street artist Banksy. But this brief rapprochement between the Two Cultures should not be taken as representative. If New Dark Age is a gothic castle with flying buttresses and puking gargoyles, Outnumbered is a Modern Sciences Department building, circa 1976.

Its focus is computer algorithms. A British professor of applied mathematics, Sumpter knows this subject intimately, and is able to convey often difficult material in a friendly and engaging way. Such is the complexity of modern computation that this is only possible at a simplified scale, so Sumpter constructs his own versions of algorithms in order to explain (or ‘mathsplain’) them. Thus Facebook’s data mining technique, which reduces massive amounts of information to certain fundamental features, is figured as a study of 32 people. The mathematics is still hard to follow, and if, like me, you have an interest in this stuff but an anti-talent for understanding it, you’ll occasionally lose sight of the guide. But the journey, though exhausting at times, is worth it.

The main finding that emerges from Sumpter’s book – one he seems to stumble on: like a good mathematician, he shows his working out – is that despite the incredible power and complexity of modern computer algorithms we still tend to overestimate or overstate what they can do. The point about that Banksy anecdote is that a program designed to locate the artist was not only highly questionable in terms of what it added to society, or indeed subtracted from it, but also so approximate as to be useless without non-computational theories about his identity already in play. The story serves as a cautionary tale about the role of hype and hyperbole in this space. Yes, ‘fake news’ and ‘filter bubbles’ are troubling phenomena; but the commentators who deplore them tend to exaggerate their power. As for Cambridge Analytica, the British consulting firm that claims to have helped Donald Trump win the 2016 US election with its analysis of Facebook data, Sumpter is sceptical, not to say dismissive. For him there is no reason to think that CA was able to sway the election. As he puts it, referring to the supposed granularity of the company’s ‘hypertargeting’ techniques: ‘While the data shows us that Democrats tend to like Harry Potter, it doesn’t necessarily tell us that other Harry Potter fans like the Democrats.’

Of course, this situation could change in the future, and computers become so powerful that they come to know us better than we know ourselves. Already there are some worrying signs: though we have a good idea of how Deep Blue beat Garry Kasparov at chess, we have only a very shaky notion of how AlphaGo beat Lee Sedol at Go. AI may give us the slip yet, and our PC’s start crooning ‘Daisy Bell’. But by taking us inside the ‘black boxes’, Sumpter’s book shows us that it is still possible to understand how many ‘robots’ work and what they may be capable of in the future. Though there’s no cause for complacency, the age feels a little less dark for his efforts.