AN UNCANNY VALLEY, IN REVERSE
Ghostly palaces shimmer on the horizon. Eyes—human, animal, or alien—peer from every surface. Faces, limbs, buildings and hills are transformed into creatures: odd combinations of fish and bird, boneless sluglike creatures, and dogs. So many dogs. Welcome to the automated surrealism of deepdreams. [1]
These bizarre images are the accidental byproduct of researchers trying to probe the behaviour of their image recognition software. Such software is essentially comprised of a neural net: an imitation of the material substrate of our visual cortex. Neural nets can be very effective, but the mechanism by which they work is opaque. A neural net can find your face in a photo or win an image recognition contest, without understanding anything about what it sees and without its creators understanding how it works. So researchers sought to visualise their operations by a feedback loop: give the neural net a random image, amplify whatever patterns it finds, and feed the results back to it.
A neural net is a stack of layers: each layer takes values from the layer below it, performs simple mathematical operations on them, and passes its values to the layer above. The lowest layer takes as its input the image to be classified, and successively larger-scale and more abstract forms are recorded as values flow up the stack. The top layer is a set of scores indicating which of the net’s categories best fit the image—the result. Before it can work, the net is ‘trained’ by showing it millions of images already classified by humans, and reinforcing the correct results. The result is a complicated tangle of software that hasn’t been ‘programmed’ in the normal sense, but which can be very effective. They’re called ‘neural’ nets because they’re based on how neuroscientists believe the visual cortex of our brains recognises images: the layers are like arrays of artificial neurons, some of which match very basic shapes, others of which respond to large-scale features.
If one ‘turns the volume up’ on a neural net, increasing its tendency to recognise features, and then feeds the results back to it ten or twenty times, we get deepdreams. The results are strongly reminiscent of the visions provoked by psychedelic drugs. Perhaps a drug like LSD triggers an analogous process of de-inhibition and feedback within our own flesh-and-blood neural nets, enhancing our tendency to see faces and patterns in anything and everything. The output varies greatly depending on which layer of the neural net is extracted for the feedback process. Lower levels produce abstract patterns of dots and spirals; at higher levels we see the doglike faces. At still higher levels the faces become distorted and eventually merge into strange tangles of biomorphic circuitry. This lets us see which features of the input images the different layers of the neural nets are responding to.
We love to anthropomorphise computers, but there is no dreaming subject behind these visions. The neural nets’ hallucinations depend directly on the images on which they’ve been trained. The annual ImageNet Large Scale Visual Recognition Challenge (ILSVRC) is a popular contest for image recognition scientists to show off their new systems and, of the 1,000 target categories it uses, over ten percent are dog breeds. GoogleNet, the neural net used for the original deepdreams images, was trained against these categories, which explains its obsession with dogs. If the same technique is used on a net trained on the 205 scene categories of the MIT Places database, the hallucinations change to palaces, pagodas, lecture theatres and windfarms, generating beautiful and strange imaginary landscapes.
Repeated illusions: from a digital simulation of neurons, a world of imaginary forms and textures is formed, which owe their convincing nature to the fact that they themselves are copied from databases of real photographs. I find deepdreams images fascinating and often very funny, although they share with most psychedelic art a tendency to fall into kitsch. Creating them is addictive. Many people, however, find them repellent and disturbing. An uncanny valley in reverse: a mechanical parody of the act of seeing, not the seen object, which is too close to the original for comfort.
As well as a writer, Mike Lynch is a developer and analyst at the University of Technology Sydney. Read more of his work on his website and follow him on Twitter @bombinans.
To play around with the deepdreams code yourself, see the Google code repository and Mike Lynch’s fork of it.
For more information about deepdreams, see the blog post Inceptionism: going deeper into neural networks by Alexander Mordvintsev, Christopher Olah and Mike Tyka, 2015.
This piece appeared in Materiality: FAKE. Would you like to download it as a cute little A5 booklet? (For a small fee). Go to our shop to download Materiality: FAKE, Part VI.