The Great Graphics Debate: Photorealism, Emotion, and the Uncanny Valley

That's not me in my profile picture. Well, it is, but not really. Those are my clothes, my hat, my shape and form, and it was certainly taken in New Orleans when I visited the art museum with some friends the week before Thanksgiving, but I swear it isn't me. It can't be because I'm typing this post. It's a moment captured in time when I was someplace in the past, a snapshot that I wasn't even aware was being taken. In fact, the figure in the photo looks a little too much like me for my comfort. It's a shell, a construct made by a machine to replicate my appearance. It's...unsettling.

In 1906, Ernst Jentsch coined the term "uncanny" as a bizarre state in which an object in perceived in terms of its uncertainty, but it was not until everyone's favorite psychoanalyst, Sigmund Freud, took the term, injected some sexual insight, and produced the meaning we have for it today. I'll spare anyone reading this the details, but the "uncanny" is, more or less, a concept of a particular moment when something is both familiar and alien at the same time, creating a feeling of discomfort. It is this concept that led to Masahiro Mori's hypothesis of the "uncanny valley" which posits that as robots and graphics become more human-like without behaving exactly like humans, people will approach these technologies with a fair degree of disgust and discomfort. The most oft used examples now involve robots that look so close to human that their flaws become magnified, making them, well, creepy. Video game graphics suffer from the same obstacles as robotics; the uncanny valley seems impassable.

Recently, 2K Games' Chris Hartmann spoke out in favor of photorealism and how it can help evolve the industry and elevate the medium in terms of emotional content.

"Until games are photorealistic, it'll be very hard to open up to new genres. We can really only focus on action and shooter titles; those are suitable for consoles now...To dramatically change the industry to where we can insert a whole range of emotions, I feel it will only happen when we reach the point that games are photorealistic; then we will have reached an endpoint and that might be the final console." Chris Hartmann, 2K Games

The photorealism Hartmann so earnestly strives to reach lies on the other side of the uncanny valley, the gulf that separates the recognizably false from the earnestly real (as seen in the following chart):

Current graphics and character models seem to be stuck in this uncanny valley. Sure, character models look great in stills, but as soon as they start moving, they seem off. The lip sync is strange. Sometimes they float above the groundf. The character's hair doesn't move like it should. And in the instance when sex is involved, it looks about as convincing as smashing two naked Barbie dolls together (and almost as arousing). In order to get to the emotional photorealism that Hartmann sees as the zenith of graphical progress, traversing the uncanny valley provides the only logical route. This is the promise held by every single next generation console since the days of the 64 bit machine. I, for one, can't help but see the ultimate goal as a calculus limit rather than the graph above. We'll get closer to zero, but I can't tell if we'll ever breach that invisible wall. I do agree with Hartmann, that there are indeed merits to striving for photorealism in graphics, but I don't think that they are the only way to get a new ranges of emotion. And I've arrived at this conclusion because this is not a new issue.

When photography and film were young, many people had adverse reactions to it, finding the replications of themselves a bit frightening at the honesty it captured. Images looked a bit too real (or uncanny), so much so that some found them initially disturbing. When the camera came into common use, painters and artists of all traditions began to question whether or not there was a purpose for what they did when an object could capture an image with realistic clarity better than a paintbrush ever could. Their goal, then, shifted to offer new perspectives of the "real" or rather to reveal alternative ways of seeing the world. Hence the rise of Impressionism, Post-Impressionism, Surrealism, Cubism, and a few other "-isms" that seemed to pop up and disappear as quickly as art trends tend to do. Video games, especially indie games, enact a similar reactionary process of rebellion and perspective shifts. As graphical fidelity approaches photorealism, some developers provide alternative experiences, similar to the ways in which artists experimented with form, structure and subject during an age of technological revolution.

For example, below are two images, the first a Post-Impressionist painting from 1891 and the second a still from an indie game in 2012:

"Plage a Heist," by Georges Lemmen (1891)

Journey, thatgamecompany (2012)

The images are strikingly similar. Lemenn's painting shows a beach, a sailboat, and a lone figure. The dreamy movement of the picture captures something that a camera cant quite do in 1891. There's an unreality to the painting and a blending of water, earth, and air that are distinguishable but somehow draw together in an array of natural and unnatural color. Quite similarly, the lone figure of the desert in Journey evokes a feeling of isolation, loneliness, and quiet contemplation amid a sweeping landscape. To say that these images cannot depict emotion because they lack the photorealism offered by the technologies of their respective eras seems absolutely ludicrous (though the Post-Impressionists were often mocked by their contemporary critics). Here's another example from a different school of art:

"Quarry Bibémus," by Paul Cézanne (1898-1900)

Landscape from Minecraft (2011)

Cezanne's proto-Cubist landscape (like the "high" Cubists would a decade later) highlights not the subject itself but the shapes that compose it. The image challenges the viewer and critic by changes perspective and forcing us to think in terms of its construction of shapes rather than its fidelity to reproducing a "real" image of the quarry. Similarly, the landscape from Minecraft foregoes realism in favor of showing the components of the game. Since Minecraft is a game about building and composition, its aesthetic enforces this concept by boldly showing the blockiness of its architecture. Gameplay is reflected through art direction in the same way that Cubists reflected the act of building art by focusing on the shapes that make up the subject rather than the subject itself.

As games evolve and attempt to bridge the uncanny, we will see games come closer to photorealism, and this is not a bad thing. In fact, it's a great goal. It will keep pushing artists to reach for smoother textures, utilize more complicated engines, and strive for the true emotional impact that Hartmann posits. As a result, we will see more independent games work with alternate aesthetics to build games that are not focused on photorealism, offering different approaches to emotional experiences. In this sense, I disagree with Markus "Notch" Persson's rebuttal to Harmann, in which he said, "No, Christoph, you LIMIT the number of new genres if you focus on photorealism." Without the focus of photorealism of the mainstream market, we wouldn't have great alternative games like Journey, Minecraft, Limbo, and Braid. The smart money says, the indie market isn't going anywhere.

However, these are just visual goals; the gameplay has to match the emotional weight of what's on screen. I can get visual emotion from paintings, photography, or film. I play games because gameplay can offer experiences that other media cannot, and unless developers can use gameplay to provide emotional context for what's on screen, they are missing the point of a game.

Cheers,

--David