The Mirror past the Reflection

A face stares at you on the television screen. He asks you what you name is and how you are feeling today. You answer wary of giving him too much information. He looks familiar like an obscured reflection. He’s a photorealistic copy of your self, he’s what could be the reality of the next decade of gaming.

The actors of L.A. Noire have attained a creepy level of realism. Cole Phelps, played by Aaron Stanton, is rendered using a global illumination, algorithms that create more realistic 3D lighting effects, and a new technology called MotionScan. He looks really, really good. He also looks really, really fake. Photorealism in video games has been attempted in the past, but never to a degree where facial expressions are mirrored in such a realistic way. However, there’s something eerie with how L.A. Noire presents its actors. It’s something that feels uncannily familiar.
If you watch the trailers, if you see the behind the scene footage, you will notice that Stanton’s performance has an uncanny feel. Phelps looks like Stanton, he moves and sounds like Stanton, and he even has the same facial expressions as Stanton, but that does little to convince me that photorealism is a good thing for video game characters. There are two sides of video game aesthetics: realism and stylization. These categories are tenuous as artistic style and the ideas of what the term “realistic” entails differs from developer to developer. However, the uncanny valley is a real place, and if we continue to trek down into it we could end up a little lost.



L.A. Noire is a great example of a game that is using photorealistic qualities to create deeper, more immersive acting. From what Rockstar Games has shown us, we’re in for quite a thrill ride, but the game’s presentation could go two ways. The big question here is: are video game developers ready to place the human visage into the video game medium?

What makes video games different than film or theatre? What makes the brushstrokes seen in Ōkami any different than the brush strokes of a painter on their canvas? These questions go back to my categories above. Video games present a stylized reality to their players. No matter how borderline photorealistic a video game may seem, it is, in essence, still just a simulation of reality. Many video game scholars at this point would say that video game technology, lighting, design, theory are all progressing at such an alarming rate that photorealism will be within our grasp in the next ten years. It’s certainly a possibility. Blogger and game designer David Hayward wrote an interesting article on this subject. He asks in his article:
The general value of aesthetics is not derived from any one particular style, as evinced by the massive variety in historical and contemporary design. So why do games seem to focus so singularly on photo-realism? Could it be that, because games lend themselves to simulation of reality, their aesthetics meekly follow? Are designers choosing a default option at the expense of aesthetic variety and potential?

This article was written in 2005, and now in 2011 we have games like L.A. Noire presenting photorealistic qualities in games that we’ve never seen before. The article presents some prudent examples of stylism versus photorealism in video games. And it poses a few interesting questions to the reader.

The realistic qualities of a video game have more to do with the stylistic choices of the video game developer than the brute force technology used behind the scenes. L.A. Noire’s faces represent an impressive assimilation of film technology. MotionScan works by placing several cameras around the actor’s face to capture his or her expressions. Global illumination and shaders do the rest to fill in their faces to fit into their surroundings. The actors are transported into the game world. We become an actively participating audience controlling the actors on the stage.
Bertolt Brecht a German poet, playwright and theatre director, not to mention one of the most influential figures in dramaturgy, utilized a technique called the “distancing effect” in his productions. His theatrical technique worked to distance the audience from the reality that was the world of the play, and to remind them that they were watching actors on a stage. Brecht’s influence in video games may be a little tenuous, but his techniques say something interesting about theatrical experiences. He wrote that the distancing effect, “prevents the audience from losing itself passively and completely in the character created by the actor, and which consequently leads the audience to be a consciously critical observer.”



Video games work through suspension of disbelief. They ask us to forget that we are being shown unrealistically depicted scenes and they ask us to participate within their fantasy. Or at least that’s how they have worked. When we engage in an experience like Mass Effect, we know that Commander Shephard is a virtual actor, a puppet through which the player interacts within her world. She is not real, though she possesses many of the qualities that make a video game character come to life. Even when it comes to narrative, character development and dialogue, a stylistic approach to interaction helps the gamer go beyond simple immersion. We forget that we’re playing a game and we start to believe that we are interacting with real people.

L.A. Noire’s presentation proposes something quite different.
The game’s presentation has hints of the uncanny. The way Aaron Stanton is presented to the audience bears certain hints of realism and something you can’t quite put your finger on. I mentioned the uncanny valley above. Mashihiro Mori, a Japanese robotosist, first coined the term in the 70’s. He called the phenomenon Bukimi no Tani Genshō (不気味の谷現象), and it’s a play on the Freudian concept of the uncanny.

Imagine looking into a mirror, but instead of a reflection there staring you in the face was a Microsoft Xbox Kinect and a video screen. The Kinect would take your face, recreate it on the screen and show it to you. You could brush your teeth, comb your hair, wash your face and do everything that you would normally, but that is not, in the technical sense, your face. You would essentially be mediated between reality and virtual reality.

When we see Cole Phelps as he is presented in L.A. Noire’s trailers, we are looking essentially at the mirror past a reflection. We are looking, not at a real face, but at something photorealistic, something even hyperrealistic. 
There’s a difference between looking at an actor on a stage and playing as a video game character on a virtual stage. Hyperreality is a notion developed by French philosopher Jean Baudrillard. He says that we no longer have access to the really real. Instead, our world is characterized by mediations of mediations all of which destabilize meaning and being. Hyperreality replaces reality: it seems real, but is only a simulation. Video games are much like film and theatre. Actors go up on a stage, they act and they perform actions. However, and unlike film and theatre, a video game represents mediation – a space in between – the real and the virtual. The actors in L.A. Noire are representations of real people, their faces instilled on inhuman 3D models. Cole Phelps become a mediated version of Aaron Stanton because he is a photorealistic representation, not an actual reflection. It’s as if a virtual dummy put on Stanton’s skin for our benefit. It’s kind of creepy if you think about the game’s faces that way.
Part of the video game aesthetic is the distance it takes from reality. Video game developers like Rockstar Games are striving to create a higher form of realism for gamers, but to what end? Hyperrealistic characters like Cole Phelps only help to further destabilize a gamer’s notion of the real within their experience. They see his face contort and react like any real human being, and immediately they are turned off, or so Mashihiro Mori would hypothesize. Taking a trip down into the uncanny valley can be a dangerous adventure. Pitfalls abound and path is unmarked.
How real do we want our video games to be? Certainly photorealism presents an interesting development in video game technology. Imagine walking through an actual forest, looking at photorealistic trees over a stunning sunset. Then imagine seeing your character’s face and realizing that you are in fact sitting in a chair while he is experiencing the splendours of a simulated outdoors. The idea of a hyperrealistic game is a worst-case-scenario. However, becoming lost within a video game experience is a cultural fallacy. No one loses their self when reality constantly returns through a phone call or a quiet breeze.
The technology present in L.A. Noire shows a step towards photorealistic human beings in video games. No doubt, we are in for a visual treat when Rockstar releases their game later this year. Cole Phelps looks really, really good. The lighting on him is perfect, his hair is freshly trimmed and his facial expressions are practiced and proper. Aaron Stanton looks good too, but he’s simply part of the reflection on the outside of the game. He’s lost within the experience, a mirror without a reflection. How real do we want our video games to be? I think we should ask Stanton what he thinks about his character being given a run for his money by sadistic gamers looking for photorealistic entertainment.