Consciousness Reframed III

Being There: Interactive Fiction in Virtual Reality

Josephine Anstey, Dave Pape

This paper describes an interactive, virtual reality fiction project, "The Thing Growing." It focuses on the problems and strategies of building a user-centered narrative.

Virtual Reality. User as protagonist. Fiction. Interaction. Manipulation.

Introduction

"The Thing Growing", is an interactive virtual reality fiction initially designed for a CAVE^® system. Its goal is to construct a user-centered story. Building this piece threw up questions about the user as a subject who we were trying to draw into and through a specific experience, but whose subjectivity was one of the creating forces of that experience. Our task was to create a seemingly empty protagonist who anyuser could slip into and fill with their own feelings and actions. Was it possible to make the user the central protagonist of a work of fiction? How could we do it? What were the issues of free will, consent, manipulation involved?

User as Protagonist

We started with a basic story idea: to explore a relationship that was cloying and claustrophobic yet emotionally hard to escape. Someone reading a book or viewing a film may identify with the protagonist but we theorized that in VR we could recreate the tensions of our story directly for the user as the protagonist . VR technology creates the preconditions for this. The graphics are drawn from the user's point of view, changing as she turns her head or travels through the 3D world. She chooses where to go. She is given an input device - in our case a wand with a joystick and three buttons - which can be programmed to provide interaction between the real user and virtual objects. The CAVE includes tracking sensors that are attached to the user's body to feed information about her position and orientation back to the program. Typically in the CAVE the user's head and one hand is tracked. We added a tracker for the other hand. In projected VR systems, like the CAVE, the user sees her own body in the 3D environment.

The CAVE is a stereo device. Jonathan Crary's description of the 19^th century stereoscope exemplifies his argument for the emergence of a differently constructed observer in the early 1800s. He writes ,"It shattered the scenic relationship between viewer and object..."Crary (1990). This relationship is the clear separation between observer and objects observed. It is broken because, on the one hand, the stereoscope embeds the observer in the scene physically positioned in relationship to other objects; and on the other, the scene is embedded in the observer because it is only produced at all by the work of the observer's senses in fusing two flat image. Crary raises the possibility that people were, and maybe still are, uncomfortable and resistant to being constructed as such an embedded subject, a point we will return to later, however, this kind of subject is exactly the kind of subject who can become the protagonist of a fiction, both immersed in the fiction and creating it.

We began our project armed with the CAVE technology, the specific story we wanted to tell, and some assumptions about how people work psychologically. We wanted to create the experience of a relationship - so we needed a virtual character with whom the user could engage. We wanted a relationship in which the other was experienced as too close, almost unbearably demanding, but still somehow loved - so we needed strategies to provoke the user's emotions. One underlying assumption was that the user would react emotionally to a computer character. Human beings personify and react emotionally to their cars. We assumed that they would be equally willing to react to a computer creature that itself appears emotional and directly solicits an emotional response. The emotional "hoops" we want people to jump through are the following; meet the virtual character, develop a variety of good and bad feelings for the character, and finally reach a point where they have the choice of killing the character, or not.

The opening scene is a plain. A voice-over suggests the user go to a small shed in the distance. Inside the shed is a box. A key appears with which the user can open the box. As the box opens the Thing emerges. Its body is composed of multi-colored, semi-transparent, pyramids. Life-like movement results from motion tracked animation and creates a strong illusion of an autonomous being formed from a collection of primitive shapes. The Thing is delighted to be free. It looks at its liberator, fulsomely thanks her for saving it and declares that it loves her. We hope that the user feels a twinge of pleasure in response. Then the Thing announces that it is going to teach the user a special dance, a dance just for the two of them...

Technically, all our information about the user comes from the tracking devices, and the state of the buttons and joystick on the wand. A stream of numerical data about her movements tells us very little, but if it is analyzed against some desired state, meaning is created and appropriate feedback can be supplied. Therefore we decided that one of the main (inter)activities between the user and the Thing should be the dance. The user's actions become an intrinsic part of the experience. If she follows the dance step the Thing is demonstrating, it can praise her. If she's not so good, it can encourage her, and dance with her as they repeat the step. If she ignores the Thing and tries to navigate using the joystick, it can ask her not to run away. A second order of interpretation also exists. If the user is dancing she is compliant. If she is not dancing she is disobedient. The Thing is programmed to change mood accordingly.

We believe the work going on during our VR fiction is very similar to that described by Judith Williamson in "Decoding Advertisements" with respect to commercials, "... we create the meaning of a product in an advertisement; ... we take meaning in from the product; ... we are created by the advertisement; ... we create ourselves in the advertisement." Williamson (1978). The user works very literally to create the 3D space and the virtual character - first by fusing two images to create stereo, second by assembling a humanoid being from a disconnected collection of pyramids animated by motion capture. The user takes in meaning from the environment and the Thing - the key says it can open the box, the Thing reveals its emotional state. The Thing creates the user by providing one side of a relationship, and boxing the user into the other side. The user can choose to do what the Thing wants or not, but she is "always, already" implicated in the story, creating herself in the story and the story itself by her actions.

The dance activity provides a rich environment for this process to develop. It is replete with patterns of behavior and meaning which we exploit to further the relationship and story. Humans learn by mimicking - its a very hard-wired response to mimic motion back to someone or someThing else. Dancing suggests intimacy - and we want this dance to be imbued with both a sense of intimacy and seduction. People can be very nervous about moving their bodies and dancing correctly - we wanted the Thing to have the edge of power, it is judging their performance. People have a definite sense of personal space - in the 3D environment the Thing can violate that boundary and get uncomfortably close, an action that can seem threatening or clinging. Underlying all this is our story-line. Essentially in this section the Thing behaves like a typical abuser. While at first it seems sweet, its dominating personality soon develops. Alternately and irrationally it showers lavish praise and abusive slap downs on the user. It is only interested in doing what it wants to do, and making the user obey it by any means necessary.

The dance scene ends with the Thing going off in a huff, either because the user is not obeying it, or after a timed interval. Later the user and the Thing are reunited and dragged to an underworld environment where the Thing's four cousins sit in judgment over them and the love relationship that they perceive between them. Relationships between a meat object (the user) and a Thing are forbidden. This scene was designed to bring the user and the Thing together against a common enemy. The Thing enables their escape and equips the user with a gun. The Thing exhorts the user to kill the cousins. After all the cousins have been dispatched or have escaped, the user is left facing the Thing, holding a gun. At this point we wanted the user to consciously decide to kill or not kill the Thing.

Iterative Creative Process

In many ways it is fortuitous that our virtual character is dominating because a very pro-active character can both "tell" and control the story - the user blames her lack of control on this character and on her own inability to wrest control from it not on the limitations of the program. And of course the program is limited. We cannot write code for every possible action a user may make. We need to find ways to constrain those actions into a manageable subset that the program can react to. Pinning down the human subject and her responses was not a one shot deal. The creation of a successful empty protagonistic space depended on testing that space with a variety of users and watching what they did in it. Then we adjusted, refined, and added to the Thing's functionality so that it had responses that could fold the users' different reactions back into our narrative thread.

This iterative process also honed the story since watching users made it very clear when people were not really "getting" the narrative we wanted to send. The culmination of the story is to bring the user to a point where she can destroy the Thing if she chooses. A major problem was creating enough ambivalence at this moment. In early versions virtually everyone shot the Thing. We adjusted the program to try and make the Thing more likable. We added a section where the user moves and the Thing mimics her. Many people liked this moment of control, when they are mirrored by the other, when the other is doing what they do for once. We clarified the story of the forbidden nature of the love affair. We made the cousins meaner and the Thing more pitiful as it begs the user to protect it from them. If the user doesn't kill the cousins the Thing becomes hysterical because she is not protecting it. If the user does, the Thing is aghast at the slaughter of its family and this bloodthirsty side of the user. In each case we provided reactions to the user that we hope channel them to the moment of truth - killing or not killing their "friend" (this is how many user's referred to the Thing), with the maximum amount of ambivalence about their own actions. After all this less people killed the Thing.

Manipulation

In "Decoding Advertisements", Williamson is showing how adverts manipulate us. Clearly "The Thing Growing" is also a work of manipulation, our main problems was trying to make the manipulation work for as many people, for as much of the time, as possible. Is this problematic? Crary suggests that in Victorian times, "Photography defeated the stereoscope as a mode of visual consumption ... because it recreated and perpetuated the fiction that the "free" subject of the camera obscura was still viable." We, however, observe a subject who is comfortable embedded in a 3D environment. Although at present few participants are very familiar with VR systems most people have some experience with immersive and interactive media from video games and the web. After her experience one participant suggested that the way to get the most out of the experience was to go along with it as much as possible, another was very aware of the way it was trying to lure and manipulate her.

We believe that users can give informed consent to experiences, such as our virtual reality fiction, and it is only through their active and continuing consent that the fictional experience happens. We observe a smart subject who responds to mediated experiences on two distinct levels, aware of both what is represented and how it is represented. This subjects flows between the levels commenting on a politician's media persona, and how and why he and his spin doctors construct that persona; referring to a character in a video game as both I and he, then discussing the kind of graphics used in the game; responding emotionally to a film and also noting the cinematography and set dressing; both immersed in a VR experiences and able to deconstruct it.

Williamson notes how rebellion, cultural criticism and we ourselves are co-opted by commercials; women's lib sells cigarettes, flower power sells cars, deconstruction sells anything and everything, "Having deciphered its surface, we then discard the surface as we "break through" to the "hidden" meaning. So this entails a reversal of the ad's slipping into you:you are invited to slip into it, to enter its space, drawn in to participate in a "discovery" of meaning [4]" Similarly we believe that social and social critics and artists co-opt tools, terms, and ideas from the commercial realm as part of their work to continually rediscover, recreate and represent society's constructions, contradictions and faults, including the construction of us. Part of that job is to create experiences for a subject who may have been constructed to decode advertisements but is not limited to that activity.

Crary, J. 1990. Techniques of the Observer. Cambridge Mass: MIT Press, p.127

Williamson, J. 1978. Decoding Advertisements: Ideology and Meaning in Advertising. London: Marion Boyars, p.41.

Crary, p.133

Williamson, p.167

Josephine Anstey has just received her MFA from, and Dave Pape is a PhD student at, the Electronic Visualization Laboratory, University of Illinois at Chicago. "The Thing Growing" has shown at SIGGRAPH, & the Ars Electronica Center amongst other venues. {anstey|pape}@evl.uic.edu http://www.evl.uic.edu/anstey/THING

Acknowledgments: "The Thing Growing" was built at the Electronic Visualization Laboratory at the University of Illinois at Chicago.