"The Thing Growing"
Catalog Text
AEC 98
Interactive Art

Text for the AEC Catalog

**Interactive Story Telling**

The Thing Growing is a virtual Frankenstein experiment: an attempt to create a Thing with a will and personality of it own who engages us in a virtual world.

It started as an approach to interactive narrative. A simple narrative structure has three acts; 1. Set up. 2. Conflict. 3. Resolution. The 'user' identifies with the protagonist and shares the emotional ride that this narrative bridge ordains. The usual method of making an interactive narrative (making the user the protagonist), is to create a sprawling hypertext with many branches that the user can choose to explore. Although interesting as a text experiment, this exponential branching method becomes much less feasible in other media (film, video, computer graphics, animation, virtual reality) when the production of just one version of a scene can be costly and time-consuming.

Nor are we convinced that the most interesting interactive narrative is created simply by increasing the user's choices between narrative branches. Many non-interactive stories are powerful because the protagonist is plunged into a difficult situation where the only choice is one which she wouldn't usually make. The narrative unfolds as she battles a loss of agency and figures out how to restore a measure of control to her life.

We decided to make the user into the protagonist by concentrating on establishing a relationship between the user and just one character. Instead of creating many possible paths to the story, we wanted to make a virtual Thing with many possible responses to the user; a Thing with a lot of agency over itself, and its environment, who would engage the user at an emotional level.

A major problem in creating a credible interaction is to convince the user that the Thing "knows" about her at some level; to make the user feel acknowledged as an individual. A central element of the piece is the Thing's bossy persistance in teaching the user a dance. The Thing's response changes depending on the user's performance and its own moods. It cajoles, whines and threatens to make the user copy dance steps; compliments good dancing; is distressed by sluggish movement; watches the user; dances with her; and recaps the steps learnt so far.

A dance was chosen because it creates a quite tightly controlled context for the interaction. Using the basic CAVE VR system, the Thing only "knows" where tracked parts of the user are - typically we track the head (so that stereo is correctly computed) and the hand (which holds the 3D mouse). We added two more sensors to track a second hand and the body. The Thing gathers information about the user's position, orientation and activity from the tracking system. It then interprets that information to determine if the user is attentive, compliant, active, lazy or indifferent.

Another method of feeding the Thing information and changing its emotional state is a networked version of the application, where a second human is a voyeur on the scenario developing between Thing and the primary user. The second user is an invisible "fly on the wall" in the virtual environment. He watches the user and passes information on how she is dancing to the Thing, in addition he can choose from a menu of moods for the Thing.

In Computers as Theater Brenda Laurel introduces the idea of "constraints" - she asks "How can people be constrained to work only with potential that is inherent in (or amenable to) that which is already in the representational world?" ("Computers as Theater", 1993, p135). She suggests that the context of the story makes the user willing to conform to the constraints and limitations of the technology - "If the escape key is defined as a self-destruct mechanism, for instance, the constraint against pressing it in the course of flying a mimetic spaceship is intrinsic to the action." ("Computers as Theater", p103)

Similarly in building the Thing's intelligence component we try to construct its character so that the paucity of the information it is working with is not apparent. The Thing teaches you to dance because it can receive data about how you are moving and give appropriate feedback. The Thing is high-handed and willful - in part because of the exigencies of the story line and in part to hide its stupidity. Its moods are inconsistent, it arbitarily praises or abuses the user for the same behavior - in part to mimic the inconsistency of many people, in part to hide its ignorance.

In The Thing Growing we also play with the use of linear and non-linear story telling. At some points we want the story to be very linear. We suggest to the user - "go to that shed, open that box and Kaboom, out pops the Thing." At other points the story consists of non-linear interactions between the Thing and the user. Questions arise as to how cooperative people will be in the linear part of the story. Once we have established the Thing as a character, we use its bossy nature to push the user in the direction we want. However, an interactive environment leaves many degrees of freedom, and a determined user will always be able to go off in another direction entirely - into a part of the virtual world that isn't built yet. Laurel suggests that people are very used to "suspending disbelief" for the greater enjoyment of art or entertainment. Perhaps interactive pieces also require the user to "suspend will." We mean this in the rather Lacanian sense that the fullest enjoyment of the work may be contingent on the user accepting contextual clues that suggest what her behavior should be. In becoming the protagonist she accepts the contextually embedded rules of that role.

**Some Technical Notes**

Virtual Reality may offer many exciting new possibilities for artistic expression, but it is not yet accessible to most artists. VR is still a very technical field, home mostly to computer engineers and research scientists. A number of interesting VR art works have already been created, but in general they are by those few artists who have the temperament and time to also be serious computer programmers (or vice versa), or by teams where the artist's vision must be implemented by others - a sometimes difficult and incomplete process.

The Thing Growing's software is based on XP, an experimental programming system we have been developing for creating interactive CAVE applications for public exhibitions. XP arose from the desire to allow artists -- ones with little expertise in such things as C++, OpenGL, scene graphs -- to create large scale CAVE environments. The system was designed to make it easy to build interactive worlds with a number of features common to these sorts of applications. It was also designed with public exhibitions in mind -- we attempt to deal with some typical problems or needs that we have observed over the course of numerous past CAVE showings.

XP builds on a set of existing toolkits (CAVElib, Performer, OpenGL); defines a standard system for building basic application elements, such as behaviors and user interactions; and provides a number of predefined tools for activities common to virtual environments. It uses a component approach, where the basic elements are programmed into modules which are plugged together to create the application's behaviors, and grouped in hierarchical scene graphs which define the 3-dimensional environment.

The predefined tools include modules for the basic positioning of objects and locational sounds in the 3D world; navigation through the world; collision detection; picking up and dropping objects; triggering events with the wand or the user's position; and playing back animations. Further behaviors beyond the standard ones can be created as additional modules by programmers, by extending the base XP class system. The plugging together of modules is done in a simple script-like scene description, a system easily learned and used by non-programmers.

In the case of The Thing Growing, the XP system was extended to create a simple intelligent agent - the Thing itself. We built a motion-capture recording tool in order to create a library of actions for the Thing's body. Each action is recorded in time to a short sound-bite of the Thing's voice. When the piece is running, the Thing's intelligence unit selects an appropriate action and sound according to the point in the narrative, the user's actions, and the Thing's own emotional state. Additional components were also built to feed the intelligence unit the necessary information about the user's actions by collecting and interpreting the data from the trackers that are attached to the user.

Besides creating virtual reality applications, XP is designed to deal with problems that typically arise when they are shown in public exhibitions. The system has to be able to reset the graphics or sound quickly, without having to restart from scratch, be able to run over and over all day without long restarts when a new group arrives, provide simple navigation, including 'guided tours' at points where the user must go a certain way or might have trouble getting through a tight spot, use collision detection to keep the user in the right part of the world, but be able to disable it when he gets stuck in a crack in the database, and be able to jump back to the starting point when the user gets lost.

The Thing Growing is an experiment on several levels; technically it uses the XP system designed to make working in VR more accessible; it attempts a solution to the problems of interactive narrative by concentrating on one character's library of responses rather than using branching story paths; it experiments with Brenda Laurel's concept of "constraints" corralling the infinite choices of the user with the narrative context; and it aims at the rather elusive grail of creating a relationship - an emotional relationship - between a human and a Thing.