Learning and Building Together in a Virtual World

Maria Roussos, Andrew E. Johnson, Jason Leigh,
Christina A.Vasilakis, Craig R. Barnes, and Thomas G. Moher
University of Illinois at Chicago
http://www.ice.eecs.uic.edu/~nice/




The NICE project is an immersive learning environment for children implemented in the CAVE and related multi-user virtual reality technologies. In NICE, children explore and cultivate virtual ecosystems by collaborating over networks with other remotely-located children, and construct narrative settings from their interactions in the real and virtual world.




Eddie is a 9-year old participant of the NICE project, a learning experiment set out to explore the educational possibilities of an immersive virtual reality (VR) world. He wears a pair of light-weight stereoscopic shutter glasses, holds a 3D mouse as his main interaction tool, and enters the CAVE, a room-sized virtual reality system displaying vivid imagery on three walls and the floor. Jim is an adult whom Eddie has never met in person. He is remotely located at a different VR system, but shares the same virtual environment as Eddie. His role is that of a facilitator who provides help and guides the activity. Both Eddie and Jim can choose from a variety of virtual characters (avatars) to represent themselves in the virtual space (Figure 1). Their interaction lasts for about an hour and the following scene captures part of their activities while tending a virtual garden:

The various seeds for planting vegetables and flowers are stored in the crates around the garden area. After observing the varieties, Eddie picks some seeds from the tomato bin and plants them in the soil. The tomatoes start to grow; he makes sure to separate the ones that are too close together and clear out the surrounding weeds by recycling them in the compost heap. He then picks the rain cloud and pulls it over his newly planted tomato patch: "Ok, perfectly on it. Now let me shrink and see how it grows... Oh cool it's growing!" When the vegetables are adequately watered, they pop up an umbrella so Eddie knows when to pull the cloud away. He discovers similar behaviour with the sun: when placing it over his tomatoes for too long, the plants wear sunglasses.

Jim joins in the activity to help keep the garden tended and healthy. They exchange seeds and flowers, and work together to gather the successfully harvested vegetables, lift the heavy barrel, and carry it away from the garden. Jim asks what they will do next. Eddie has a plan already: "I'm going to shrink and go underground." "What do you think will be underground?" Jim asks. Eddie replies, "I don't know, maybe I'll see the roots of the vegetables that are growing". He also discovers several animated subterranean dwellers such as the ladybug.



Figure 1. Children, represented by a variety of avatars, in the NICE garden.

The garden Eddie and Jim created will continue to evolve, even after they leave. Eddie can check its progress on the web or come back at a later time to interact with the other children participating in the ongoing work. Next time he may use his familiarity with the environment to take on Jim's role as guide, and help the new kids with their gardening activities. Next time he will take on Jim's role as guide, and help the new kids with the gardening activities. Eddie can also reflect on his time in the garden and his interaction with Jim, by taking home the illustrated story generated by the system.

Combining Constructionism, Narrative, and Collaboration
The above virtual experience is implemented as part of the NICE project11, 12, an effort to build Narrative-based, Immersive, Constructionist/Collaborative Environments. Developed at the Interactive Computing Environments Laboratory and the Electronic Visualization Laboratory of the University of Illinois at Chicago, NICE aims to create a virtual learning environment based on current educational theories of contructivism, narrative, and collaboration. The theoretical framework focuses on combining concepts of constructive group activity and narrative, within a motivating, exploratory setting.


Constructivism, as a theory which believes that learners construct their own knowledge by engaging in self-directed learning activities, is being applied to an increasing number of educational projects. A few decades ago, John Dewey {\cite Dewey, 1960} argued that children must actively construct knowledge (and link new ideas to existing knowledge) by drawing it out of experiences that have meaning and importance to them. Jean Piaget believed that initiative, play, experimentation, and discovery are fundamental to the development of the child's learning. More recently, Papert calls for a further distinction to constructivism, named constructionism, which involves the engagement of children with materials in the design and construction of a personally meaningful artifact, such as a piece of art work, a story, or an interactive computerized object {\cite I. Harel}.

We are attempting to blend these (the above) philosophical approaches in our implementation of constructionism in VR: an informal learning environment which is engaging, based on exploration, discovery, play, and the design and generation of an external object. NICE is constructionist in a direct (literal) sense. Students construct an electronic artifact -the garden- as an object of their own design and nurturing. The virtual tools they have provide them with creative possibilities for making, manipulating and exploring the objects and (soon) the systems within the virtual environment. The children may choose to experience the world at a normal size, or shrink themselves to observe the animated effects of the flora and fauna which inhabit the garden, or cruise underneath the garden to see the root system (and subterranean dwellers) firts-hand. Picking up and stacking large or heavy objects, exchanging items with other remotely located children, instantly observing their changes over time, disappearing, flying, or leaping in the air, are only a few of many ways to explore and discover, design and create, observe and understand the surrounding environment by being an active part of it. The goal is for these tools to foster creativity within an engaging context.


Figure 2. Shrinking in the carrot patch

[ now introduce the narrative discussion which should address these points:

The constructed product is, importantly, something which can be shared with teachers, parents, and other students, and because it is an electronic artifact, geographic constraints become moot.

In addition to the electronic artifact, a more tangible, in someways, product constructed in NICE is the narrative, the stories formed and created by the kids that participate. In its present form, the narrative revolves around the construction of the ecological microworlds and the interaction of the characters. Every action in the environment adds to the story that is being formed continuously, even when there is no interaction by the kids. The action is captured through a simple transcript like this:

Eddie pulls the cloud over his carrot patch and waters it. 
The tomatoes complain that they don't get enough water.
Ginger recycles a weed in the compost heap.
The transcript is then processed by a simple parser, which replaces some of the nouns in the text with their iconic representations, and is posted on a Web page in the form of a Rebus-like visual narrative. (Figure here)

While simple, these "stories" produced by NICE are not dissimilar from the stories of emergent writers: most of the time they do not accomplish narrative closure but rather they come across as a series of actions and interactions between the characters and the world\cite Storywriter . They are, however, multi-threaded and dynamic: each child may choose to join or leave the story set at any time, while the microworlds continue to evolve just as in real life; the plants grow, the animals populate newly formed ecosystems or migrate to other areas, the weeds take over neglected gardens, and the flowers wilt when not watered. The child can reenter and continue the story or start another one. This way, when looking back at the printed story, the child may gain a better understanding of the subject matter and the process, reflect on the interaction, and enjoy the acheivement of a completed product.

The children enjoy the process of creating stories, especially when they get to take home a written or illustrated version of the narrative, or the electronic version of the garden, generated as a virtual model.


[ ... and now Collaboration:

Vygotsky's (constructivist) theory emphasized the social context in learning through the experience gained by participation in group activity. Social life among children offers extensive opportunity for co-operation (including conflict etc). We experiment with collaboration modalities in ways which focus on the combination of cooperative learning across both virtual and physical communities.

Theories that emphasize the importance of social interaction in cognitive growth suggest that successful collaborative learning involves more than the final creation of a learning product16. Verbal interaction, collective decision making, conflict resolutions, are developed as group dynamics of a classroom, and form part of the knowledge construction process.

The obvious traditional methods of collaboration involve social interaction among children located in the same physical space. As the CAVE supports multiple simultaneous physical users, and requires only that users wear special lightweight glasses which allow them to see both the virtual and physical worlds simultaneously, a number of kids are able to participate in the NICE learning activities at the same time. We have conducted such activities with classrooms of 28 children at a time, distributed amongst 2 VR systems...

Collaboration of virtual communities refers to communication and shared experience between children who are geographically separated, and is implemented through the use of VR over a high speed network. Children using different platforms at different locations can interact with each other and share the same space. The representation of all participants in the NICE garden is accomplished through the use of avatars which consist of a separate head, body, and hands. As each person's head and hand are tracked on each end, this allows the environment to transmit gestures between the participants, such as the nodding of the head, or the waving of the hand to the other participants without having to learn a formal gestural language. As the child waves her hand in the real world, her avatar waves its hand in the virtual world. The representations are sufficiently detailed to easily allow the communication of notions of relative position. Additionally, the communication process is enhanced with the use of audio between the various sites through the use of wireless microphones

mention ASYNCHRONOUS communication here

Collaboration in the NICE virtual environment also includes the interaction between the kids and the "intelligent" agents which populate it. The agents may be simple computer characters, or actual people which are disguised as agents and simulate agent behavior. In terms of their physical movement, agents and avatars are similar so the children can be unaware of which characters in the environment hide a computer agent and which a human. Why is this important? In the real world classroom, the teacher may have difficulties finding the right balance between intervention and facilitation. VR affords opportunities to modulate differences in status by allowing teachers to observe and direct by participating as other characters in the environment. A teacher may be just another gardener or a less conspicuous character, such as a bird. Also it's important as mechanism for role differentiation and individual responsibility amongst the learners.

The Underlying Technology
The above ideas are being developed in a system which is an outgrowth of two previously designed systems, CALVIN and the Graphical StoryWriter. CALVIN (Collaborative Architectural Layout Via Immersive Navigation) 6, 7 is a networked collaborative environment for designing architectural spaces. The Graphical Storywriter 14, 15 is a shared workspace, where young children can develop and create structurally complete stories.

VR hardware
The NICE software is primarily designed to run in the CAVE2. The CAVE is a 10 x 10 x 10 ft. room constructed of three translucent walls. Stereoscopic images are projected onto the walls and floor (Figure 3a). A participant entering the CAVE dons a pair of light-weight LCD shutter glasses to mediate the stereoscopic imagery. A magnetic tracker, attached to the glasses relays the position and orientation of the user's head to the computer. One user interacts with the virtual environment using `the wand', a simple tracked input device containing a joystick and 3 buttons. A smaller, one wall version of the CAVE is the ImmersaDesk, with a tilted screen which resembles a drafting table (Figure 3b). The graphics for the CAVE and the ImmersaDesk are generated by a Silicon Graphics Onyx with a PC to support the tracking system.


Figure 3. (a) The CAVE(TM) and (b) the ImmersaDesk(TM) VR systems.

The CAVE's room-sized structure allows for multiple users to move around freely, both physically and virtually. However, only one person is the navigator in each CAVE, and it is this person's viewing perspective that every other participant is seeing. A typical session has one child wearing the tracker glassesand handling the wand to pick and place objects, while the other children observe and provide suggestions.

The wand is the physical interface to the virtual world. It is used to navigate around the virtual world, and to manipulate virtual objects within that world. The user can move around within the confines of the CAVE, walking around or through virtual objects, and use the joystick on the wand to move the CAVE through the virtual environment. The system provides the options of flying over the world, jumping into the air, or adjusting the floor of the CAVE to coincide with the height of the landscape, thus allowing the user to climb over terrain or ascend and descend stairs, by physically walking in the CAVE. Visually, the wand is mapped to the arm of the child's avatar. If Eddie is moving a carrot, he moves the wand over it, clicks on the wand button to pick it up, moves it to the desired location, and clicks on the same button to let it go. When he is waving at Jim, he just waves the wand back and forth.

System architecture
The NICE system is organized into three main components: the behavioral module, the networking module, and the graphics and interaction module (Figure 4).


Figure 4. The NICE system architecture.

LIFE: the behavioral system
NICE's behavior system (LIFE) is the central behavioral server that handles the actions and attributes of the characters, the objects, and the story structures. The plants, trees, and flowers are simple agents which follow rules of evolution and behavior. They share a common set of factors which contribute to their growth, such as their age, the amount of water and sunlight they need, and their proximity to other plants. The combination of these attributes determines the growth and affect of each plant. As soon as Eddie drops the carrot seed on the ground, the carrot starts to grow. When he pulls the cloud over to water it, the carrot grows faster. The carrot pops up a small animated umbrella to let Eddie know that it doesn't require any more water.

The genies are more sophisticated agents that differ in purpose and personality. Their action is triggered when the children approach them or ask them questions. An example of a less intelligent genie is the talking signpost, which points and directs to various locations. The more intelligent genies, like the owl which provides wise advise, are currently simulated by humans.


Figure 5. A NICE story on the WWW.

Networking
The network component in NICE allows multiple networked users to explore the same virtual space with the number of participants limited only by the bandwidth and latency of the underlying network. Multiple distributed applications running on separate VR systems are connected via the central LIFE server to guarantee consistency across all the disparate environments. The communications library uses UDP or multicasting to quickly broadcast position and orientation information about each participant's avatar, and uses TCP/IP to broadcast state information between the participants and the central LIFE server. This information is also sent to the NICE web page where visitors can watch what is happening in the virtual environment. We are currently working on a java interface to allow web-based clients to actively participate in the vitual world.

Future versions of the network component will use CAVERN (CAVE Research Network, see evlweb.eecs.uic.edu/spiff/covr) as its main networking and database broker. CAVERN is networking middleware, optimized for VR, which supports both reliable and unreliable networking services, as well as provides richer database support for sustaining persistence in VR.

Graphics and Interaction
The core of the graphics component is the CAVE library, the programming library originally written to support the CAVE hardware 2. It is, however, capable of supporting a number of different VR platforms, including ImmersaDesks, head-mounted displays and graphics workstations 8. On a higher programming level, Silicon Graphics' Performer is used, which allows us to maintain the necessary frame rate to support VR, and OpenGL to render the virtual environment. All the objects, avatars and agents in the world are VRML models, which can be moved and scaled by the child in real time.





Observing interaction in NICE... Evaluation ...

points here:

In order to justify the educational value of the technology, effective evaluation methods need to be established to discover if conceptual learning takes place. This is a difficult challenge, as there is little consensus regarding assessment within the education community.

As VR is a dynamic learning tool, we believe that evaluation must be inextricably coupled with the actual learning process. We therefore use VR as the evaluation medium of the VR learning experience. As seen from the above scenario, Jim, an actual mentor disguised as a virtual character, serves multiple purposes as guide and evaluator: to answer questions, direct action, ask for clarification, prompt for interpretation. We also use the produced narrative for post-experience reflection and conversation with the learners. These forms of assessment embedded in the learning process provides meaningful reflections on learners' skills and knowledge, such as .... [ OUR FINDINGS HERE...]

The case studies with individual or pairs of children showed...blah (basically that every kid is different but what they all have in common is ). The classroom showed ... blah. We are continuing our studies by bringing through the same learners as we are not interested in studying the episodic occurences of knowledge, but the effect of the learning technology over time ..... blah blah

Our developed paradigm attempts to see through the fascination and novelty effect which surrounds this new technology and explore the actual learning experiences it has to offer. In the formative evaluation stage, we carefully included interface and orientation, immersion and presence, affective, pedagogical, and cognitive learning issues, after observation of the following:

It is known that interface problems interfere with the learning experience, as they draw the attention toward the apparatus. During development, we tried to make interactivity as seemless as possible by using tricks in the software : a yellow baloon around objects to provide feedback when they are selected and a virtual representation of the child's hand, to provide a frame of reference for the location and orientation of the hand in the virtual world. The hand is also chosen because it is the best "gardening tool" and it doesn't limit the actions as other tools would. It also applies to other interaction supports we are currently experimenting with, such as data gloves and physical gardening tools as replacements for the wand.

explore the following more in terms of theatrical improvisation, CAVE (VR) as a performance space, role-playing etc.
The childrens' collaborative experiences in the NICE environment have raised a number of interesting research issues, particularly in relation to self-representation and non-verbal communication. The children greatly enjoyed seeing other avatars in the virtual space, but they also expressed the wish to see what they look like. Although they chose or even designed their own avatars, they wanted to visualize the presence of their body in the environment. We have recently modified NICE to include a "reflecting sea" where the users can see what they look like on the surface of the water surrounding the island, and we are exploring mechanisms for allowing users to adopt alternative ("out of body") viewing perspectives within the environment. While the NICE environment supports real-time remote audio, one of the childrens' favorite activities was to exchange flowers and thank each other non-verbally by waving, moving their heads up and down, or bowing. We are also exploring techniques for providing access to the rules of affective behavior. Following the lines of our earlier StoryWriter project, we are investigating methods for allowing children to "act out" social interaction rules to govern the behavior of avatars in their interaction with users and other agents in the environment.

Virtual Reality in the Real World
Do Eddie and Jim really need to enter an expensive virtual environment to learn about ecology or social interaction? Wouldn't it be better if they planted a natural garden as part of a real outdoor classroom activity? Is VR really effective for learning?

While there are no substitutes to planting a natural garden or interacting with other children and teachers in the playground or at school, a VR environment can provide rewarding learning experiences that are otherwise difficult to obtain. As an example, ecological systems are usually complex models with many variables and behaviors that young children have difficulty visualizing. Computers can be effective at reducing the complex models into simpler qualitative relationships (e.g., no water is bad, a moderate amount of water is good, too much water is bad). VR adds the qualities of immersion, direct engagement 6, and exploration to such models. On the virtual island, the children have the ability to scale and position parts of an ecosystem, or accelerate time to observe quickly and directly the effects of their changes. While in a real garden they can learn how to plant, in the virtual garden they can learn how to think about plants, take on different roles, change their own size, talk and interact with children at distant locations.


Figure 6. Children interacting with the NICE world.


MENTION DISTINCTION BETWEEN IMMERSIVE VR AND MUD's & MOO'S, VALUE OF VR vs. MOO HERE
Traditional MUDs are text-based distributed systems, and as such make good learning tools for promoting creative writing, but lack the visual, audible, and haptic interactivity provided by the immersive VR space. The multi-user, theatrical aspects of the CAVE give it an additional advantage to other immersive VR systems (HMD's). Sure one can claim that MOO's are great "verbal theaters", but they require learning a certain "language", unatural conventions to make up for the lack of visual, gestural and audio communication.

The goal for NICE is to combine advantages of both text-based and immersive VR. NICE has a MUD, where Web visitors can interact with the characters in the VR space....


Given the many problems today facing the effective deployment of technology in support of education, it may seem that projects based on largely inaccessible technology are at best redundant. Even if the current novelty of VR technology generates great interest among students, it cannot be an effective tool while students are limited to short, infrequent immersive experiences, and teachers cannot tune the experience to the needs of their particular students. However, although the cost and size of advanced VR technologies, such as the CAVE, prohibits their use in today's classroom, we believe that VR, in various forms, is a medium destined for widespread availability in the coming decades. Broadly interpreted, VR is already making inroads into the home in the form of inexpensive game systems or text-based MUDs and MOOs, and museums and cultural centers are already beginning to acquire multi-user VR systems which allow for many children to participate in the experience simultaneously.

the following needs rewording
(We believe that the development of research-based environments such as NICE, coupled with concrete evidence of their value to learners, is essential to help create demand for inexpensive, widely available virtual reality technologies which schools could justify and afford.)


Ongoing Work in NICE

In this paragraph write about:

Give motivation for doing the above, but not experience with it yet.
"metaconstructivism". (do not use the term, just talk about what we mean)

While NICE is a constructivist environment, in its current form the user is bound by the rules established by the developers. One way to expand the opportunity for learning is through "meta-constructivism": giving users access to the models themselves 13. We are currently developing a modeling capability which allows NICE users to specify the growth model of the garden plants qualitatively, all while staying within the environment. Think of it as a visual programming language within the VR environment. In the simplest single variable model, children define growth as a function of rainfall. The children are presented with an array of seedlings, arranged with an increasing number of clouds overhead. The user "pulls" the seedlings up to various heights to reflect the relationship to the amount of rainfall, thus defining a growth function. This new rule can then be used to replace the default growth rule. Multivariate models can be supported by small two-dimension plots and groups of plots. Learners will then qualitatively specify plant growth rules as a function of rainfall, sunlight, and spacing. In a virtual greenhouse, students can isolate individual controlling variables, "stretch" plants to reflect the growth response to these variables, and observe and measure the effects of their models by accelerating time.

Acknowledgements
We would like to thank the following people who make this work possible: the members of the Electronic Visualization Laboratory and the Interactive Computing Environments laboratory; the educators and children that participate in the evaluation process; the original "Yet Another World" group for their participation in many fruitful discussions. This research is partially supported by NSF grant CDA-9303433 which includes support from ARPA.

References -will be refreshed

  1. C. Cruz-Neira, D.J. Sandin, and T.A. DeFanti, "Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE," Proc. ACM SIGGRAPH, ACM Press, New York, 1993, pp. 135-142.
  2. J. Dewey, Democracy and Education, Free Press, New York, 1966.
  3. I. Harel, Children Designers. Interdisciplinary Constructions for Learning and Knowing Mathematics in a Computer-Rich School, Ablex Publishing Corporation, Norwood, NJ, 1991.
  4. K. Kahn, "ToonTalk - An Animated Programming Environment for Children," Journal of Visual Languages & Computing, Vol. 7, No. 2, 1996, pp.175-196.
  5. B. Laurel, Computers As Theater, Addison-Wesley Publishing Co., 1991.
  6. J. Leigh and A. E. Johnson, "Supporting Transcontinental Collaborative Work in Persistent Virtual Environments," IEEE Computer Graphics and Applications, July 1996, pp.47-51.
  7. J. Leigh et al., "Multi-perspective Collaborative Design in Persistent Networked Virtual Environments," Proc. IEEE Virtual Reality Annual International Symposium, IEEE Computer Society Press, Los Alamitos, CA, 1996, pp.253-260.
  8. D. Pape, "A Hardware-Independent Virtual Reality Development System," IEEE Computer Graphics and Applications, July 1996, pp.44-47.
  9. S. Papert, Mindstorms: Children, Computers, and Powerful Ideas, Basic Books, New York, 1980.
  10. J. Piaget, To Understand is to Invent: The Future of Education, Grossman, New York, 1973.
  11. M. Roussos et al., "Constructing Collaborative Stories Within Virtual Learning Landscapes," Proc. European Conference on AI in Education, Colibri Editions, Lisbon, Portugal, September 1996, pp. 129-135.
  12. M. Roussos et al., "The NICE project: Narrative, Immersive, Constructionist/Collaborative Environments for Learning in Virtual Reality," to appear in Proc. ED-MEDIA, AACE Editions, Calgary, Canada, 1997.
  13. M. Roussos et al., "NICE: Combining Constructionism, Narrative, and Collaboration in a Virtual Learning Environment," SIGGRAPH 1997.
  14. E. Soloway et al., Learning Theory in Practice: Case Studies of Learner-Centered Design, http://www-personal.umich.edu/~spit/Hi-C/DIS.html, 1996.
  15. K.E. Steiner and T. Moher, "Graphic Storywriter: An Interactive Environment for Emergent Story Construction," Proc. CHI, Monterey, CA, 1992, pp. 357-363.
  16. K.E. Steiner and T. Moher, "Scaffolding Story Construction with Interactive Multimedia," Journal of Educational Multimedia and Hypermedia, Vol. 3(2), pp.173-196.
  17. L.S. Vygotsky, Mind in Society: The development of higher psychological processes, Harvard University Press, Cambridge, MA, 1978.
  18. J. Wilson and D. Rosenberg "Rapid Prototyping for User Interface Design," Handbook of Human-Computer Interaction, Helander M., North Holland, pp.859-875.