Figure 1 - a map of all of the sites that were involved in the experiment.
Multiway Tele-Immersion at Supercomputing '97
or
Why We Used $6,000,000 Worth of VR Equipment to do the Hokey Pokey
Andy Johnson, Jason Leigh, Jim Costigan
Electronic Visualization Laboratory
University of Illinois at Chicago
Chicago, IL 60607
ajohnson | jleigh | costigan @eecs.uic.edu
The Alliance consists of more than 50 universities and government institutions led by the National Center for Supercomputing Applications. Supercomputing '97, held in San Jose this year, provided an excellent opportunity to connect several of the Alliance partners together and prototype the kind of shared interaction my colleagues and I will be focusing on in the next few years. Rather than show a tele-immersion demo at Supercomputing, we wanted to use the conference as an opportunity to do research on the show floor.
At previous conferences, such as Supercomputing '95 and SIGGRAPH '96, we shared virtual environments using a handful of CAVEs and Immersadesks from the east coast to the west coast and from the Gulf of Mexico to Chicago. At Supercomputing 97 we wanted to do more, both in terms of the number of participating sites, and the geographic distance between them. In the past we also had a great deal of control over the hardware and software at the various sites, and over the networks that connected the sites together. This time we would be relying on our collaborators.
Setting Up the Experiment:
Several of the Alliance partners currently had limited experience with collaborative VR applications, so we wanted to get them immersed in a shared environment where they could interact with the virtual space and the other users with as little hand-holding as possible. The software had to be easy to set up and run, and it had to work over the partners' existing networking. This would allow us to investigate the following:
The NICE environment consists of an island which the children can explore. In the center of this island is a persistent garden. A simulation program which has been continuously running since the summer of 1996, controls the garden's growth. Children can enter the NICE environment, check on the progress of the garden (simulation), tend the garden (alter the parameters of the simulation), and discuss their work with other children in the space who are represented via avatars. Another reason we chose to use NICE was that it is a `fun' environment that encourages social interaction, and we wanted the partners to have fun during this experiment.
One month before Supercomputing began, we contacted the members of the CAVE Research Network User's Society ( CAVERNUS), a user group of CAVE-based and CAVE-like VR systems, to see who wanted to participate. Since more than 50 CAVEs and Immersadesks exist worldwide, we didn't have any trouble finding collaborators. We gave the NICE application to those participants and encouraged them to run the application, enter the garden, and interact with it. At this point, we found to our surprise that the various sites were setting the viewing parameters of their Immersadesks slightly differently. This meant that even if two collaborators stood at exactly the same place in the virtual space, they would not see exactly the same thing. We also found that the avatars were helped diagnose problems at a remote site - if we looked at an avatar for a remote user and saw that her head was tilted at an unnatural angle or that her body was in a an odd position, then the remote site might have problems with tracking that were not visible locally. This has prompted us to think about prescribing a set of protocols to guarantee proper CAVE/ImmersaDesk calibration.
At Supercomputing:
At the conference, we ran our experiment twice, for one hour each time. On the exhibits floor of the conference we used six Immersadesks and one Immersadesk2. Remote participants included four CAVEs, five Immersadesks, several CAVE simulators running on workstations, and one web-based java interface. The collaborators ranged from coast to coast in the United States and included sites in the Netherlands and Japan. These collaborators also used a variety of Silicon Graphics hardware to run their VR systems. There were Maximum Impact based CAVE simulators, Immersadesks run from Octanes, deskside Onyxs and Onyx 2s, and CAVEs run from rack Onyxs and Onyx 2s.
The full list of collaborators included:
- Virtual Environments Lab at the Center for Coastal Physical Oceanography Old Dominion University Norfolk, VA Immersadesk - Electronic Visualization Laboratory University of Illinois at Chicago Chicago, IL 2 CAVEs Immersadesk simulator - Virtual Reality / Virtual Environments Lab Indiana University Bloomington. IN CAVE - SARA (Stichting Academisch Rekencentrum Amsterdam) Amsterdam, Netherlands CAVE simulator - Cray Research Eagan, MN Immersadesk - The Scientific Computing and Visualization Group's Laboratory for Immersive Virtual Environments Boston University Boston, MA Immersadesk - Towa University Fukuoka, Japan Immersadesk
participants on the exhibit floor at Supercomputing 97 included:
- Army Research Laboratory
Immersadesk
- Laboratory for Advanced Computing
University of Illinois at Chicago
Immersadesk
web based Java client
- Argonne National Laboratory
2 Immersadesks
- National Computational Science Alliance
Immersadesk2
- Department of Computer Science
University of Illinois at Urbana-Champaign
Immersadesk
- Virtual Reality / Virtual Environments Lab
Indiana University
Immersadesk
Figure 1 - a map of all of the sites that were involved in the experiment.
During our experiment, we had a few networking problems - partly this was due to intermittent networking at the conference itself and partly due to running both the server for the garden and the UDP reflector on the same Indy, which also acted as an active web server. We had intentionally placed the server on an active web site since this made it easy to distribute updated models for the garden. We were also curious to see how well a single server would perform under these conditions. From discussions with our partners, it seemed that there would typically be at most a handful of people collaborating in such a world, though there could be more for a distributed presentation within the space. As the number of clients increased, the server became overloaded, but it recovered well and all the client software recovered dynamically to keep the environment going. This kind of fault tolerance meant that we didn't have to continually re-start every client when the network went down. Another problem we encountered was with remote clients who tried to connect through their local firewalls. The other collaborators could see them, but they could not see anyone else.
Within the space itself we were partly a victem of our own success. We had designed several avatars for the original NICE application, and added several more for this experiment, but not enough to give everyone a unique character (more than a few people wanting to be the Dennis Rodman avatar as shown in Figure 2.) The plan was that each site would develop their own VRML-based avatar body parts, but there was not enough time to do this. While each participant could choose a name for their chosen avatar and have that name appear on their character's chest, a certain amount of confusion still existed. We encouraged names such as 'Kukimoto_in_Japan' or 'Jim_at_SC97' so we could not only tell who we were talking to, but how far away they were.
Figure 2 - the popular 'Dennis Rodman' avatar. When we participate in trans-continental or trans-global collaborations from our lab, it helps to have a recognizable Chicago personality as the avatar from Chicago.
Differences between the various hardware platforms quickly became noticeable as the number of users increased beyond 10. The Slower SGI workstations (ie anything less than an Onyx) could not keep up with the rendering. This was a larger hinderance to collaboration than network lag, though the guaranteed real-time audio link kept the situation from getting too chaotic.
We were interested in whether people would be sociable in the space, and whether they could coordinate their efforts and work together. Part of the demo was devoted to allowing various sub-groups to form and explore parts of the island (as in a VR MUD). Another part focused on 'centralized' activity such as clearing the garden of weeds.
Since the audio link was a real-time link between all of the sites, this allowed us to look at the lag in the transmission of avatar position data. In order to get a sense of this within the environment we gathered most of the collaborators together and did the Hokey-Pokey. The hokey-pokey is a rather silly American dance where the instructions and lyrics follow the general pattern of "You put your right hand in, you take your right hand out, you put your right hand in and you shake it all about, you do the Hokey Pokey and you turn yourself around, that's what it's all about". Since our avatars have a head, a body, and one arm, we modified the dance slightly, but you could get a very good idea what each user wss doing (at least when the server wasn't overloaded). So with everyone listening and several people singing the instructions with minimal lag, it allowed us to see how much time it took for the avatars took to go through the motions.
Figures 3 and 4 show several photos taken during the multi-way demo, both live shots taken on the floor of Supersomputing '97 and shots taken by a virtual camera within the virtual space.
Figure 3 - photos taken by Tom Coffin at Supercomputing during the collaboration
Figure 4 - snapshots taken within the virtual space during the collaboration. Clockwise from top left: SARA in the Netherlands offering a flower to one of the participants in Chicago, the Army Research Lab claiming the high ground, a small gathering in the catecombs, and a crowd in the garden.
We found that it was possible to quickly field a collaborative VR environment over existing networks, though we need to work on ensuring that all of our collaborators have properly calibrated equipment. We also observed that the participants were very sociable and they could coordinate action within the space. We found that building in automatic recovery into the server and clients was very important, that the server really should reside on a decent machine, and that guaranteed quality of service is a necessity for the next generation of these kinds of VEs. And we discovered that our collaborators had a lot of fun participating in our experiment.
We believe that it is important to build these collaborative worlds and let domain scientists, and educators play with them to really see where we should focus our efforts. This experiment showed us that a lot of stuff works, and there is a lot that can be improved upon. The venue for those improvements will be CAVERNsoft [2,3] - the hybrid realtime networking & persistent datastore library we are currently developing to better support data distribution in Tele-Immersion. CAVERNsoft uses NEXUS from Argonne National Labs for its networking support and uses PTOOL from the Laboratory for Advanced Computing at the University of Illinois at Chicago for its database support. There are also many interesting communications issues that are brought up by such collaborative spaces, and we are continuing to investigate those issues as well.
So what's in store for us at Supercomputing 98? Our experiment allowed us to look at logistical issues, both outside and inside the virtual space. Supercomputing '98 may be a good place to look more in depth at the communication issues within a couple specific collaborative visualization and design spaces. Plus we still have several more continents to connect to ... and then there's Mir.
Acknowledgements:
We would like to thank the following people for helping to make our experiment possible at the various sites:
Joe Alexander, Rick Angelini, Stuart Bailey, Lilac Berniker, Edward Berrdveld, Hisham Bizri, Glenn Bresnahan, Erik Brisson, Maxine Brown, Russell Burgett, Tom Coffin, Junna Cui, Kathleen Curry, Marek Czernuszenko, Fred Dech, Tom DeFanti, Ralph De Stefano, Margaret Dolinsky, Ian Foster, Tom Frisch, Sara Graffunder, Karen Green, Bob Grossman, John Hicks, Tomoko Imai, Joe Insley, Wilfred Janssen, Ted Jordan, Hank Kaczmarski, Abhinav Kapoor, Kelly Kirk, Cal Kirchhof, Anton Koning, Dick LaRoche, Cathy Lascara, Johnny Lawson, Gary Lindahl, Nobuyuki Kukimoto, John Melchi, Bill Nickless, Dave Pape, Mike Papka, Kyoung Park, Dana Plepys, Robert Putnam, Larry Ramey, Dan Reed, Paul Rossman, Nancy Rowe, Bill Sherman, Shuji Suma, Larry Smarr, Craig Stewart, Rick Stevens, Matt Szymanski, Samroeng Thongrong, Jeroen van Hoof, Rene van der Moolen, Luc Verburgh, Alan Verlo, Fang Wang, Eric Wernert, and Glenn Wheless.
References:
[1] Johnson, A., Roussos, M., Leigh, J., Barnes, C., Vasilakis, C., Moher, T., "The NICE Project: Learning Together in a Virtual World,." In the proceedings of VRAIS '98, Atlanta, Georgia, Mar 14-18, 1998, Pp. 176-183.
[3] Leigh, J., Johnson, A., DeFanti, T., "CAVERN: A Distributed Architecture for Supporting Scalable Persistence and Interoperability in Collaborative Virtual Environments." In Virtual Reality: Research, Development and Applications, Vol 2.2, December 1997, Pp. 217-237.
[3] Leigh, J., Johnson, A., DeFanti, T., "Issues in the Design of a Flexible Distributed Architecture for Supporting Persistence and Interoperability in Collaborative Virtual Environments,." In the proceedings of Supercomputing '97 San Jose, California, Nov 15-21, 1997.
Related Web Pages:
Tele-Immersion at EVL: http://www.evl.uic.edu/spiff/ti/
NICE: http://www.ice.eecs.uic.edu/~nice
CAVERNUS: http://www.ncsa.uiuc.edu/VR/cavernus/
NEXUS: http://www.mcs.anl.gov/nexus/
PTool: http://www.lac.uic.edu/