Abstract. CAVERNsoft is an architecture for creating Tele-Immersive applications, with the goal of making synchronous and asynchronous trans-oceanic collaboration a routine matter. This paper briefly discusses CAVERNsoft and then discusses several applications that have been built using CAVERNsoft with a focus on those with trans-oceanic concepts.

1. Introduction

Tele-Immersion (TI) is the integration of collaborative virtual reality with audio and video conferencing in the context of data-mining and significant computation. When participants are Tele-Immersed, they are able to see and interact with each other in a shared virtual environment. This environment may be a new car design, an environmental visualization, or an educational space. The participants may be represented as articulated computer-generated avatars or live-feed video avatars. When the collaborators are within the same city or within the same continent it is straightforward to hold synchronous sessions in the shared space since time-zone differences are within a few hours. When the collaborators are spread across the planet this becomes much harder. Asynchronous collaboration, where the collaborators work in the same virtual space at different times, allows everyone to work in the shared space during their own normal workday. It can be used to enhance productivity by allowing a collaborator to hand off his work at the end of his day to someone who is just starting her day. There are however many cultural and language issues that must be dealt with if these spaces are to be effective supporters of collaboration.

Our focus is on supporting both synchronous and asynchronous collaboration over trans-oceanic distances. Our model is that of a persistent virtual world where the virtual environment is sustained by a computer simulation that is left constantly running. This world continues to exist and evolve even when there are no participants – it may autonomously control supercomputing computations, query databases, or gather the results for visualization when the participants return.

Our users are the members of CAVERN - the CAVE Research Network. CAVERN is a collection of participating industrial and research institutions equipped with CAVEs, Immersadesks, and high-performance computing resources. High-speed networks connect them to support Tele-Immersive engineering and design; education and training; scientific visualization; and computational steering. With over 80 CAVE and ImmersaDesk installations around the world, one of the most important problems facing this growing community, and the VR community as a whole, is how best to provide a mechanism to support long term collaborative work. Our goal is to make these systems convenient, so scientists and designers can do real work within these shared environments without worrying about how the collaboration is sustained. CAVERNsoft [1,2] is our architecture for achieving the goals of Tele-Immersion.

This paper will briefly describe the CAVERNsoft architecture and then describe several tele-immersive applications developed using CAVERNsoft with a focus on those with trans-oceanic capabilities.

2. CAVERNsoft

In light of the complex interaction of computer graphics, networking, databases, and human factors in Tele-Immersion, developing these applications can be a daunting task. Many approaches and systems have been proposed and developed in the past. Kessler presents an excellent summary of the more recent work in this area [3]. The temptation and common mistake, made by application developers faced with building tele-immersive applications for the first time, is to initially build a non-collaborative application and then attempt to retro-fit it for Tele-Immersive capabilities.

It is important to provide tools that encourage application developers to envision Tele-Immersive scenarios early on and at a high-level so that they can determine how such capabilities would be most useful in their own applications. A high-level set of tools, however, does not help those trying to retrofit existing applications. A high-level library of well-integrated tools often assumes a specific software design that may be incompatible with the software that is being retrofitted.

To address this issue, we are currently developing a software infrastructure called CAVERNsoft that supports the rapid creation of new Tele-Immersive applications and eases the retrofitting of previously non-collaborative VR applications with Tele-Immersive capabilities. The primary difference between CAVERNsoft and previous approaches is its focus on integrating collaborative VR with supercomputers and terra-byte data stores, that are connected over high speed nation-wide (vBNS, MREN) and world-wide (STAR TAP) networks. This affords us the ability to explore new applications of this collaborative technology without being hindered by the limits of the existing Internet. CAVERNsoft is also independent from the CAVE’s graphical library so it can support heterogeneous VR devices and even non-graphical applications. CAVERNsoft was initially developed on Silicon Graphics hardware and a Sun Solaris version is currently under development.

CAVERNsoft, diagrammed in Figure 1, consists of a central structure called the Information Resource Broker (IRB) surrounded by layers of support software. Although these layers appear to increasingly hide the lower layers from the main application, they are accessible at every level. The lower levels facilitate the construction of new components, as well as the retrofitting of existing applications. The higher levels facilitate the rapid development of new Tele-Immersive applications.

The IRB (inspired by CORBA ORBs but with lower overhead) is a relatively low-level merging of networking and database capabilities that is completely separate from graphics. Hence the basic IRB core can be placed in any software application regardless of its graphics capabilities. This permits graphical applications to communicate with non-graphical applications and it also allows existing non-collaborative applications to achieve networking capabilities with minimal disturbance to their existing graphics.

The IRB is spawned as a lightweight thread, with both integrated networking and database capabilities, so that it can serve both client and server needs. Any application using an IRB is automatically a client and a server at the same time. As a client, the application may connect to other servers or clients to access their information resources. As a server, the application may accept connections from other clients. This dual capability is transparent to the application developer but allows the construction of a wide variety of distributed topologies. Its symmetry allows distributed applications to treat one another as information resources.

IRB-based applications communicate with each other by establishing single or multiple communications channels. Each channel is individually customized to meet the specific needs of the Tele-Immersive data being transmitted. The application may negotiate with the IRB interface for the desired networking protocol (reliable TCP, unreliable UDP, and unreliable multicast) and the desired bandwidth, latency, and jitter. These QoS capabilities will be provided by Nexus, a multi-threaded communications library developed by Argonne National Laboratory [4] as part of their larger Globus project [5].

Besides providing customized communications channels between IRBs, another key feature that distinguishes the IRB from traditional notions of distributed shared memory is that each arena can be made persistent. A database interface layer provides the persistent capabilities of the IRB. This layer will eventually allow the applications to negotiate the QoS of the underlying database system. That is, an arena may choose to have safe updates or less safe rapid updates; or, it may choose to relax database integrity requirements to achieve speed.

At a layer above the IRB, non-graphical template libraries support the coordination of avatars, as well as audio and video data compression algorithms. On top of this is a higher-level layer that consists of graphical versions of the previous layer, like OpenGL, Performer, and video avatar templates. These higher level templates can be gathered into even higher-level, fully functional generic Tele-Immersion spaces called LIMBO spaces.

LIMBO spaces provide varying degrees of avatar rendering and recording; model importing, distribution, manipulation and version control; and audio/video teleconferencing. Using a generic LIMBO space, collaborators can work in a virtual space immediately. They can start with an avatar of their choosing and import 3D models (e.g., car designs, scientific data sets). LIMBO can visualize and share models in any of the currently popular 3D file formats such as OpenInventor, VRML1, DXF, and OBJ. This allows external applications to generate visualizations and deposit them inside LIMBO for collaborative visualization. The LIMBO space will ensure proper distribution of the model to all remote participants and once objects are distributed, participants may collectively modify them. The LIMBO space is persistent so the space and its objects will remain extant. Allowing domain experts to quickly see their own data in a collaborative virtual space gives them ideas for how to customize that space for their own needs. The LIMBO source code is provided to jump-start the development of these domain-specific applications.

As more domain-specific applications are developed, a growing library of CAVERNsoft-based reusable components will emerge. These can be added to the library of existing templates and collected to build DOMAIN spaces that are specializations of LIMBO spaces. For example, this will enable a designer to build a Tele-Immersive design application starting with an existing DOMAIN space equipped with collaborative tools specifically for collaborative design, rather than starting from the basic LIMBO space.

3. CAVERNsoft based Tele-Immersive Applications

In the year since CAVERNsoft was first introduced, over a dozen CAVERNsoft applications have been built in several domains: scientific visualization, data-mining, industrial design and manufacturing, youth education, training, anthropology and art. CAVERNsoft's success has been mainly the result of its variety of adopters who have helped motivate its continual evolution.

Some developers have chosen to retrofit CAVERNsoft onto their existing applications to give them Tele-Immersive capabilities, or enhance their existing Tele-Immersive capabilities.

Tele-Immersive VisualEyes. General Motors developed VisualEyes, which allows designers to import 3D CAD models into the CAVE for quick visual inspection and design reviews. CAVERNsoft was used to extend VisualEyes to allow GM's trans-globally-situated research and design teams to collaborate in synchronous and asynchronous remote design reviews.

Virtual Temporal Bone. The Virtual Reality in Medicine Lab at UIC has created a Tele-Immersive application allowing a physician to teach medical students the 3D structure and function of the inner ear. This application originally used the CAVE library’s limited networking to coordinate the collaboration. To support this installed base, the CAVE2CAVERNsoft library was created to allow an easy one-to-one replacement of the CAVE's networking calls with those in CAVERNsoft giving these applications increased capabilities.

CAVE6D. The Center for Coastal Physical Oceanography at Old Dominion University is developing CAVE6D for Tele-Immersive visualization of environmental data. CAVE5D is supported by Vis5D, a powerful graphics library providing techniques to display multidimensional numerical data from atmospheric, oceanographic, and other similar models. CAVE6D allows multiple CAVE5D users to share a common data set and interact with each other. Participants can now simultaneously view different sets of parameters in the visualization to customize and/or reduce the ‘cluttered-ness’ of the visualization. Time is globally shared so all participants view the time-varying data synchronously.

Virtual Reality Telecollaborative Integrated Manufacturing Environment. Searle and the Industrial Virtual Reality Institute at UIC are creating VRTIME for facilities design, regulatory review and operational review. Tele-Immersion is necessary here as these reviews involve geographically separated participants: architects, engineers, consultants, reviewers, Federal, state and local officials, and environmental interest groups.

Collaborative Battlefield Monitoring. The Naval Research Laboratory is using CAVERNsoft to allow users to create and manage shared objects.

Others have built new environments on top of LIMBO.

Collaborative Impact Analysis. The Institute of High Performance Computing (IHPC) in Singapore has used the SingAREN STAR TAP high-speed link to collaboratively visualize a finite element crash-test of a cellular phone. Engineers at Motorola feel that this kind of visualization in the CAVE is more natural than using a workstation. Tele-Immersion allows engineers in Singapore and Chicago to observe how dropping a phone will impact its internal circuitry.

Data-Mining Visualization. The National Center for Data Mining at UIC is using LIMBO to collaboratively visualize the decision trees generated by their data mining algorithms. Visualizations of the raw data can also be imported so that the raw data and the trees modeling them can be viewed side by side.

Silk Road CAVE Shrines. Northwestern University and EVL are using LIMBO to create a 3D exhibit of the caves within the Mongoa Grottos of Dunhuang in the Gobi Desert. The models of the caves are imported into LIMBO as a quick collaborative evaluation and brainstorming tool.

Future Camp 1998. The Indiana University Purdue University Indianapolis used LIMBO in their Future Camp - a one-week multidisciplinary VR day camp for eighteen 9-11 year olds. 3D objects were created in Alias Wavefront and loaded into LIMBO worlds where the campers could then position the objects in the space to create several environments.

Others are extending LIMBO to add new features.

LIMBO/VTK. The National Center for Supercomputing Applications is merging LIMBO with the Visualization Toolkit allowing developers to use the visualization tools built into VTK to generate sharable three-dimensional objects in LIMBO.

Virtual Whiteboard. Tohwa University in Japan is extending LIMBO by providing a shared writing slate within the collaborative environment that is carried along with the participant as he/she traverses the virtual space. This slate can then be used to sketch diagrams or write kanji using the CAVE’s wand giving users an additional mode of communication.

CAVE Collaborative Console. Virginia Tech is extending LIMBO by providing a generic set of collaboration tools such as radar to tell you where the other participants are, the ability to teleport around the shared space, and primitive 3D drawing tools.

4. Supporting Trans-Oceanic Collaboration

CAVERNsoft is also being used to develop tools and evaluate performance in trans-oceanic Tele-Immersion.

EVL is developing CAVERNsoft-based virtual-mail to provide an effective asynchronous collaboration tool for Tele-Immersion. Asynchronous collaboration is attractive because it reduces traveling cost and frees people from tight meeting schedules, however there is a lack of immediate feedback. In international collaborations, it often takes a day to receive a response and collaborators can easily waste days clarifying questions, plans, and instructions. Ambiguity should be minimized as much as possible so that the collaborators can concentrate on their tasks.

When a Tele-Immersed participant sends a V-mail message, both audio and gesture from the collaborator are captured in addition to the surrounding scenery. When the message is played back, as in Figure 2, the avatar of the original sender of the message materializes to re-enact the message in the space as though they were talking ‘live.’ Participants are able to see the sender’s avatar naturally gesturing while describing a change to be made or a feature that has been discovered in the virtual environment. This is particularly effective when the concept being described has intrinsically spatial properties. Since the message is communicated aurally and gesturally instead of textually, it isn’t limited by specific character sets. The additional gestures also seem to help non-native speakers understand the message better than transmitting audio alone, and the message can be replayed multiple times if necessary to aid in understanding. Furthermore, threads of V-Mail can be woven together to serve as avatar-based tutorials in VR environments.

EVL is using CAVERNsoft to study how collaborative coordination is affected by differing levels of network latency and jitter, so that in the future we can predict the kinds of tasks that will be possible over networks with particular characteristics. While high bandwidth will be important for moving audio and video streams across the networks, and low latency is required, our experiments show that low jitter may be most important. Initial experiments have been performed within a local-area network and over a trans-oceanic high-speed network (STAR TAP/SingAREN) between EVL in Chicago, and IHPC in Singapore. The available TCP bandwidth measured 1.4 Mbits/s. UDP bandwidth measured 8Mbits/s. Qualitative and quantitative experiments were performed [6]. These experiments showed 250ms round trip latency, similar to ISDN quality, with lower jitter than over local area ISDN. The audio was sent via a long-distance phone call, our standard procedure, and the latency in the audio seemed to roughly match the latency in the movements of the collaborative participants. This is highly encouraging as it suggests that high performance networks will allow effective trans-oceanic collaboration. CAVERNsoft is designed to take advantage of Quality of Service (QoS) systems when they become available so these continuing experiments will help quantify the requirements for QoS in trans-oceanic networks.

5. Ongoing Work

Research in CAVERNsoft is progressing along three fronts:

Firstly we are experimenting with two networking QoS schemes: Qualis (developed by ISI and ANL) and AMInet (developed by Nippon Telephone and Telegraph and the University of Tokyo.)

Secondly we are working with the database experts at the National Center for Data Mining to replace CAVERNsoft's existing database layer with one that will allow applications to negotiate database access policies.

Finally, we are continuing to work with our partners to develop new CAVERNsoft applications, as we believe the best way to learn how to do Tele-Immersion is by creating and evaluating applications in multiple domains. As multiple trans-oceanic Tele-Immersion sites come on line we will deploy an experimental suite to isolate bottlenecks in the network. The problems are not merely technological however. Being able to hear is not sufficient for understanding, and being able to communicate is not sufficient for collaboration. There are many interesting psychological, social, and cultural issues yet to be investigated.

Additional details about CAVERNsoft's capabilities and implementation, as well as a copy of CAVERNsoft, LIMBO, and CAVE2CAVERNsoft, can be downloaded from EVL’s Tele-Immersion web site at http://www.evl.uic.edu/cavern. The CAVE library can be acquired by contacting http://www.vrco.com.

Acknowledgements

The virtual reality research, collaborations, and outreach programs at EVL are made possible through major funding from the National Science Foundation, the Defense Advanced Research Projects Agency, and the US Department of Energy; specifically NSF awards CDA-9303433, CDA-9512272, NCR-9712283, CDA-9720351, and the NSF ASC Partnerships for Advanced Computational Infrastructure program. The CAVE and ImmersaDesk are trademarks of the Board of Trustees of the University of Illinois.

References