Paper at the 1st International Symposium on Multimedia Virtual Laboratory. Tokyo, Japan, March 25, 1998
Andrew E. Johnson, Jason Leigh, Thomas A. DeFanti,
Maxine D. Brown, and Daniel J. Sandin
Electronic Visualization Laboratory
University of Illinois at Chicago, Chicago, IL 60607, USA
(312) 996-3002 voice, (312) 413-7585 fax
ajohnson@uic.edu www.evl.uic.edu
CAVERN, the CAVE Research Network, is an alliance of industrial and
research institutions equipped with CAVEs, Immersadesks, and high
performance computing resources, interconnected by high-speed networks
to support collaboration in design, training, education, scientific
visualization, and computational steering, in virtual
reality. Supported by advanced networking on both the national and
international level, the CAVE research network is focusing on
Tele-Immersion - the union of networked virtual reality and video in
the context of significant computing and data mining.
The CAVE (tm) [3] as shown in Figure 1, which premiered at the SIGGRAPH '92 conference in Chicago, had its motivation rooted in scientific visualization - and the challenge became one of attracting leading-edge computational scientists and engineers to adopt it. Virtual reality (VR), then a fledgling technology, had to help scientists make scientific discoveries faster without compromising the color, resolution, and flicker-free qualities they had come to expect using workstations. Scientists had been doing single-screen stereo graphics for more than 25 years; any VR system had to successfully compete. Most importantly, the VR display had to couple remote data sources, supercomputers, and scientific instrumentation in a functional way. In total, the VR system had to offer a significant advantage to offset its packaging and cost.
In 1992, the goals of the Electronic Visualization Laboratory (EVL) were to provide an infrastructure for computer scientists to work with computational scientists to collect, maintain, develop, distribute, and evaluate VR tools and techniques for scientific computing. These included computer-based models, simulators, data libraries, programming libraries, and user interfaces. EVL also wanted to provide researchers with access to the best VR equipment to explore their data in highly immersive, highly interactive environments.
EVL partnered with the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and the Mathematics and Computer Science Division of Argonne National Laboratory (Argonne) in a multi-year, ongoing effort to develop highly leveraged national collaborations at major professional conferences, notably SIGGRAPH and Supercomputing [4,5]. These events emphasized high performance computing and communication technologies, VR, and scientific visualization. The overall purpose was to encourage the development of teams, tools, hardware, system software, and human interface models on an accelerated schedule to enable national-scale, multi-site collaborations applied to National Challenge and Grand Challenge problems.
The number of companies making projection-based VR displays has risen from none in 1992, to at least six in 1997 , so now EVL turns its attention to the design and implementation of CAVERN, the CAVE Research Network. CAVERN will be comprised of dozens of network-connected CAVEs, ImmersaDesks, other VR devices such as Head-Mounted Displays and Responsive Workbenches, and large computational resources. To support the scientists, engineers, and artists who are members of CAVERN, the CAVERN Users' Society (CAVERNUS) was formed (www.ncsa.uiuc.edu/VR/caverus). This group is now collaborating in investigating Tele-Immersion.
The term Tele-Immersion was first used in October 1996 as the title of a workshop organized by the Electronic Visualization Laboratory at University of Illinois at Chicago (UIC) to bring together researchers in distributed computing, collaboration, virtual reality, and networking. Workshop attendees paid specific attention to the future needs of applications in the sciences, engineering, and education. EVL defines Tele-Immersion as the union of networked VR and video in the context of significant computing and data mining. Some researchers use the terms Collaborative Virtual Environment (CVE) or Distributed Virtual Environment (DVE) to describe the field of networked VR [1]. Tele-Immersion, as defined by the authors, encompasses more image-based technology, like video and bit maps, than CVE/DVE researchers typically envision. It also has the benefit of being pronounceable.
In the past two years, progress has been made toward linking CAVEs and ImmersaDesks (tm) with networks, from ISDN to ATM. CALVIN [9], a project first tested cross-country at Supercomputing '95, focused on collaborative architectural design in VR . It led to NICE [13] (Narrative, Immersive, Constructionist/Collaborative Environments), which linked sixteen CAVE and Immersadesk users on three continents (North America, Europe and Asia) at Supercomputing '97. This multi-continental multi-user experiment, shown in Figure 2, prototyped the kind of connectivity we are currently investigating. NICE has in turn led to a design for CAVERNsoft[10,11], the distributed memory software architecture for CAVERN.
The NICE project, as shown in Figure 3, is a distributed collaborative virtual environment for K-6 students. It uses the concept of a continuously running simulation, in this case a garden, and allows children and teachers to interact with each other in planning and discussing the garden's evolution over weeks and months. NICE can automatically generate Web-based printable reports, in essence a narrative transcription, that documents the interactions. Students can check the garden's progress with a normal workstation if they are away from the VR devices, and desktop users can interact with VR users through a java applet. NICE incorporates many features that scientists desire in a Tele-Immersive intelligent data mining application, but in a sufficiently limited context to facilitate formal evaluation.
NICE is an example of a Persistent Tele-Immersive world. Persistent Tele-Immersion is defined as collaborative computer-generated worlds that remain extant (i.e., continue to exist and evolve) even when there are no human participants. The ``world'' is a computer generated representation of a computer simulation.
The concept of a substantial simulation running continuously that adult scientists can enter and explore alone or in collaborative sessions is very appealing. EVL aims to lower the barriers to this kind of computer usage by extending the NICE approach to test domains where significant computation and data mining are needed. EVL intends to provide the computation required by adopting Condor, software from the University of Wisconsin, that allows unused processors to be allocated automatically to background jobs [6]. At EVL, this means dozens of Onyx processors that are idle when the CAVEs are not being used. This will allow codes to be run that can benefit from computing, disk, and memory on the order of 20 CRAY Y-MPs operating nightly. EVL hopes scientists subsequently adapt these techniques on computing and visualization resources they have available locally or at supercomputing centers. Such intensive use of CAVE time is difficult to accommodate on EVL's research and development CAVE, which evolves rather often, so in the past year, the University of Illinois at Chicago (UIC) built a second CAVE for applications development and testing.
EVL is currently working with CAVERNUS members to develop Tele-Immersive applications in several areas including collaborative design review with General Motors, collaborative medical education with the Virtual Reality in Medicine Laboratory in UIC's School of Biomedical and Health Information Sciences [12], and collaborative scientific visualization with the Center for Coastal Physical Oceanography at Old Dominion University [15]. These projects are illustrated in Figure 4.
Other CAVERN-related projects include NCSA's work with Caterpillar and the German National Research Center for Information Technology using a transatlantic ATM connection for collaborative design in VR[8], and work to make the virtual director [2] a collaborative tool for real-time recording of analysis and design sessions in VR. We are also working with the National Center for Data Mining and Data Intensive Computing to allow collaborative visualization and analysis of mined data.
EVL is interested in investigating the technical issues involved in the creation of these environments, the social and communication issues of collaborating through this medium, and performing initial evaluations of their effectiveness. Our goal is not simply to make this kind of collaboration `possible', but to make it `convenient.'
Tele-Immersion has entered the Next Generation Internet (NGI) (www.ngi.gov) and Internet2 (www.Internet2.edu) vocabulary. In the applications section of the Computing Research Association's ``Research Challenges for the Next Generation Internet,'' five key enabling technologies were identified as common to the future use of the NGI [14]: Database Access, Audio and Video, Real-Time and Delayed Collaboration, Distributed Computing, and Tele-Immersion.
The networking research community considers Tele-Immersion to be an ``application driver'' to help fine-tune networks for large datasets and/or multimedia, where latency and bandwidth are of major importance.
UIC has participated in building the following networks:
EVL has been very fortunate that many of the early adapters of CAVE and ImmersaDesk technologies, introduced to it by responding to national Calls for Participation as part of SIGGRAPH and Supercomputing conference events organized by EVL, wanted to continue their collaborations afterwards. These computational scientists and engineers were gaining experience with VR technologies and wanted to turn anecdotal testimonies of their benefits into publishable scientific discoveries. Subsequently, to handle the needs of these ``friends,'' EVL started what has become a modestly successful commercialization effort.
There are CAVEs and ImmersaDesks being used worldwide, as detailed in Figure 5. Most buyers expressed interest in either continuing joint research projects or in starting new ones. This presents a wonderful opportunity to solve real-world problems, not only locally at UIC, but internationally, where other problems, notably networking latency and bandwidth considerations, become important.
The requirements to support persistent Tele-Immersion may be viewed at three levels: the end user level, the application developer level and the core technology developer level.
End user issues concern the high-level support of communication, collaborative interaction/manipulation, persistence/recording, and connectivity:
Application developer issues revolve mainly around the integration of software tools into applications that meet end-user needs. In the past, we used diverse technologies to build new collaborative applications (such as CALVIN and NICE), which proved highly successful but the underlying toolsets had limited reuse. From these experiences, however, needed software patterns began to emerge. These include:
Figure 6: Patterns for networking and databases for a variety of Tele-Immersion data
Core development issues. Application programmers should not have to worry about core technology development; that is, the software and hardware infrastructure to support the expected patterns of resource access. Figure 6 summarizes the patterns for networking and databases for a variety of Tele-Immersion data:
Each of these data categories is further grouped into three access categories (real-time operation, intermittent snapshots, and continuous recording) that reflect the differing database requirements needed to support them. For example, real-time avatar data does not require database operations, snapshots of avatar data require safe writes, and continuous recordings require rapid updates even if it means a reduction in database integrity.
The general conclusion that can be derived from Figure 6 is that persistent Tele-Immersion consumes networking and database resources with a number of real-time demands for potentially long periods of time. Patterns for graphics access are not included in the Table, as 30 frames per second is the constant desired frame rate. However, even with the fastest graphics processors currently available (such as the SGI Onyx2 Reality Monster) only a fraction of the data typically present in scientific or engineering datasets can be visualized in real time. Filtering of the data is needed to create meaningful abstractions that can be rendered in near real time.
In light of the complex interaction of computer graphics, networking, databases, and human factors in Tele-Immersion, developing these applications can be a daunting task. The temptation and common mistake, made by application developers building Tele-Immersive applications for the first time, is to first build a non-collaborative application and then attempt to retro-fit it for Tele-Immersive capabilities.
It is important to provide tools that encourage application developers to envision Tele-Immersive scenarios at a high-level so that they can determine how such capabilities would be most useful in their own applications. A high-level set of tools, however, does not help those trying to retrofit existing applications. A high-level library of well-integrated tools often assumes a specific software design that may be incompatible with the software that is being retrofitted.
To address this issue, a software infrastructure called CAVERNsoft that supports the rapid creation of new Tele-Immersive applications and eases the retrofitting of previously non-collaborative VR applications with Tele-Immersive capabilities, is currently under development.
CAVERNsoft, diagrammed in Figure 7, consists of a central structure called the Information Resource Broker (IRB) surrounded by layers of support software. Although these layers appear to increasingly hide the lower layers from the main application, they are accessible at every level. The lower levels facilitate the construction of new components, as well as the retrofitting of existing applications. The higher levels facilitate the rapid development of new Tele-Immersive applications.
The IRB is a relatively low-level merging of networking and database capabilities that is completely separate from graphics. Hence the basic IRB core can be placed in any software application regardless of its graphics capabilities. This permits graphical applications to communicate with non-graphical applications and it also allows existing non-collaborative applications to achieve networking capabilities with minimal disturbance to their existing graphics.
At a layer above the IRB, non-graphical template libraries support the coordination of avatars, as well as audio and video data compression algorithms. On top of this is a higher-level layer that consists of graphical versions of the previous layer, like OpenGL, Performer, and Video avatar templates. These higher level templates can be gathered into even higher-level, fully functional Tele-Immersion spaces called LIMBO spaces.
LIMBO spaces provide varying degrees of avatar rendering and recording; model importing, distribution, manipulation and version control; and audio/video teleconferencing. By using a basic LIMBO space, collaborators can work in a virtual space immediately. They may start with an avatar of their choosing and import 3D models (e.g., car designs, scientific datasets). The LIMBO space will ensure proper distribution of the model to all remote participants. Once objects are distributed, participants may collectively modify them. In addition, application developers may use the well-documented source code of LIMBO space to jump start the development of their own domain-specific Tele-Immersive applications.
As more domain-specific applications are developed, a growing library of CAVERNsoft-based reusable componentswill emerge. These can be added to the library of existing templates and collected to build DOMAIN spaces that are specializations of LIMBO spaces. For example, this will enable a designer to build a Tele-Immersive design application starting with an existing DOMAIN space equipped with collaborative tools specifically for collaborative design, rather than starting from the basic LIMBO space.
The IRB is a small, highly configurable nucleus that can be imbedded into every distributed software system that is expected to interoperate. That is, every client or server application will possess an IRB that it will use to communicate with other compatible applications.
The IRB is spawned as a lightweight thread, with both integrated networking and database capabilities, so that it can serve both client and server needs. Any application using an IRB is automatically a client and a server at the same time. As a client, the application may connect to other servers or clients to access their information resources. As a server, the application may accept connections from other clients. This dual capability is transparent to the application developer but allows the construction of a wide variety of distributed topologies. Its symmetry allows distributed applications to treat each other as information resources.
IRB-based applications communicate with each other by establishing single or multiple communications channels. Each channel is individually customized to meet the specific needs of the Tele-Immersive data being transmitted. The application may negotiate with the IRB interface for the desired networking protocol (reliable TCP, unreliable UDP, unreliable multicast) and the desired bandwidth, latency, and jitter. These QoS capabilities will be provided by Nexus, a multi-threaded communications library developed by Argonne National Laboratory [7].
After communications channels have been established, each application may create an arena of data that can be linked to one another so as to emulate a distributed shared memory (DSM) segment between the connected applications. Using the same channel or different channels, many arenas can be created, each of which can be used to store different forms of data (avatar tracking, 3D models, etc.) As a default capability, any time one arena is updated, any other applications that have arenas linked to the modified arena will also see their arenas updated. This default coherence protocol can be altered so that updates occur only when requested. A locking mechanism is provided to support entry consistency.
Besides providing customized communications channels between IRBs, another key feature that distinguishes the IRB from traditional notions of DSM is that each arena can be made persistent; i.e., the IRB may commit an arena to a persistent store. This is useful for Tele-Immersive applications; e.g., the arena could hold a 3D model that was downloaded from a remote application, and rather than downloading the model each time the application is restarted, the model could be retrieved from the persistent arena. Since each commit of an arena is marked with a time stamp, data modified by a remote application prior to a re-start would be downloaded once the local and remote arenas were re-linked.
The persistent capabilities of the IRB are provided by a database interface layer. This layer will eventually allow the applications to negotiate the QoS of the underlying database system. That is, an arena may choose to have safe updates or less safe rapid updates; or, it may choose to relax database integrity requirements to achieve speed. For example, in order to record a continuous Tele-Immersive session, rapid updates at the expense of losing database integrity are needed, whereas for storing static snapshots, performance takes second priority over database integrity. Given the desired ``database QoS,'' the database interface will either assign the arena to be managed by the database system best able to meet the requirements of the arena, or re-negotiate with the client-application for a slightly lower QoS.
Since CAVERNsoft does not depend on any part of the CAVE library, it can easily be run on VR devices that may not be running the CAVE library.
This current alpha release version of the IRB can be downloaded from www.evl.uic.edu/spiff/cavernsoft.
Additional background information can be found on EVL's Tele-Immersion pages at www.evl.uic.edu/spiff/ti.
EVL develops software tools and hardware technologies with the end user in mind - notably, the computational scientist and engineer. EVL's goal is to make its VR displays fully functioning ``discovery'' resources for the computational science and engineering community. Research activities, in addition to technology development, include: quantitative assessment, human-centered intelligent interfaces, improved real-time imagery, fast-access methods to very large databases, distributed computing, real-time navigation and successive refinement with large-scale computational models, and high-definition VR.
Now that commercial vendors sell CAVEs and ImmersaDesks, as well as similar projection-based VR displays, EVL's has shifted its attention from technology development to Tele-Immersion. The CAVE research network will develop, deploy and evaluate tele-immersive applications in a variety of areas, allowing CAVE and Immersadesk users, as well as users of other VR hardware, to effectively and conveniently collaborate over large distances.
EVL's VR research, collaborations, and outreach programs would not be possible without major funding from the National Science Foundation (NSF), the Defense Advanced Research Projects Agency (DARPA), and the U.S. Department of Energy.
This material is based upon work supported by NSF under grants CDA-9303433, CDA-9512272, ASC-9418068, NCR-9712283, CDA-9720351, and the ASC Partnerships for an Advanced Computational Infrastructure (PACI) program.
The CAVE and ImmersaDesk are trademarks of the Board of Trustees of the University of Illinois.
CAVERN: The CAVE Research Network
This document was generated using the LaTeX2HTML translator Version 96.1 (Feb 5, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -no_navigation -show_section_numbers mvl.tex.
The translation was initiated by Andy Johnson on Tue Feb 17 16:23:23 CST 1998