In this paper, we present a multi-rail design as a promising approach towards achieving Terabit/s performance for advanced E-Science applications. 

We evaluate the effects of system parameters such as interrupt, memory, thread and core affinities on the achievable network throughput as we scale the performance towards Terabit/s. 

We demonstrate the additive effects of Thread and Interrupt affinities for achieving 20Gbps line rate throughput. 

These results can help guide the development of future intelligent middleware that will automatically optimize the performance based on system parameters and conditions, thus making it easier for applications developers, who are often not systems experts, to make full use of the system capabilities.

Venkat Vishwanath presenting the poster at SC07

Towards Terabit/s Systems: Performance Evaluation of Multi-Rail Systems

This thesis presents the motivations, inspirations, design and architecture of an ecology learning environment for middle school science classrooms. The application, WallCology, places a virtual ecosystem within the walls of a classroom, drawing from ubiquitous computing concepts. The system is designed to give learners easy access to affordances for the study of ecological phenomena as ecologists would in the field. Using a set of computers as portals into the virtual ecosystem, students study creatures with different morphologies and behaviors moving about on pipes and walls. These portals, called WallScopes, are linked together using a remote server which allows the creatures to move within the larger virtual space based on their own environmental preferences. A two month pilot study of WallCology in an urban middle school classroom comprised of species differentiation and population estimation units demonstrated the feasibility of this approach. 

The interactions of Earth&rsquo;s inhabitants have given rise to a wide variety of ecological principles and methods of study and many are not readily accessible for study in elementary school classrooms. As teachers strive to produce scientifically meaningful activities which promote ecology learning for their students, they are often required to work within the space of their classroom because of the logistical and financial constraints involved in taking a group of students into the field. Often teachers bring flora and fauna into the classroom for study but the inherent lack of control can make study difficult. Additionally the removal of such organisms from their natural habitat inhibits the learning of &ldquo;in the field&rdquo; ecology principles. WallCology seeks to provide a platform for studying many ecological principles by placing a controlled virtual ecosystem that is readily accessible for everyday use within the confines of elementary science classrooms. 

Ecology in the field is widely observational and seeks to limit the impact by scientists on natural habitats. Additionally, ecologists can only view a sample of the overall system at any one time, thus science requires meticulous data collection as well as collaboration with other observers to gain an overall perspective. These are principles that are difficult to teach with textbooks and can be better taught though practice. The computers of WallCology offer only small samples of the overall space at observational &ldquo;sites&rdquo; and thus correlate directly to these principles. 

WallCology is built on the embedded phenomenon framework (Moher, 2006) and extends many of its ideas. Previous embedded phenomena have mapped scientific phenomena to the interior of a classroom. WallCology modifies this idea slightly by mapping the phenomenon of an ecosystem to the unseen space between the walls of a classroom. This small distinction in setup lends well to the incorporation of multiple classrooms. Past embedded phenomena have been largely task based and directed at specific curricular goals. WallCology was initially conceived as a platform for teaching many ecological principles and its architecture has been designed for extension and code reuse. The initial implementation, which aimed at species differentiation and mobile population estimation, can be extended to include food webs, life cycles, evolution, and other ecological principles increasing the duration from weeks to months and possibly years of study by elementary learners.

WallScope interface running on a tablet PC grants access to WallCology&rsquo;s virtual ecosystem

WallCology: A Learner&rsquo;s Interface to a Persistent Virtual Ecosystem

This thesis presents a methodology for rendering very large volume data on scalable high-resolution displays using a distributed-memory cluster. 

The methodology uses a multi-resolution octree, an image-order data distribution method, a distributed shared-memory data management system, a multi-level cache, and hardware accelerated rendering techniques to produce a solution that is scalable in terms of input size and output resolution. 

An analytical cost model validated by experimental results predicts the system&rsquo;s behavior. The methodology&rsquo;s usefulness is demonstrated with a number of domain specific datasets. 

The primary contributions of this thesis include: 
<ol>
<li>A review of research in the field of volume rendering and parallel volume rendering.</li>
<li>A methodology for rendering very large volume data on scalable high-resolution displays using a commodity distributed-memory cluster of computers that scales with the size of input data and the output resolution.</li>
<li>An analytical model validated by experimental results that predicts the methodology&rsquo;s behavior.</li>
<li>The application of this methodology to domain specific problems in the fields of bioscience, geoscience and medicine.</li>
</ol>

Researcher analyzing a Purkinje neuron on the LambdaTable

Distributed Volume Rendering of Very Large Data on High-Resolution Scalable Displays

This paper describes the development of an artistic visualization project that restores the architectural history of the world famous wood churches and architectural museum, Kizhi. 

Kizhi is the architectural complex located on an island in Lake Onega in northern Karelia, Russia. The Church of Transfiguration, the Church of Intersession and the Belfry form the central part of the complex. 

The churches were built in harmony with the surrounding landscape and contain a priceless collection of icons and relics of Russian heritage. This historically important site is included in the List of Most Endangered Sites of the World Monuments Watch protected by World Heritage List of UNESCO. 

This project aims to restore the original architecture of Kizhi museum and to visualize the architectural changes of the museum area over the last 300 years. The visualization is being developed based on high resolution photographs, video surveys, textural data obtained during the initial visit of the site and resources from the Kizhi State Open-Air Museum archives. 

The project strives to capitalize on the advances of computer graphics technology and stereoscopic high-resolution display technology to advance the development of historical restoration in an artistic direction.

Figure 8. The 3-D model of the Belfry. Perspective view

Passing Excellence, The Interactive Art Visualization Of the Kizhi Ensemble

The Sensimmer platform represents our ongoing research on simultaneous haptics and graphics rendering of 3D models. For simulation of medical and surgical procedures using Sensimmer, 3D models must be obtained from medical imaging data, such as Magnetic Resonance Imaging (MRI) or Computed Tomography (CT). Image segmentation techniques are used to determine the anatomies of interest from the images. 3D models are obtained from segmentation and their triangle reduction is required for graphics and haptics rendering. This paper focuses on creating an integrated interface between Sensimmer and medical imaging devices, using available software. Existing tools are evaluated, as well as aspects that require further development are identified. Solutions to overcome limitations and increase the degree of automation of the process are examined.

Segmentation of Skull in ITK-SNAP

Automating the Extraction of 3D Models from Medical Images for Virtual Reality and Haptic Simulations

Most haptic libraries allow user to feel the resistance of a flexible virtual object by the implementation of a point-based collision detection algorithm and a spring-damper model. Even though the user can feel the deformation at the contact point, the graphics library renders a rigid geometry, causing a conflict of senses in the user&rsquo;s mind. 

In most cases, the CPU utilization is maximized to achieve the required 1-kHz haptic frame rate without leaving any additional resource to also deform the geometry, while on the other hand, the Graphics Processing Unit (GPU) is underutilized. 

This paper proposes a computationally inexpensive and efficient GPU-based methodology to significantly enhance user perception of large existing haptic applications without compromising the original haptic feedback. To the best of our knowledge, this is the first implemented algorithm that is able to maintain a graphics frame rate of approximately 60 Hz as well as a haptics frame rate of 1 Khz when deforming complex geometry of approximately 160K vertices. 

The implementation of the algorithm in a virtual reality neurosurgical simulator has been successful to handle, in real time, complex 3D isosurfaces created from medical MRI and CT images.

Virtual human brain being deformed by the insertion of a neurosurgical catheter

GPU-based elastic-object deformation for enhancement of existing haptic applications

This paper proposes GMPLS-capable Lambda-on-Demand to adjust the number of load-balanced end-to-end lambdas according to the traffic volume while successfully sharing lambda resources to connect between any two of three nodes for the first time.

Lambda Sharing Demonstration via Traffic-Driven Lambda-on-Demand

The depth and relief present in stereoscopic art are sculptural qualities similar to those found in dioramas. Unlike traditional sculpture, projected stereoscopy does not adapt to the viewer&rsquo;s movement or position without the additional aid of external systems, such as motion tracking. However, the phenomena of depth and relief in stereoscopic digital media are powerful and important tools for artists to explore and utilize. This paper details the methodology and research involved in the creation of a digital stereoscopic video composited with stereoscopic computer graphics, entitled The Far Dark Shore. Created with Maya 3d modeling / animation software and a high-definition (HD) stereoscopic camera apparatus, The Far Dark Shore is the product of a year-long cycle of research concerning the implications of digital media attaining the power and relief of sculpture. With the intent to present the piece in a gallery setting, research has also been centered on the exploration of the space that exists in stereoscopy - the space behind the screen, the space existing at the screen plane, and the space between the screen plane and the viewer. The Far Dark Shore features a dancer portraying a young woman encountering her mortality through her relationship with orchid blossoms.

Detail 1

The Far Dark Shore: Sculptural Depth and Relief in Digital Stereoscopic Video and Composited Stereoscopic Computer Graphics

Object. The purpose of this study was to evaluate the accuracy of ventriculostomy catheter placement on a head- and hand-tracked high-resolution and high-performance virtual reality and haptic technology workstation. 

Methods. Seventy-eight fellows and residents performed simulated ventriculostomy catheter placement on an ImmersiveTouch system. The virtual catheter was placed into a virtual patient&rsquo;s head derived from a computed tomography data set. Participants were allowed one attempt each. The distance from the tip of the catheter to the Monro foramen was measured. 

Results. The mean distance (6 standard deviation) from the final position of the catheter tip to the Monro foramen was 16.09 mm (6 7.85 mm). 

Conclusions. The accuracy of virtual ventriculostomy catheter placement achieved by participants using the simulator is comparable to the accuracy reported in a recent retrospective evaluation of free-hand ventriculostomy placements in which the mean distance from the catheter tip to the Monro foramen was 16 mm (6 9.6 mm). 

(DOI: 10.3171/JNS-07/09/0515) 

Keywords: Haptic technology, neurosurgical simulation, ventriculostomy, virtual reality

Photo of ImmersiveTouch system in operation

Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback

Researchers in the Solar Physics Branch at Naval Research Laboratory (NRL) have been involved in observational and theoretical studies of the solar atmosphere since the early years of the space age. They are viewing impressive solar disturbances whose depth and violent nature are now clearly visible in the first true stereoscopic images ever captured of the Sun. These new views, recently released by NASA, are providing scientists with unprecedented insight into solar physics and the violent solar weather events that can bombard Earth&rsquo;s magnetosphere with particles and affect systems ranging from weather to our electrical grids. 

The Electronic Visualization Laboratory (EVL) has helped establish a solar imagery display environment, two EVL-developed display systems capable of viewing and managing files on the scale of thousands of pixels per square inch: a 9-panel tiled LCD wall ideal for viewing high-resolution 2D imagery, and an ImmersaDesk4 for viewing high-resolution 3D imagery. The tiled display consists of 9 cinema LCD displays with an overall resolution of approximately 8k by 5k pixels and will be used to work with multiple HD quality video feeds, enabling multi-spectral analysis of synchronized image streams from sensors on each satellite. The ImmersaDesk4, invented at the University of Illinois at Chicago (UIC) Electronic Visualization Laboratory (EVL), is a tracked, 4-million-pixel display system driven by a 64-bit graphics workstation. Its compact workstation design is comprised of two 30-inch Apple LCD monitors mounted with quarter-wave plates and bisected by a half-silvered mirror enable circular polarization. It permits scientists to visualize the stereo imagery in high resolution 3D. Multiple users can view the head-tracked 3D scene using lightweight polarized glasses. 

Solar physicists at NRL hope that this new technology will allow them to better visualize the three dimensional structure of coronal mass ejections and evaluate the impact these events have on the earth&rsquo;s atmosphere.

Solar Imagery Displayed on EVL&rsquo;s LambdaTable

Development of Display Environment for LASCO, EIT and SECCHI

Optically virtual concatenated parallel lambdas over multiple-domains that are provided using reservation-messaging between network resource managers for a high-end parallel visualizing application are experimentally shown for the first time in the Chicago metro area network. 

High-end visualizing applications are targeting parallelism using tiled displays and cluster computers to ensure unlimited scalability. Inevitably there will be a need for load-balanced parallel lambdas to carry their traffic. Then, these lambdas possibly traverse multiple network domains as needed. On the other hand, such applications are quite sensitive to latency deviations among these lambdas. There fore, we have proposed Optical Virtual Concatenation (OVC) to de-skew these lambdas. There have been no reports of the demonstration of load-balanced &amp; OVC-enabled parallel lambdas over multiple-domains. 

A network resource manager (NRM) with the capability to accept reservation requests of lambdas, to compute available routes, and to provide lambdas over a photonic network domain comprising Photonic Cross Connects (PXCs) in response to reservation requests form those applications will be a key device in the lambdaGrid network. It has the capability to dynamically configure parallel lambdas in collaboration with other network devices. 

In this paper, we present the first filed trial of the interworking of NRMs and OVC over two photonic netowrk domains against advanced reservation requests for netowrk resources form a visualization middleware, with is the Scalable Adaptive Graphic Environment (SAGE) developed by the Electronic Visualization Laboratory (EVL).

Experimental Configuration

The First Optically-Virtual-Concatenated Lambdas over Multiple Domains in Chicago Metro Area Network Achieved Through Interworking of Network Resource Managers

OBJECTIVE: Mastery of the neurosurgical skill set involves many hours of supervised intraoperative training. Convergence of political, economic, and social forces has limited neurosurgical resident operative exposure. There is need to develop realistic neurosurgical simulations that reproduce the operative experience, unrestricted by time and patient safety constraints. Computer-based, virtual reality platforms offer just such a possibility. The combination of virtual reality with dynamic, three-dimensional stereoscopic visualization, and haptic feedback technologies makes realistic procedural simulation possible. Most neurosurgical procedures can be conceptualized and segmented into critical task components, which can be simulated independently or in conjunction with other modules to recreate the experience of a complex neurosurgical procedure. 

METHODS: We use the ImmersiveTouch (ImmersiveTouch, Inc., Chicago, IL) virtual reality platform, developed at the University of Illinois at Chicago, to simulate the task of ventriculostomy catheter placement as a proof-of-concept. Computed tomographic data are used to create a virtual anatomic volume. 

RESULTS: Haptic feedback offers simulated resistance and relaxation with passage of a virtual three-dimensional ventriculostomy catheter through the brain parenchyma into the ventricle. A dynamic three-dimensional graphical interface renders changing visual perspective as the user&rsquo;s head moves. The simulation platform was found to have realistic visual, tactile, and handling characteristics, as assessed by neurosurgical faculty, residents, and medical students. 

CONCLUSION: We have developed a realistic, haptics-based virtual reality simulator for neurosurgical education. Our first module recreates a critical component of the ventriculostomy placement task. This approach to task simulation can be assembled in a modular manner to reproduce entire neurosurgical procedures. 

Copyright &copy; by the Congress of Neurological Surgeons

Collocation of the virtual volumes (virtual patient head and catheter) with real-space is critical to the platform&rsquo;s realistic virtual reality

Virtual Reality in Neurosurgical Education: Part-task Ventriculostomy Simulation with Dynamic Visual and Haptic Feedback

Nanoscience and engineering are the wave of the future, and nanoscience education is key to developing the next generation of innovative developers. However many K-12 students have difficulty relating to science that they cannot directly observe. 

Virtual reality based simulations are helpful for exploring these sorts of phenomena, however they must be firmly grounded back to a frame of reference with which students are familiar. 

This paper presents a progression of learning which uses pixels on a computer screen as a unifying theme to lead students from physical models to microscopic exploration and finally to nanoscale exploration in a virtual world.

A Conceptual Rendition of a Student Exploring a Virtual Nanoscopic Environment

A Learning Progression to Effectively Implement Virtual Reality as an Educational Tool for K-12 Nanoscience Education

Low-cost computing, displays and networking is making it possible to create collaboration environments that are truly able to bridge distance. 

This paper describes EVL&rsquo;s work since 1992 that has pursued the goal of creating collaboration environments that have been able to seamless merge high-resolution 2D information with 3D information incorporating a variety of viewing and interaction modalities.

The Next-Generation CAVE

Emerging from the CAVE: Collaboration in Ultra High Resolution Environments

Higher resolution content is collected or generated everyday by domain scientists, such as remote sensing imagery, large scale simulations and computer graphics animations. In the meantime, high-resolution devices are becoming extremely affordable: high-definition (1920 by 1080 pixels) displays and cameras are available for $1000, a very small fraction of their costs a few years ago. Super-high-resolution (SHD) is already being deployed from manufacturers&rsquo; laboratories to the professional market (Sony 4k SXRD projector, Sharp 4k LCD panel, Red and Dalsa 4k cameras, 4k film scanners, &hellip;). 

Since today&rsquo;s science relies heavily on collaborative exploration involving distributed groups of researchers, collaborative environments must allow the interactive sharing and analysis of such large content. To achieve interactive speed, realtime compression techniques play a key role. Even in this day of exponential increase of networking bandwidth (i.e. deployment of multi-10Gbps networks), compression is required to alleviate the next bottlenecks in modern computer systems: storage speed and memory bandwidth. 

In this paper, we introduce the use of software DXT compression for high-resolution content at an interactive speed. DXT compression allows to stream full HD video over a gigabit connection where multiple gigabits were required. Moreover, it enables 4K streaming without the need for an high-end storage system or an expensive codec [7] (dedicated hardware system for encoding and decoding frames). This work will enable the AccessGrid community to stream video and visualizations at much higher resolutions while minimizing bandwidth requirements. 

The contributions of this paper are as follows: 
<ul>
<li>Compression of HD and SHD in real-time using a commodity PC at a ratio of 6:1, with a good quality</li>
<li>A low bandwidth requirement for high-resolution content</li>
<li>The integration of DXT compression in the streaming environment SAGE</li>
</ul>

Real-Time Compression For High-Resolution Content

Multi-rail approach for terabit/s system

We present a high-fidelity monitoring infrastructure that enables real-time analysis and self-adaptation at both the systems and applications level in virtual computing environments. We believe that such an infrastructure is needed as each paradigm shift (in this case to virtual computing environments) brings new challenges along with new capabilities. 

Monitoring the performance and health of large-scale computing environments can be a daunting challenge. Without appropriate tools, tracking down problems, whether caused by hardware or software, is akin to finnding a &ldquo;needle in a haystack.&rdquo; 

In a clustered computing environment, monitoring tools such as Ganglia and Supermon have emerged and rapidly matured to address this problem in a scalable manner. However, these tools sacrifice information to achieve scalability, e.g., they use the /proc filesystem as their data source. 

Consequently, such tools are sample-based and can only provide measurements at the granularity of an OS time slice. Furthermore, these tools are designed to operate in the context of a physical environment rather than a virtual one. Therefore, as an alternative, we present a dynamic, high fidelity, event-based infrastructure for both physical and virtual computing environments called MAGNET.

High-Fidelity Monitoring in Virtual Computing Environments

Jinghua Ge&rsquo;s PH.D. Dissertation - Submitted as partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Chicago, 2007. 

State-of-the-art Virtual Reality technologies such as Varrier&trade; bring better interaction and comprehension into visualization experience. But VR applications are still limited in the area of large-scale scientific visualization mostly because of the intensive graphics computation for VR viewing. 

The goal of this thesis is to design and implement a distributed visualization framework which combines VR technologies and remote computing resources through a high speed network, so that large-scale scientific datasets can be visualized in real-time on local VR devices. 

The framework is designed to be a scalable distributed system with pipelined data retrieval, computation, and visualization for various datasets. Scalability makes the system adaptive to the available computing and visualizing resource configurations. Each subsystem of the distributed system can perform either cluster-based parallel computing or single workstation-based sequential computing. The pipeline configuration can be optimized based on a balanced granularity as the ratio of computation to communication. The pipeline is an MIMD design which explores computing and networking parallelism along with the data flow.

Point patch (L) and Point Cluster (R) representation of a point packing

A Point-Based Remote Visualization Pipeline For Large-Scale Virtual Reality

As scalable high-resolution displays have become increasingly prevalent and networking is diminishing in cost at a rate exceeding that of computing and storage [Stix01], EVL envisions situation-rooms and research laboratories in which all the walls are made from seamless ultra-high-resolution displays fed by data streamed over ultra-high-speed networks from distantly located visualization, storage servers, and high definition video cameras [Leigh03, Smarr03]. To help realize this vision, my PhD thesis has focused on the Scalable Adaptive Graphics Environment (SAGE) [Jeong06], a scalable graphics architecture for supporting collaborative scientific visualization environments with hundreds of megapixels of continuous display resolution fed by tens to hundreds of gigabits of network bandwidth. SAGE enables high-performance real-time pixel streaming from remote rendering clusters to scalable high-resolution tiled-display systems. For example, SAGE has streamed ultra-high-resolution image data and video at a rate of 9Gb/s over a 10Gb/s private network on National LambdaRail between San Diego and Chicago, and pushed the data onto LambdaVision, a hundred-megapixel tiled display. 

This graphics architecture addresses two non-trivial problems in scientific visualization. One is heterogeneity: since most visualization applications are closely tied to their graphics environment, it is difficult to integrate various visualization applications into a unified graphics environment. The other is scalability: the ability of visualization software and systems to scale in terms of the amount of data they can visualize and the resolution of the desired visualization [Blanke00]. SAGE addresses the heterogeneity problem by decoupling graphics rendering from graphics display so that visualization applications developed on various environments can easily migrate into SAGE by streaming their pixels into the virtual frame buffer. Also, SAGE provides scalability by supporting any number of rendering and displaying nodes, number of tiles, and screen resolution. 

SAGE provides users with a full multitasking environment on a tiled display by enabling them to concurrently run multiple visualization applications freely repositioning and resizing the application windows. Every window movement or resize operation requires dynamic and non-trivial reconfiguration of the involved pixel streams. I have experimented with two stream reconfiguration approaches: central reconfiguration and distributed reconfiguration. In the first approach, a central SAGE controller generates all necessary control information and sends it to all other SAGE components to reconfigure them. The second approach performs much faster than the first approach by generating control information on each sender and streaming the information together with pixel data to reconfigure receivers. The second approach performs fast enough to support real-time animation of application windows (20 to 60 moves/resizes per second). 

SAGE is the graphics middleware of the OptIPuter [Leigh03, Smarr03], a major NSF-funded initiative to design advanced cyberinfrastructure for data-intensive science using optical networks, and has been used by international OptIPuter partners as well as numerous universities. To support distance collaboration among these international SAGE users in scalable high-resolution display environments, I extended SAGE to stream pixel data to multiple collaboration endpoints. This significantly increases the complexity of the pixel routing problem because the endpoints can have a variety of tiled display configurations and window layouts. 

Visualcasting is a new SAGE network service to address this problem. It supports global collaboration by enabling two or more users to share application content, sending multi-gigabit streams as required, enabling two or more remote users to interact with one another, as well as with their data, at the same time. Connected, participating endpoint sites form a virtual laboratory, as Visualcasting enables everyone to see the same content at the same time. 

Endpoints can be of any size and configuration, varying from a single high-resolution monitor to room-sized tiled display walls. Each site maintains control of the layout (position, size, overlapping) of its displays. The technology currently used to broadcast information to two or more sites is called multicast [Deering91], and while effective, today&rsquo;s networking infrastructure does not automatically support it, and requires network engineering to implement. Unlike multicast, which requires network engineering, Visualcasting is application-centric. 

Visualcasting addresses the problem by enabling applications to select what information to send, and to whom to send it, without the need for technical support and network modifications. Each endpoint runs SAGE to send and/or receive real-time streams of ultra-high-resolution 2D and 3D content. Visualcasting manages the streams so the same content is made available to all participants. Visualcasting is supported by a bridging system that receives pixel streams from SAGE applications and replicates these streams to each endpoint. The bridge nodes are strategically placed in core network facilities and allocated to service distant collaborators. 

SAGE is similar to IBM&rsquo;s SGE and Deep Computing Visualization (DCV), which support high-resolution visualization on a local scalable display system. DCV supports Remote Visual Networking (RVN) but it is targeted to low-bandwidth networks (10/100 networks) and a single desktop display. In contrast, SAGE Visualcasting supports collaborative visualization among distantly located scalable display environments using graphics streaming protocols specialized for high-performance wide-area networks with high-round-trip-time and multi-ten gigabits of network bandwidth. Furthermore, the SAGE architecture is so flexible that new network protocols can be easily applied to SAGE graphics streaming protocols. 

References 

[Blanke00] Blanke, W., Bajaj, C., Fussell, D., and Zhang, X., &ldquo;The Metabuffer: a Scalable Multiresolution Multidisplay 3-D Graphics System using Commodity Rendering Engines.&rdquo; Tr2000-16, University of Texas at Austin, February 2000. 

[Deering91] Deering, S. E., &ldquo;Multicast Routing in a Datagram Internetwork,&rdquo; PhD thesis, Stanford University, December 1991. 

[Jeong06] Jeong, B., Renambot, L., Jagodic, R., Singh, R., Aguilera, J., Johnson, A., and Leigh, J., &ldquo;High-Performance Dynamic Graphics Streaming for Scalable Adaptive Graphics Environment,&rdquo; ACM / IEEE Supercomputing 2006. 

[Leigh03] Leigh, J., Renambot, L., DeFanti, T. A., et al, &ldquo;An Experimental OptIPuter Architecture for Data-Intensive Collaborative Visualization&rdquo;, Third Workshop on Advanced Collaborative Environments, Seattle, WA, June 2003. 

[Smarr03] Smarr, L., Chien, A. A., DeFanti, T., Leigh, J., and Papadopoulos, P. M., &ldquo;The OptIPuter&rdquo; Communications of the ACM, Volume 46, Issue 11, November 2003, pp. 58-67. 

[Stix01] Stix, G., &ldquo;The Triumph of the Light,&rdquo; Scientific American, January 2001.

Byungil Jeong

Collaborative Visualization Architecture in Scalable Adaptive Graphics Environment

Tom Peterka&rsquo;s PH.D. Dissertation - Submitted as partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Chicago, 2007. 

Parallax barrier strip autostereoscopic (AS) virtual reality (VR) displays have been constructed by placing a barrier of lines in front of an LCD display device. Together with a head tracking system and associated software to interleave portions of left and right eye images in appropriate positions on the display device, an AS effect is achieved that can be quite compelling. In the past, the barrier was a static component that imposes certain imitations on the system. The barrier is manufactured to produce various system parameters such as number of viewers, type of autostereo system, size of working area. For example, in a static barrier head-tracked system the number of viewers is invariably one because of these static barrier limitations. 

Substituting a dynamic parallax barrier for the static barrier frees the system from such constraints. Not only can barrier parameters be easily adjusted, but the barrier can be disabled entirely, converting the AS system to a 2D display. View distance range is expanded; moreover optimum optical conditions are maintained throughout the range. The inherent sensitivity to latency in a head-tracked parallax barrier system is mitigated by translating the dynamic barrier image in response to rapid head movements. Finally, by dynamically controlling the parallax barrier in real time, more than two eye channels are produced, for example to accommodate two viewers and provide each viewer with an independent pair of stereo perspectives. 

A prototype system called Dynallax is researched and developed to demonstrate the qualities of a dynamic parallax barrier for AS. This is a small single-panel prototype based on a stacked dual-LCD display that is scalable by tiling. Besides investigating Dynallax on a purchased stacked display, improvements in dual stacked LCD construction are investigated by constructing a custom display from dissimilar components. Finally, results are shared with the scientific community through publication in a distinguished virtual reality conference.

Experimental apparatus for capturing stereo images

Dynallax: Dynamic Parallax Barrier Autostereoscopic Display

Over the past three years, we have defined and begun to explore the affordances of a design framework that we call Embedded Phenomena, intended to support such &ldquo;patient,&rdquo; extended classroom investigations. 

The design space of the embedded phenomena framework is characterized by four common elements: 
<ul>
<li>Simulated dynamic scientific phenomena are &ldquo;mapped onto,&rdquo; and asserted to occur within, the physical space of the classroom.</li>
<li>The phenomena are represented through distributed media (conventional classroom computers) located around the classroom representing &ldquo;portals&rdquo; into that phenomenon, depicting local state information and control affordances corresponding to the mapping between the room and the phenomenon. (Embedded phenomena are &ldquo;delivered&rdquo; using our &ldquo;Phenomenon Server,&rdquo; a web portal that supports configuration and scheduling of the simulation for delivery to any Internet-connected computer running a conventional web browser with a Flash plug-in.)</li>
<li>The simulations are persistent, running and being presented continuously over extended time periods, concurrently but asynchronously with respect to the regular instructional flow.</li><li>
As individuals, in small groups, and as whole classes, students monitor and manipulate the state of the simulation through those media, gathering evidence to solve a problem or answer a question.</li>
</ul> 

The framework &ldquo;embeds&rdquo; learning in both space and time. Learners are spatially embedded by situating their activity (and their representations of accumulating evidence) physically within the phenomenon under investigation. 

Learners are temporally embedded within the time course determined by the simulation of the phenomenon itself, rather than the time course of classroom instruction. This &ldquo;here and now&rdquo; approach embodied by the embedded phenomena framework stands in contrast to the &ldquo;there and then&rdquo; nature of much of traditional science instruction. 

The central conjecture underlying this work is that this dual embedding, within the context of appropriate instructional designs, can have beneficial impacts on cognitive and affective learner outcomes. 

We have designed and implemented three illustrative systems and deployed them in over a dozen elementary school classrooms within the context of instructional units on seismology, insect ecology, and astronomy. 

In a series of empirical studies, we have presented preliminary support for the conjecture that the embedded phenomena framework can positively impact individual learner outcomes in the development of skill in science practice, conceptual learning, and the development of stances as legitimized investigators. 

In the discussion that follows, we will briefly describe the outcomes of those studies. But the primary goal of this paper is to introduce the concept of embedded phenomena to the AERA community and to make the case for the embedded phenomena framework as a paradigm worthy of further exploration.

Using sticky dots on large posters, students maintain historical representations of bug populations found in their regions

Classroom Space, Classroom Time, and the Representation of Dynamic Phenomena

The use of virtual environments (VE) for many research and commercial purposes relies on its ability to generate environments that faithfully reproduce the physical world. However, due to its limitations the VE can have a number of flaws that adversely affect its use and believability. One of the more important aspects of this problem is whether the size of an object in the VE is perceived as it would be in the physical world. One of the fundamental phenomena for correct size is size-constancy, that is, an object is perceived to be the same size regardless of its distance from the observer. This is in spite of the fact that the retinal size of the object shrinks with increasing distance from the observer. 

We examined size-constancy in the CAVE and found that size-constancy is a strong and dominant perception in our subject population when the test object is accompanied by surrounding environmental objects. Furthermore, size-constancy changes to a visual angle performance (i.e., object size changed with distance from the subject) when these surrounding objects are removed from the scene. As previously described for the physical world, our results suggest that it is necessary to provide surrounding objects to aid in the determination of an object&rsquo;s depth and to elicit size-constancy in VE. These results are discussed regarding their implications for viewing objects in projection-based VE and the environments that play a role in the perception of object size in the CAVE.

Size-Constancy in the CAVE

In this paper, the effects of three visual factors: scene complexity, stereovision and motion parallax on correct perception of a virtual object&rsquo;s size were analyzed in an immersive virtual environment. 

We designed a controlled experiments set to incorporate visual conditions that reflected all twelve different configuration combinations of the three visual factors. Under each visual condition, subject performed the task of making judgments of the sizes of a virtual object displayed at five different distances from him / her. A total number of eighteen subjects participated in our study. The subjects&rsquo; judgments and the corresponding actual sizes of the virtual object were recorded. 

Based on the colleted data, two quantitative measures of subjects&rsquo; performance were derived and analyzed. The results of our experiments were consistent across the majority of the subject population and suggested that scene complexity and stereovision could have significant impact on the performance of a user of virtual environments to make correct judgments on a virtual object&rsquo;s size. On the contrary, motion parallax, either produced by the virtual environment or by the observer, might not be a significant factor in determining that performance.

Virtual Environment Setup

The Effects of Scene Complexity, Stereovision, and Motion Parallax on Size Constancy in a Virtual Environment

Tracking technology for Virtual Reality (VR) applications typically requires the user to wear head-mounted sensors with transmitters or wires. This paper describes a video-based, realtime, low-latency, high-precision 3D face tracker specifically designed for VR displays that requires no sensors, markers, transmitters, or wires to be worn. One center camera finds the 2D face position using Artificial Neural Networks (NN) and recognizes and tracks upright, tilted, frontal and non-frontal faces within visually cluttered environments. Two more left and right (L/R) cameras obtain the 3D head coordinates using a standard stereopsis technique. This paper presents a novel idea to train the NN to a new face in less than two minutes, and includes background training to avoid recognition of false positives. The system utilizes diffuse infrared (IR) illumination to avoid computing time-consuming image normalization, to reduce illumination variations caused by the physical surroundings, and to prevent visible illumination from interfering with the user&rsquo;s field of view. 

CR Categories: I.5.1 [Computer Methodologies]: Pattern Recognition-Models-Neural nets; I.4.8 [Image Processing and Computer Vision]: Scene Analysis - Tracking; I.3.7 [Computer Graphics]: 3D Graphics and Realism - virtual reality; 

Keywords: Head, face, or camera-based tracking; Kohonen neural networks; autostereoscopic 3D display; virtual reality

3 Head Tracker Cameras, IR Illuminators &amp; Filters

Real-Time Neural Network-based Face Tracker for VR Displays

A novel barrier strip autostereoscopic (AS) display is demonstrated using a solid-state dynamic parallax barrier. A dynamic barrier mitigates restrictions inherent in static barrier systems such as fixed view distance range, slow response to head movements, and fixed stereo operating mode. By dynamically varying barrier parameters in real time, viewers may move closer to the display and move faster laterally than with a static barrier system. Furthermore, users can switch between 3D and 2D modes by disabling the barrier. 

Dynallax is head-tracked, directing view channels to positions in space reported by a tracking system in real time. Such head-tracked parallax barrier systems have traditionally supported only a single viewer, but by varying the barrier period to eliminate conflicts between viewers, Dynallax presents four independent eye channels when two viewers are present. 

Each viewer receives an independent pair of left and right eye perspective views based on their position in 3D space. The display device is constructed using a dual-stacked LCD monitor where a dynamic barrier is rendered on the front display and the rear display produces a modulated VR scene composed of two or four channels. A small-scale head-tracked prototype VR system is demonstrated. Performance data are analyzed while advantages, disadvantages, ongoing and future work are identified. 

CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism - virtual reality 

Keywords: Autostereoscopic display, 3D display, virtual reality, Dynallax, Varrier, parallax barrier

The Varrier AS VR system is shown in the smaller desktop version on the Personal Varrier system

Dynallax: Solid State Dynamic Parallax Barrier Autostereoscopic VR Display

Autostereoscopic displays enable unencumbered immersive virtual reality, but at a significant computational expense. This expense impacts the feasibility of autostereo displays in highperformance real-time interactive applications. A new autostereo rendering algorithm named Autostereo Combiner addresses this problem using the programmable vertex and fragment pipelines of modern graphics processing units (GPUs). 

This algorithm is applied to the Varrier, a large-scale, head-tracked, parallax barrier autostereo virtual reality platform. In this capacity, the Combiner algorithm has shown performance gains of 4x over traditional parallax barrier rendering algorithms. It has enabled highperformance rendering at sub-pixel scales, affording a 2x increase in resolution and showing a 1.4x improvement in visual acuity.

Interleaved images displayed on parallax barrier (left) and lenticular (right) autostereoscopic displays

A GPU Sub-pixel Algorithm for Autostereoscopic Virtual Reality

Although Image-Plane techniques have been associated with user fatigue and selection ambiguity problems, they can act as a bridging strategy between 2D desktop and 3D immersive environments. We introduce a framework that facilitates such transitions by using Image-Plane techniques with a virtual cursor visible only in the dominant eye on 2D interfaces below the hand of the user. 

We have developed a proof of concept virtual world building application for the Ygdrasil authoring platform that demonstrates our framework. We propose that this framework facilitates the authoring and utilization of 3D user interfaces in general by leveraging existing computer skills, reinforcing the interface in the 2D environment and unifying the development effort for both platforms.

Image Plane as a Bridging Strategy

A Single-Authoring and Bridging Framework for Virtual and Augmented Reality Interfaces

SAGE Display Controller, a web-based application designed and developed to access and control the SAGE environment. The application allows a novice user to interact with the environment from standard web browsers such as Firefox, Safari, or Internet explorer. The user interface represents the tiled display along with the application windows currently running on the SAGE environment. Interactivity is achieved by allowing multiple users to move and resize the application windows simultaneously.

Display Controller Application Architecture

Design and Implementation of SAGE Display Controller

Autostereoscopy (AS) is an increasingly valuable virtual reality (VR) display technology; indeed, the IS&amp;T / SPIE Electronic Imaging Conference has seen rapid growth in the number and scope of AS papers in recent years. The first Varrier paper appeared at SPIE in 2001, and much has changed since then. What began as a single-panel prototype has grown to a full scale VR autostereo display system, with a variety of form factors, features, and options. Varrier is a barrier strip AS display system that qualifies as a true VR display, offering a head-tracked ortho-stereo first person interactive VR experience without the need for glasses or other gear to be worn by the user. 

Since Varrier&rsquo;s inception, new algorithmic and systemic developments have produced performance and quality improvements. Visual acuity has increased by a factor of 1.4X with new fine-resolution barrier strip linescreens and computational algorithms that support variable sub-pixel resolutions. Performance has improved by a factor of 3X using a new GPU shader-based sub-pixel algorithm that accomplishes in one pass what previously required three passes. The Varrier modulation algorithm that began as a computationally expensive task is now no more costly than conventional stereoscopic rendering. Interactive rendering rates of 60 Hz are now possible in Varrier for complex scene geometry on the order of 100K vertices, and performance is GPU bound, hence it is expected to continue improving with graphics card enhancements. 

Head tracking is accomplished with a neural network camera-based tracking system developed at EVL for Varrier. Multiple cameras capture subjects at 120 Hz and the neural network recognizes known faces from a database and tracks them in 3D space. New faces are trained and added to the database in a matter of minutes, and accuracy is comparable to commercially available tracking systems. 

Varrier supports a variety of VR applications, including visualization of polygonal, ray traced, and volume rendered data. Both AS movie playback of pre-rendered stereo frames and interactive manipulation of 3D models are supported. Local as well as distributed computation is employed in various applications. Long-distance collaboration has been demonstrated with AS teleconferencing in Varrier. A variety of application domains such as art, medicine, and science have been exhibited, and Varrier exists in a variety of form factors from large tiled installations to smaller desktop forms to fit a variety of space and budget constraints. 

Newest developments include the use of a dynamic parallax barrier that affords features that were inconceivable with a static barrier. 

Keywords: Varrier, autostereoscopic 3D display, virtual reality, camera-based tracking, parallax barrier

6 Panel Varrier System with Camera-based Face Tracking

Evolution of the Varrier&trade; Autostereoscopic VR Display: 2001-2007

CytoViz is an artistic, real-time information visualization driven by statistical information gathered during gigabit network transfers to the Scalable Adaptive Graphical Environment (SAGE) at various events. Data streams are mapped to cellular organisms defining their structure and behavior as autonomous agents. Network bandwidth drives the growth of each entity and the latency defines its physics-based independent movements. The collection of entity is bound within the 3D representation of the local venue. This visual and animated metaphor allows the public to experience the complexity of high-speed network streams that are used in the scientific community. 

Moreover, CytoViz displays the presence of discoverable Bluetooth devices carried by nearby persons. The concept is to generate an event-specific, real-time visualization that creates informational 3D patterns based on actual local presence. The observed Bluetooth traffic is put in opposition of the wide-area networking traffic by overlaying 2D animations on top of the 3D world. Each device is mapped to an animation fading over time while displaying the name of the detected device and its unique physical address. 

CytoViz was publicly presented at two major international conferences in 2005 (iGrid2005 in San Diego, CA and SC05 in Seattle, WA). 

Keywords: Visualization, network performance, Bluetooth, information visualization

CytoViz Exhibited at iGrid 2005

CytoViz: an Artistic Mapping of Network Measurements as Living Organisms in a VR Application

This paper describes the research and development of a virtual reality visualization project &ldquo;Passing excellence&rdquo; about the world famous architectural ensemble &ldquo;Kizhi&rdquo;. 

The Kizhi Pogost is located on an island in Lake Onega in northern Karelia, Russia. The building housing the museum is a historical structure that was constructed in the ancient wood-based architecture tradition which is native to the region. The structure is a particularly fine example of the traditional wood architecture of medieval and post-medieval orthodox settlements, and is considered to be a unique artistic achievement, preserving a concentration of masterpieces of Russian heritage. The site is currently included in the List of Most Endangered Sites of the World Monuments Watch protected by World Heritage List of UNESCO. 

The Kizhi Project strives to create a unique virtual observation of the dynamics of the architectural changes of the museum area beginning from the 15th Century up to the 21st Century. The visualization is being created to restore the original architecture of Kizhi island based on the detailed photographs, architectural and geometric measurements, textural data, video surveys and resources from the Kizhi State Open-Air Museum archives. 

The project is being developed using Electro, an application development environment for the tiled display high-resolution graphics visualization system and can be shown on the virtual reality systems such as the GeoWall&trade; and the C-Wall. 

Keywords: Art, historical restoration, visualization, computer graphics, Virtual Reality (VR)

3D Model of Kizhi Belfry in Maya

Passing Excellence

It is the uniqueness of Virtual Reality as a medium that calls for the creation of hybrid realities which blur the finite boundaries between physical and digital existence. Virtual Reality&rsquo;s distinguishing features as an artistic medium embody a distinct form of aesthetics: it is a stereoscopic, immersive, interactive, performative, dynamic, and experiential medium. 

A Virtual Reality art piece manifests in multiple ways. It can present itself as an interactive virtual archetype, exploring concepts rendered from different perspectives, and as an impetus to challenge the platform&rsquo;s capabilities, not only theoretically as an artistic practice, but also by calling for the instantiation of authoring tools for the development of virtual reality experiences. 

The paradigm presented in this paper is a Virtual Reality art piece, called skin, 2006, was developed on Electro, which is an open-source cross-platform development environment. skin, 2006, is an interactive hypersteroscopic high-definition audiovisual installation that explores a dialogue between physical and digital senses of &ldquo;touch&rdquo;. 

The installation consists of a screen representing artificial skin, onto which hyper-stereo skin-related video imagery is projected. The imagery is accompanied by a generative composition of digitally manipulated and synthesized skin sources: porous breathing, the friction of skin on various surfaces, and synthesized representations of the electrical pulses inherent in skin at the molecular level. 

These skin-derived media serve as an exploration of a new territory: the intersection of the human body as a landscape / mindscape and the body&rsquo;s own traces of touch. By reacting to the imagery, the participating audience deforms virtually the skin-derived media. Physical gestures are translated into accentuated and non-literal digital deformations of the projected imagery, while new sounds, both organic and synthetic, are added to the mix to accompany these deformations. 

Keywords: Virtual reality, hyperstereo video, electro, open source

Publications - 2007

Towards Terabit/s Systems: Performance Evaluation of Multi-Rail Systems

WallCology: A Learner’s Interface to a Persistent Virtual Ecosystem

Distributed Volume Rendering of Very Large Data on High-Resolution Scalable Displays

Passing Excellence, The Interactive Art Visualization Of the Kizhi Ensemble

Automating the Extraction of 3D Models from Medical Images for Virtual Reality and Haptic Simulations

GPU-based elastic-object deformation for enhancement of existing haptic applications

Contact