Recognizing the value of autostereoscopy for 3D displays in public contexts, we pursue the goal of large-scale, high-resolution, immersive virtual reality using lenticular displays. Our contributions include the scalable tiling of lenticular displays to large fields of view and the use of GPU image interleaving and application optimization for real-time performance. In this context, we examine several ways to improve group-viewing by combining user tracking with multi-view displays. 

Keywords: Autostereo, hardware, architecture, immersive, lenticular, multi-user virtual reality, VR 

ISBN: 978-1-4503-0441-2, <a href="http://doi.acm.org/10.1145/1889863.1889899">doi 10.1145/1889863.1889899</a>

The tiling of lenticular displays achieved by shifting the center channels of all displays into alignment at the plane of focus

A multi-viewer tiled autostereoscopic virtual reality display

As the amount and the resolution of collected scientific data increase, scientists are realizing the potential benefits that large high-resolution displays can have in assimilating this incoming data. Often this data has to be processed on powerful remote computing and storage resources, converted to high-resolution digital media and yet visualized on a local tiled-display. This is the basic premise behind the OptIPuter model. While the streaming middleware to enable this kind of work exists and the optical networking infrastructure is becoming more widely available, enabling multi-user interaction in such environments is still a challenge. In this paper, we present an interaction system we developed to support collaborative work on large high-resolution displays using multiple interaction devices and scalable, distributed user interface widgets. This system allows multiple users to simultaneously interact with local or remote data, media and applications, through a variety of physical interaction devices on large high-resolution displays. Finally, we present our experiences with using the system over the past two years. Most importantly, having an actual working system based on the OptIPuter model allows us to focus our research efforts to better understand how to make such high-resolution environments more user-friendly and usable in true real-world collaborative scenarios as opposed to constrained laboratory settings. &copy; 2010 Elsevier B.V. All rights reserved. 

<a href="http://www.sciencedirect.com/science?_ob=ArticleURL&amp;_udi=B6V06-51HMWWS-2&amp;_user=10&amp;_coverDate=11/21/2010&amp;_rdoc=1&amp;_fmt=high&amp;_orig=search&amp;_origin=search&amp;_sort=d&amp;_docanchor=&amp;view=c&amp;_acct=C000050221&amp;_version=1&amp;_urlVersion=0&amp;_userid=10&amp;md5=84bdbfda11ef97e38f5a3206a739ba93&amp;searchtype=a">Link to this publication on-line at ScienceDirect</a>

The media browser, developed using SAGE widgets, is a tool for easily browsing and showing multimedia files on large high-resolution displays

Enabling multi-user interaction in large high-resolution distributed environments

* Authors: DeFanti, T.A., Acevedo, D., Ainsworth, R.A., Brown, M.D., Cutchin, S., Dawe, G., Doerr, K., Johnson, A.E., Knox, C. Kooima, R., Kuester, F. Leigh, J., Long, L., Otto, P., Petrovic, V., Ponto, K., Prudhomme, A., Rao, R., Renambot, L., Sandin, D.J., Schulze, J.P., Smarr, L., Srinivasan, M., Weber, P., Wickham, G. 

The CAVE, a walk-in virtual reality environment typically consisting of 4-6 3m-by-3m sides of a room made of rear-projected screens, was first conceived and built in 1991. In the nearly two decades since its conception, the supporting technology has improved so that current CAVEs are much brighter, at much higher resolution, and have dramatically improved graphics performance. 

However, rear-projection-based CAVEs typically must be housed in a 10m3 room (with space behind the screen walls for the projectors), which limits their deployment to large spaces. The CAVE of the future will be made of tessellated panel displays, eliminating the projection distance, but the implementation of such displays is challenging. Early multi-tile, panel-based, virtual-reality displays have been designed, prototyped, and built for the King Abdullah University of Science and Technology (KAUST) in Saudi Arabia by researchers at the University of California, San Diego, and the University of Illinois at Chicago. New means of image generation and control are considered key contributions to the future viability of the CAVE as a virtual-reality device. 

Keywords: CAVE, Computer-supported collaborative work (CSCW), Graphics packages, Image displays, Immersive environments, Interactive environments, Sonification, Tele-immersion, Virtual reality, Scalable multi-tile displays.

EVL&rsquo;s CyberCommons and SAGE

The Future of the CAVE

Simulation and modeling of turbulent flow involve solving for and analyzing time-dependent and spatially dense tensor quantities, such as turbulent stress tensors. The interactive visual exploration of these tensor quantities can effectively steer the computational modeling of combustion systems. At the same time, visualizing such data poses significant challenges in terms of clutter, occlusion
and navigation. We present an interactive framework which combines a detailed 3D inspection view based on volume rendering with glyph-based representations - used as 2D probes -, while leveraging interactive filtering and flow salience cues to clarify the structure of the tensor datasets. The result is a preliminary visual analysis tool to be utilized in debugging, benchmarking, and verification of models and solutions in turbulent combustion. We demonstrate this analysis tool on two example configurations and report feedback
from combustion researchers, in particular with respect to the merits of the visualization techniques employed. 
Keywords: Tensor Visualization, Application, Strain Tensor, Computational Turbulent Combustion.

Combined tensor visualization techniques in computational turbulent combustion.

Tensor Visualization in Computational Turbulent Combustion: A Case Study

The human ability to express and recognize emotions plays an important role in face-to-face communication, and as technology advances it will be increasingly important for computer-generated avatars to be similarly expressive. In this paper, we present the detailed development process for the Lifelike Responsive Avatar Framework (LRAF) and a prototype application for modeling a specific individual to analyze the effectiveness of expressive avatars. In particular, the goals of our pilot study (n = 1,744) are to determine whether the specific avatar being developed is capable of conveying emotional states (Ekman&ouml;s six classic emotions) via facial features and whether a realistic avatar is an appropriate vehicle for conveying the emotional states accompanying spoken information. The results of this study show that happiness and sadness are correctly identified with a high degree of accuracy while the other four emotional states show mixed results.

Skin Texture Enhancement. Top: default, Bottom: acquired by projecting high quality photos of a target person onto 3D mesh model

Designing an Expressive Avatar of a Real Person

This thesis focuses on the research and development of a novel Haptics-based Augmented Reality system that is able to realistically simulate open surgical procedures, called ImmersiveTouch&reg;. 

As a proof of concept, a neurosurgical procedure called ventriculostomy is simulated. This surgical procedure consists of the insertion of a catheter in the brain ventricles for draining excess spinal fluid and reducing intracranial pressure. While performing the surgery, neurosurgeons must rely on their fine eye-hand coordination and sense of touch to properly orient and introduce the catheter, as a subtle puncturing sensation is felt when the catheter cannulates the ventricles. 

The new high-fidelity simulator can help trainees to develop both tactile and psychomotor skills required to perform a ventriculostomy. As an interdisciplinary project between the Department of Mechanical and Industrial Engineering and the Department of Neurosurgery at UIC, the ultimate goal of this doctoral dissertation is to provide an interactive educational instrument to better train medical residents, allowing them to become proficient prior to working with real patients, unrestricted by time and patient safety constraints.

Visualization of the cannulated ventricles

Open Surgery Training Simulator Using Haptics and Augmented Reality Technologies

The Fall Beyond Tomorrow&rsquo;s Life is an interactive environment that unites water and sound. The primary components of the piece are a volume of water, a hexagonal multichannel speaker array and a computer vision apparatus. The computer vision component is positioned above the volume of water and captures the motion of the surface of the water. This interaction and the resultant waves are captured by the camera and translated into spatialized sound via custom software. The volume of water is an analog of the room in which it is placed - interaction in specific areas of the volume cause sounds in a corresponding area of the room. 

The impetus of this project was the investigation of water as an interface for the creation of sonic events. Water is inextricably tied to human existence and therefore holds the potential of a unique membrane of interaction. It has alternately symbolized life, love, death, rebirth and renewal in myriad cultures and is never far from anyone&rsquo;s conscious mind in this era of mass environmental concern. I seek to create an environment that has at its core a volume of water, which the viewer encounters with little or no added artifice. When the viewer touches the surface of the water, sounds resonate accordingly.

MFA Thesis Opening Reception: The Fall Beyond Tomorrow&rsquo;s Life

The Fall Beyond Tomorrow&rsquo;s Life

While many visualization tools exist that offer sophisticated functions for charting complex data, they still expect users to possess a high degree of expertise in wielding the tools to create an effective visualization. This paper presents Articulate, an attempt at a semi-automated visual analytic model that is guided by a conversational user interface to allow users to verbally describe and then manipulate what they want to see. We use natural language processing and machine learning methods to translate the imprecise sentences into explicit expressions, and then apply a heuristic graph generation algorithm to create a suitable visualization. The goal is to relieve the user of the burden of having to learn a complex user-interface in order to craft a visualization. 

Keywords: Visual analytics, natural language processing, conversational interface, automatic visualization.

Graphical user interface for the Articulate system

Articulate: A Semi-automated Model for Translating Natural Language Queries into Meaningful Visualizations

<a href="http://grapp.visigrapp.org">GRAPP 2010 Website</a> 

This work presents a distributed image-order volume rendering approach for scalable high-resolution displays. This approach reprocesses data into a conventional hierarchical structure which is distributed across the local storage of a distributed-memory cluster. The cluster is equipped with graphics cards capable of hardware accelerated texture rendering. The novel contribution of this work is its unique data management scheme that spans both GPU and CPU memory using a multi-level cache and distributed shared-memory system. Performance results show that the system scales as output resolution and cluster size increase. An implementation of this approach allows scientists to quasi-interactively visualize large volume datasets on scalable high-resolution display arrays. 

Keywords: High-resolution display arrays, direct volume rendering, parallel rendering, graphics systems and distributed shared-memory.

Scientist examining the rat kidney dataset on a 100 mega pixel display

Distributed Volume Rendering for Scalable High-Resolution Display Arrays

The Scalable Adaptive Graphics Environment (SAGE) is high-performance graphics middleware that enables ultrascale collaborative visualization using a display-rich global cyberinfrastructure. Distributed ultraresolution displays and high-performance-computing resources connect over high-speed networks to form this cyberinfrastructure. Over 40 sites worldwide, comprising universities, research institutions, and companies, have adopted SAGE to drive their ultraresolution displays.

Ultrascale Collaborative Visualization Using a Display-Rich Global Cyberinfrastructure

The Echo Coat Series is an exploration of the contemporary relationship of sound, women and public spaces. Inspired by the tradition of music, modernistic movements within sound, and the current ubiquity of individualized soundtracks, this series of historically aesthetic, technological, sound-driven garments creates a female-centered narrative of embodied wandering. The current environments of retail, architecture, and transit are transformed by deflections of consumerist language, real time echoes of boot heels, and the rapture of durative tones, impacting the nature of public sonic space in new ways. 

My project includes three coats that are designed in modernistic styles. They celebrate the point in history that sound began to open up conceptually and noise was incorporated into the classical music tradition. The coats are entitled Andante Coat, Staccato Coat and Largo Coat. Each coat has a character associated with them that have been further developed in a video work accompanying their display. The technology includes using RJDJ1, a platform for reactive music for the iPod Touch and iPhone2, using re-configurable &ldquo;scenes&rdquo; to create real-time sounds. Speakers embedded on the outside of the coats project the sounds to claim sonic space, celebrating sound embodiment through the body carrier. The names are based in the classical tradition of music; Andante and Largo connoting tempo in beats per minute (Andante: at walking pace, 76 to 108 bpm; Largo: the slowest tempo, 40 to 60 bpm), and Staccato describing a stiff attack of a note. The coats celebrate noise, experiments with randomized language, and minimal composition, in an attempt to celebrate public movement.

Staccato Coat, Andante Coat, Largo Coat: Thesis Exhibition

Fashion, Technology and the Sonic Wander: The Echo Coat Series

We present the design and process of an interactive high-resolution visualization system for diverse and distributed real-world geological core drilling expeditions. The high domain knowledge barrier makes it difficult for a person who is outside this field to imagine the user experience, and the globally distributed core drilling community imposes more design constraints in space and time. In addition to activities proposed in prior literatures, we used the &ldquo;immersive empathic design&rdquo; approach of having a computer scientist trained as a junior core technician. Through in-situ observation and interview evaluations from on-going expeditions, we present the system and the lesson learned in the process. It makes the best use of precious co-located opportunities. It allows the developer to build up domain knowledge efficiently. It establishes a trust relationship between the developer and scientists. The system designed through this approach formed a sustainable foundation that was adapted in the following design iterations. This process allows the software developer to experience authentic user activities. The designed system is innovative and helps scientists solving real-world problems. This approach can be a useful example to HCI practitioners who work with potential users or communities that share similar properties. 

Keywords: HCI, Visualization, Empathic Design 

ACM Classification Keywords: H.5.3 [Information Interfaces and Presentation]: Group and Organization Interfaces - Computer-supported cooperative work, Synchronous interaction, Collaborative computing 

General Terms Design, Human Factors

CoreWall User Interface Screen Shot

Case Study - Designing An Advanced Visualization System for Geological Core Drilling Expeditions

Images unlock hidden memories. 

This is not about memories from my childhood or my past. Rather, these are enigmatic images lingering within me from unknown territory. 

Paramnesia I is an experimental moving image installation. Deciphering the meaning of &ldquo;paramnesia&rdquo; in terms of psychiatry: the definition is a distortion of memory in which fact and fantasy are confused. I reflect and depict the poignant derivation of a pregnant female figure with her garment, furniture, and natural force based on my experience and indistinguishable recollections. A glitch clouds my present-day memory and reanimates my subconscious in a series of startlingly vivid eidetic images: old garments, chairs, gravity, air and water. The images of old garments are mostly related to huge dresses worn by noble Korean and European women from long ago. Fabric layered heavily with deep luxurious flowing colors; patterns and textures are floating in the air. At times, the thick murky air suspends them with no gravity or below the bottomless water. Motions are getting slower and smoother due to the pressure of air and/or water. Other images consist of a sofa and an arm chair which relate to texture and color of these garments. The overall shape and design of the furniture is antique and worn, but still reserving qualities of opulence. These images linger within me. They are as much a part of me as the blood that runs through my veins. I experience them as cinematic motion and still images. I still cannot connect them to anything I know. Could it be scenes from my past-life? - or a false surreal vision? The unconscious exists as unexplainable images, but what is fact and what is fiction? What is authentic and what is borrowed? What rises to the surface and what remains submerged? The answer is ambivalent. 

The computerized body of a woman constructed in three dimensional software (Maya), and wrapped with drawn or scanned textures. The ways that digital artists and traditional painters approached the works are inevitably different; however, they are of equal significance in manner of practice. The process of creating those digital images involves nostalgia for traditional painting and sculpture. As I manipulate reflections, lighting and shadows, I recall portraits and still lives in oil painting and charcoal drawing. Virtual figures, garments, and environments incorporate elements of drawing, painting, and sculpting meticulously. The simulated reality is completed within the intricate digital world. 

The images of Paramnesia are a collection of the envisioning maternity wear with floating bodies, in which I expand upon, based on my subconscious and eidetic images. Captivating woman&rsquo;s body is depicted with physical manipulation dressed in garments from reality and unrealistic fashion from my hidden memories. The character is posed in virtual nature or interiors, appealing with serenity of movements, yet simultaneously possessing an awareness of their unsure situation. Their inner strength is seemingly in contrast with fragility of their appearance. My figures, garments, objects and surroundings represent another realm of emotion. This environment is layered in meaning, ambiguity and supported virtual domains.

Pregnant woman figure from Paramnesia I

Fluid Cinema - Alternative Experimental Moving Images : Paramnesia I

Data management in scientific drilling programs such as the Integrated Ocean Drilling Program (IODP), the International Continental Scientific Drilling Program (ICDP), and the Antarctic Drilling Program (ANDRILL) performs two functions: firstly, the capture of drilling and scientific data during an expedition, and secondly, the long-term storage and dissemination of these data. Here we describe the progress in linking data management with stand-alone data capture and visualization applications. This provides a two-way flow of data between the database and the applications, and a more integrated data environment for scientists. The new system has been tested, so far, with cores from the IODP Expedition 313 New Jersey Shallow Shelf and the ICDP Lake El&rsquo;gygytgyn Drilling Project. 

The image on the right shows the Corelyzer station at the IODP Exp. 313 New Jersey Shallow Shelf onshore science party at MARUM (University of Bremen). This station was fed continuously with scanned images and MSCL data of new core sections as they were added to the ExpeditionDIS. In conjunction with Correlator, this station was used for correlation between sites using lithological features and/or logging data.

Corelyzer station at the IODP Exp. 313 New Jersey Shallow Shelf onshore science party at MARUM (University of Bremen)

New Integrated Data Analyses Software Components

An algorithm to provide haptic feedback directly from volumetric datasets is introduced in this paper. The algorithm is able to reproduce and extend the functionality of surface-based haptics rendering methods. It can be easily implemented on top of existing haptic libraries, such as Sensable&rsquo;s OpenHaptics. In addition, our approach overcomes poor performance in OpenHaptics for models consisting of a large number of polygons (1Million+). The algorithm allows detecting collisions with multiple 3-dimensional shapes. After comparing alternative approaches, such as Depth Buffer and Feedback Buffer, as well as SenseGraphics&rsquo; Volume Haptics ToolKit (ScalarSurfaceFriction mode), our algorithm delivers the highest performance in terms of rendering time.

User&rsquo;s View of Augmented Reality Session Using Haptics

Haptic Interaction with Volumetric Datasets Using Surface-based Haptic Libraries

Ultra-high-resolution tiled-display walls are typically driven by a cluster of computers. Each computer may drive one or more displays. Synchronization between the computers is necessary to ensure that animated imagery displayed on the wall appears seamless. Most tiled-display middleware systems are designed around the assumption that only a single application instance is running in the tiled display at a time. Therefore synchronization can be achieved with a simple solution such as a networked barrier. When a tiled display has to support multiple applications at the same time, however, the simple networked barrier approach does not scale. In this paper we propose and experimentally validate two synchronization algorithms to achieve low-latency, intertile synchronization for multiple applications with independently varying frame rates. The two phase algorithm is more generally applicable to various high resolution tiled display systems. The one-phase algorithm provides superior results but requires support for the Network Time Protocol and is more CPU-intensive. 

Keywords: Frame synchronization, Tiled display, Cluster computing 

General Terms: Algorithms, Performance

In SAGE, a compute cluster drives the individual displays, acting as a lightweight client that receives pixels from remote rendering resources

Multiapplication Intertile Synchronization on Ultra-High-Resolution Display Walls

Driving safety is sensitive to conditions and diseases that affect sleep, and individuals with obstructive sleep apnea syndrome have a high risk of at-fault accidents. Most driving simulators used for research are unrealistic, stationary, prohibitively expensive, or not specifically designed to study sleep disorders. The purpose of this study was to test the feasibility of a new lowcost, portable, realistic driving simulator intended for use in sleep disorders research. 

Objectives 
<ul>
<li>Free, open-source software, based on Ogre3D graphics engine</li>
<li>Portable, inexpensive hardware</li>
<li>Realistic</li>
<li>Specifically aimed at studying daytime cognition and performance of individuals with sleep disorders</li>
<li>Track lane position, speed, steering wheel position, braking, acceleration, collisions, and reaction times</li>
<li>Configurable via XML files</li>
</ul>

Driving Simulator Long, Straight Road Scenario

A New Driving Simulator for Sleep Research

Communication services, architecture, and technologies are rapidly evolving in response to application demand and research innovation. These changes are motivating a fundamentally new approach to the design and provisioning of services, facilities, and infrastructure. Traditionally, such resources have been designed and implemented as centralized fixed utility services, with almost no options for specialization and customization, especially by processes at the network edge. Supporting infrastructure has been created and deployed much like buildings are constructed, using a set of highly defined plans, resulting in hardened structures intended to exist basically unchanged for many years. However, this approach leads to severe restrictions on the creation, deployment, enhancement, and customization of services. Furthermore, this approach does not recognize the inherent potential of digital resources for affecting ongoing improvements through rapid continual change and for specialization and customization. Consequently, a new communication design model is being created, implemented in prototype, and placed into production at select sites around the world. This new model is based on multiple emerging trends in advanced network research. This model supports a much broader range of communication services than traditional systems and highly versatile functionality by providing a structural programmable framework. This structure encapsulates network resources as addressable modules that can be discovered, integrated, and customized to create many different types of communication services in advance or dynamically, in direct response to changing requirements.