Just as NSF cyberinfrastructure produces greater volumes and varieties of data from data storage systems, online instrumentation and major computational resources like XSEDE and Blue Waters, advanced visualization instruments serve as the &ldquo;lenses&rdquo; of these data generators (similar to the telescopes and microscopes of yore), enabling researchers to peer into cyberspace to manage, access and analyze big data. 
Continuum is a cyberinfrastructure instrument that unifies high-resolution computer-enhanced group collaboration workspaces with arrays of ambient sensors, enabling users to collaborate with local and remote colleagues and their data more effectively by giving the room itself the ability to anticipate their needs, improve their interaction, and focus their time on discovery rather than the technology. Continuum is a digital assistant, benefiting global scientific collaboratories and providing a foundation for new computer science research. War rooms benefit collaboration because they support continuous interaction and awareness of team members, enable persistent information sharing, and encourage mobility among team members and information. A computer-enhanced war room, such as Continuum, which (1) connects distributed teams with their data, whether stored in the cloud or streamed from major computational resources over high-speed networks, (2) supports persistent digital artifacts, (3) enables ultra-high-resolution 2D and 3D stereoscopic digital media to be simultaneously viewed, and (4) supports ubiquitous and intuitive interaction, creates a powerful and easy-to-use information-rich environment in support of scientific discovery. 
Continuum is the culmination of the University of Illinois at Chicago’s (UIC) 20+ years experience developing interactive instruments, from the original CAVE in 1992, to the ultra-high-resolution LambdaVision tiled LCD wall in 2004, and the hybrid-reality CAVE2 in 2012. Each new generation of visualization instrumentation provided scientific communities with more advanced features Continuum development focuses on three areas: (1) transforming passive display walls that respond to user commands into attentive display walls that monitor the users in a room and anticipate their needs to improve their experience; (2) providing an alternative approach to constructing high-resolution visualization display walls using thin-border 4K passive stereo OLED displays that have impressive brightness, contrast, uniformity, off-axis viewing, and low reflectivity in brightly lit rooms; and (3) utilizing cloud computing to prototype future Continuum spaces that don’t require local high-end compute and display resources. Twelve funded research projects from local institutions, in Astronomy, Astrophysics, Biology,
Combustion, Geophysics, Neuroscience, Pathology, Physics, Psychiatry, and High Performance Computing, are poised to use Continuum for their data visualization needs. Continuum will open up new opportunities in computer science research at the intersection of large-scale data visualization, human-computer interaction, and high-speed networking. Continuum will also directly support 10 classes taught in the UIC Computer Science, Art and Design, and Biomedical Science departments.

Artist's rendering of Continuum space.

MRI - Development of Continuum: A Virtualized Attentive Environment for Amplified Collaboration

The visualization of aleatoric uncertainty in ensemble simulations - i.e. the study of high concentrations in mixtures - provides many challenges due to the size and dimensionality of the data sets. EVL researchers have developed a web-based application designed to locate and track areas of high concentration in salt-solution simulations. 
The goal is to learn more about the behavior of Viscous Fingers, or contiguous areas of high concentration of salt within a cylinder of pure water. The application allows viewers to explore 3-D ensemble simulation data using both 2-D vertical slices of the cylindrical data to get a more detailed view of the data in a specific time step, as well as a feature tracking merge tree to better understand the evolution of these Viscous Fingers over the course of a simulation run.

Screen Shot of Viscous Fingers Visualization Application

Interactive Exploration and Tracking of Viscous Fingers in Large-Scale Ensemble Simulations

The Chicago 0,0 Riverwal AR experience provides a novel, interactive way for users to explore historical photographs sourced from museum archives. As users walk along the Chicago River, they are instructed to use their smartphone or tablet to view these photographs alongside the current views of the city. By matching location and view orientation the Riverwalk application creates an illusion of &ldquo;then and now&rdquo; co-extant. This superimposition of the historical photographer&rsquo;s view and the user view is the basis of educational engagement for the user and a key factor in curating the images and the narrative surrounding them, facilitating a meaningful museum experience in a public, outdoor context. 
As such, creating the AR experience involves a complex back-and-forth between 3D and 2D experiences of locations: the historical
images are 2D, taken from specific locations through specific optics and views; the user, present at the real-world 3D location is
orienting their own camera and 2D phone video screen in space in order to achieve the AR experience and illusion; and the ultimate experience of superimposition is one of seeing two integrated views simultaneously: the stream of data taken from a smartphone&rsquo; camera and the historic image. The site-specific nature of the publication makes it necessary to utilize a virtual 3D environment in which to place 2D augmented content that enhances the experience. This content needs to be placed in such a way so that the designer can accurately visualize what the user will see on screen from a particular location. 
The first episode of the Riverwalk AR experience focuses on a single block between N. LaSalle and Clark Streets; the site of
the Eastland Disaster in 1915. The site was selected because of the importance of this historical event - the sinking of the Eastland
cruise ship 100 years ago was the largest single loss of life in Chicago’s history - and because of the abundant media available
in the archive including extensive photographic documentation, newspapers and film reels. Moreover, the site potentially offers
natural way, finding characteristics that are amenable to developing our project, including a newly built pedestrian walkway along
the river with viewing platforms, views from pedestrian walks along the bridges, and historical markers. Additionally, the urban environment offers many features providing sufficient quality for robustly tracking user location based on their cameras from a variety of views and positions. This is a typical scenario in the creation of public outdoor AR experiences; the quality of content and narrative are the primary motivations for their creation.

Chicago 0,0 is a mobile application aimed at presenting 2D media content from archives of historical media through the use of augmented reality.

Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences

Patients come to Centers for Physical Therapy and Rehabilitation from all over the globe, &ldquo;often with catastrophic or unusual conditions and always with great hope for an improved life ahead&rdquo;. In the quest to assist these patients, Virtual Rehabilitation is becoming popular in clinical research thanks to the development and availability of new immersive technologies. For example, the rehabilitation of post-stroke patients frequently uses virtual reality, and sometimes employs a first-person Serious Games approach to make therapy more enjoyable for patients. In such approaches, the patient experiences the virtual environment in a first-person view. 
Unlike the existing body of work, we present and evaluate an immersive virtual rehabilitation application which uses a third person perspective: the patient controls with her body a 3D avatar. The application was commissioned by biomechanics researchers at a major center for rehabilitation research, and could be used to compare the learning-curve and time needed for rehabilitation across different therapies and environments. One of the main challenges in this work is the design of simple, non-distracting avatars and virtual world, which allows the patient to focus on performing correctly the rehabilitation exercises. An additional challenge comes from the trade-offs typically associated with virtual reality, for example, loss of depth perception. 
Our solution, RehabJim, is a third-person serious game for the rehabilitation of upper limbs in post-stroke patients, with particular emphasis on reaching movements. Our solution uses a state-of-the-art virtual reality environment, Unity 3D, and a Kinect controller in the CAVE2&trade; Hybrid Reality Environment.

External view of CAVE2 environment with a running instance of our application. The screens cover 320 degrees of visibility and a monitor at the entrance is used for managing the hardware resources.

RehabJim: A Third Person Approach To Virtual Reality Biomechanical Rehabilitation

Google Street View can be considered as an online browsable dataset consisting of billions of street-level panoramic images acquired all around the world. Since 2007, the company has continuously improved and updated its global image database, now collecting on average one panorama each 5-10m in urban environments. Google provides public APIs for requesting virtual camera views of a given panorama from its GPS position or unique identifier. These views are rectilinear projection of the spherical panorama with a user selected field of view, orientation and elevation angle. These views can be considered as undistorted images taken from pinhole cameras, free of distortion, and need to be partially overlapped in order to reconstruct the 360 degree panorama. 
Additionally, the speed with which Google&rsquo;s laser scanners get reflected by surfaces allows the company to calculate the distance from the camera to objects or buildings, allowing the creation of 3D models to be used in other applications such as Google Earth. Google API indeed provides a way to obtain the depth map corresponding to a specific panorama, information that we will exploit in order to reconstruct the 3D scene. Once decoded and visualized in space, this depth data and the models generated from it may turn out to be useful for many different applications like navigation and augmented reality, but also virtual reality oriented videogames.

Research - 2016

MRI - Development of Continuum: A Virtualized Attentive Environment for Amplified Collaboration

Interactive Exploration and Tracking of Viscous Fingers in Large-Scale Ensemble Simulations

Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences

RehabJim: A Third Person Approach To Virtual Reality Biomechanical Rehabilitation

3D City Reconstruction From Google Street View

Contact