The Chicago 0,0 Riverwal AR experience provides a novel, interactive way for users to explore historical photographs sourced from museum archives. As users walk along the Chicago River, they are instructed to use their smartphone or tablet to view these photographs alongside the current views of the city. By matching location and view orientation the Riverwalk application creates an illusion of &ldquo;then and now&rdquo; co-extant. This superimposition of the historical photographer&rsquo;s view and the user view is the basis of educational engagement for the user and a key factor in curating the images and the narrative surrounding them, facilitating a meaningful museum experience in a public, outdoor context. 
As such, creating the AR experience involves a complex back-and-forth between 3D and 2D experiences of locations: the historical
images are 2D, taken from specific locations through specific optics and views; the user, present at the real-world 3D location is
orienting their own camera and 2D phone video screen in space in order to achieve the AR experience and illusion; and the ultimate experience of superimposition is one of seeing two integrated views simultaneously: the stream of data taken from a smartphone&rsquo; camera and the historic image. The site-specific nature of the publication makes it necessary to utilize a virtual 3D environment in which to place 2D augmented content that enhances the experience. This content needs to be placed in such a way so that the designer can accurately visualize what the user will see on screen from a particular location. 
The first episode of the Riverwalk AR experience focuses on a single block between N. LaSalle and Clark Streets; the site of
the Eastland Disaster in 1915. The site was selected because of the importance of this historical event - the sinking of the Eastland
cruise ship 100 years ago was the largest single loss of life in Chicago’s history - and because of the abundant media available
in the archive including extensive photographic documentation, newspapers and film reels. Moreover, the site potentially offers
natural way, finding characteristics that are amenable to developing our project, including a newly built pedestrian walkway along
the river with viewing platforms, views from pedestrian walks along the bridges, and historical markers. Additionally, the urban environment offers many features providing sufficient quality for robustly tracking user location based on their cameras from a variety of views and positions. This is a typical scenario in the creation of public outdoor AR experiences; the quality of content and narrative are the primary motivations for their creation.

Chicago 0,0 is a mobile application aimed at presenting 2D media content from archives of historical media through the use of augmented reality.

Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences

MS candidate and Italian exchange student, Marco Cavallo presents his research in Augmented Reality (AR). His presentation will introduce a user-centered AR approach for publishing 2D media archives as AR content. Relevant technical considerations for developing an effective application in the Unity 3D game engine for public outdoor AR experiences that leverage context-specific elements in a challenging, real-world environment will be discussed. This research provides enabling technology for the the Riverwalk project - Chicago 0,0, a novel app-based AR experience that superimposes historical imagery onto matching views in downtown Chicago along the Chicago river Riverwalk pedestrian mile. Historical photographic and filmic views of the sites along the river are superimposed onto buildings, bridges, and other architectural features through image-based AR tracking, providing a striking experience of the city&rsquo;s history as rooted in extant locations within the community. 
This work is done in collaboration with Geoffrey Alan Rhodes, Assistant Professor at the School of The Art Institute of Chicago.

Riverwalk Augmented Reality Experience Presentation

A sample photo from the historical archive superimposed on the live camera stream. In order to create an appealing AR effect, the photo needs to appear aligned with specific environmental features.

Riverwalk: Incorporating Historical Photography in Public Outdoor Augmented Reality Experiences

This paper presents a novel approach for the design of creative location-based mixed reality applications. We introduce a framework called DigitalQuest that simplifies adding geolocated virtual content on top of real-world camera input. Unlike previous work, which relies solely on markers or image pattern recognition, we define a &ldquo;mirror world&rdquo; that facilitates interactive mixed reality. DigitalQuest consists of an editor that allows users to easily add their own content as desired and a mobile application that loads content from a server based on the location of the device. Each piece of virtual content can be organized through the editor so that it appears only in certain circumstances, allowing a designer to determine when and where a virtual object is attached to a real-world location. We have used our editor to create a series of futuristic scavenger hunts in which participating teams must solve puzzles in order to access new virtual context appearing in a mixed reality environment via a mobile phone application. In this paper, we introduce our editor and present an example scavenger hunt game, Morimondo, that was built using it. Specifically, we describe our technique to utilize camera and motion sensors on the mobile phone to enable an appropriate level of user engagement within this game. We are able to obtain realistic augmentations with accurate positioning by leveraging sensor fusion and through the use of filters that compensate for sensor noise, using image processing only for error correction or in special situations. The initial success of this project leads us to believe that DigitalQuest could be used to design a wide range of creative multi-user mixed reality applications. 
Index Terms: H.5.1 [Information interfaces and presentation (e.g., HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; H.5.3 [Group and Organization Interfaces]: Collaborative computing-Games.

A sample camera view of the mobile application showing the user arriving in proximity of a virtual object. The on-screen buttons provide user interaction; the upper-left bar provides a score and the current GPS accuracy.

DigitalQuest: A Mixed Reality Approach to Scavenger Hunts

Despite much progress and increased interest, mixed reality applications are not yet in widespread use by the general public. Our framework, DigitalQuest, facilitates the creation of mixed reality applications, providing application designers with the ability to add custom virtual content to the real world. Specifically it supports the creation of futuristic &ldquo;scavenger hunts&rdquo;, where multiple users search for virtual objects positioned in the real world, and where each object is related to a riddle or a challenge to be solved. Each player can compete with the other participants by finding virtual objects and solving puzzles, thereby unlocking additional challenges. Virtual objects are represented by animated 3D meshes locked to a determined position in the real world. The objects are activated when a player gets within a proximity threshold and then taps the object on the screen of his or her mobile phone. In our demonstration application, configurable virtual content appears, followed bya question that must be answered in order to be pass to the next challenge. The displayed content may consist of images, video and audio streams, graphical effects, or a text message that provides hints on how to advance in the game. The editor makes it easy to create puzzles that can be solved by exploring the surroundings of the virtual object in order to discover clues and making use of location-specific knowledge. When a participant figures out the correct solution, he or she scores points related to the complexity of the challenge and also unlocks remaining puzzles that cause new virtual object to appear in the world. At the end of the event, the player with more points wins. 
DigitalQuest provides a general approach for adding user-defined virtual content to the real world, allowing the personalization of a &ldquo;mirror world&rdquo; that could be explored by anyone with a GPS-enabled mobile device simply by using our application. Though these Quests were originally meant for creative exploration and mixed reality scavenger hunts, our framework also could potentially support a wide range of applications, for example: team building and team-work enhancement events, tourism, virtual galleries, and cultural heritage. As we demonstrate in our case study, DigitalQuest can also be used to create augmented narratives, engaging users by telling an interactive story that makes use of real-world architecture. We also believe that our framework could be used to provide context-aware opportunities for learning. DigitalQuest could additionally be used for marketing purposes, allowing companies to organize events with customized content in order to make customers more attached to the brand. Ultimately, the framework is meant to provide a way for designers to flexibly express creative approaches to generating virtual content within the real-world.

Google Street View can be considered as an online browsable dataset consisting of billions of street-level panoramic images acquired all around the world. Since 2007, the company has continuously improved and updated its global image database, now collecting on average one panorama each 5-10m in urban environments. Google provides public APIs for requesting virtual camera views of a given panorama from its GPS position or unique identifier. These views are rectilinear projection of the spherical panorama with a user selected field of view, orientation and elevation angle. These views can be considered as undistorted images taken from pinhole cameras, free of distortion, and need to be partially overlapped in order to reconstruct the 360 degree panorama. 
Additionally, the speed with which Google&rsquo;s laser scanners get reflected by surfaces allows the company to calculate the distance from the camera to objects or buildings, allowing the creation of 3D models to be used in other applications such as Google Earth. Google API indeed provides a way to obtain the depth map corresponding to a specific panorama, information that we will exploit in order to reconstruct the 3D scene. Once decoded and visualized in space, this depth data and the models generated from it may turn out to be useful for many different applications like navigation and augmented reality, but also virtual reality oriented videogames.

Sample point cloud reconstruction based on the output of merging 6 different panoramic images along a street.

3D City Reconstruction From Google Street View

Despite laser scan 3D point cloud acquisition has greatly improved over the next few years, the process of creating 3D large scale city models is still quite expensive and not straightforward. At the same time, nowadays services such as Google Street View provide a vast amount of geo-registered panoramic imagery, guaranteeing a decent resolution for dense locations at zero cost. Our idea is indeed to leverage this free information provided by Google Street View in order to obtain a cheap and automatizable 3D reconstruction of an urban area, by extracting the depth information related to the great number of panoramic images available online. 
ACM Classification Keywords 
H.5.m. Graphical user interfaces; Computer Graphics; Visualization 
Author Keywords 
Google Street View; Point Clouds; 3D reconstruction

Combination of multiple panoramas along a street.

A range of platforms facilitate the design of virtual environments. Most commonly, game engines, such as Unity and Unreal, are used for developing and simulating virtual realities. However, such platforms are primarily oriented towards visual design and provide only limited audio functionality. Furthermore, these platforms utilize separate editing and rendering modes, and operate on desktop platforms. Making use of existing research into the development of interactive virtual soundscapes, we introduce a novel user interface that enables the rapid design of both virtual sonic environments, and the assets (i.e. sound objects and sound zones) contained within them. 
Specifically, our UI offers the following contributions: 
<ul>
<li>provides a 3D design environment specific to sonic virtual realities, with specialized components such as sound objects and sound zones;</li> 
<li>offers both interactive and parametric control over the attributes of such components, enabling a precise authoring of highly-detailed environments;</li> 
<li>introduces a multi-cone model for creating 3D sound objects with complex propagation characteristics;</li> 
<li>enables adding dynamism to objects via hand drawn motion trajectories that can be edited in 3D;</li> 
<li>makes it possible to design virtual sound fields at various scales using multiple view and attribute windows;</li> 
<li>offers a unified interface for the design and the simulation of such realities, allowing the user to modify a sound field in real-time;</li> 
<li>operates on the web-browser so that it supports mobile devices, which therefore makes it possible for the user to simultaneously explore and edit augmented sonic realities.</li>
</ul>

A screenshot of our user interface on a desktop computer displaying an object with two cones and a motion trajectory being edited.

Designing and Controlling Virtual Sonic Environments Using a Browser-based 3DUI

TEDxUofIChicago 2016: Rethinking the System 
Victor Mateevitsi, a PhD candidate in the UIC Computer Science Department and Electronic Visualization Laboratoary (EVL), will talk about his SpiderSense human augmentics research at TEDxUofIChicago on April 8, 8 2016, 5:00-9:30pm, at the James Stukel Towers (JST) Event Center (near Halsted and Roosevelt). The event is currently sold out, but you can watch via a live web stream. 
For details, see the <a href="https://www.facebook.com/TEDxatUIC/">Facebook page</a>. 
<a href="https://m.youtube.com/watch?v=gtU-6cxi1OA">Watch a video of Mateevitsi's presentation</a>. 
See additional information on <a href="http://tedxatuic.com/">TEDxUofIChicago 2016: Rethinking the System.</a>
<a href="https://www.eventbrite.com/e/tedxuofichicago-2016-rethink-the-system-tickets-22801827881">https://www.eventbrite.com/e/tedxuofichicago-2016-rethink-the-system-tickets-22801827881</a>

Victor Mateevitsi

Victor Mateevitsi participating in TEDxUofIChicago, April 8, 2016

This research explores a novel approach to the design of Mixed Reality (MR) applications, which combines elements from markerless and location-based Augmented Reality in order to explore new types of interaction, leveraging a spatial representation of virtual content. In particular, a method is derived for developing both a mobile application, that can be used by people to see augmented content in the real world, and an editor application, which can be used to design and create MR experiences. Currently available mobile technologies needed to achieve these results are exploited, and the integration of sensors in order to improve tracking with a fully location-based approach, in which every content has a defined position in real space is explored. The advantages and the new interaction possibilities enabled by this method are weighed, in order to define the requirements for a custom MR authoring tool and its implementation. Particular emphasis is placed on the mapping between real-world and virtual coordinates and on the possibilities enabled by real-time editing, which allows the designer to preview the MR experience and also to interact with users.

Case Study - Overlaying Historical Photos

Merging Worlds: A Location-based Approach to Mixed Reality

UIC Computer Science and EVL exchange student from the Politecnico di Milano in Italy, Marco Cavallo presents his thesis research on Mixed Reality. 
&ldquo;Merging Worlds: A Location-based Approach to Mixed Reality&rdquo; presents a novel approach to the design of Mixed Reality (MR) applications, which combines elements from markerless and location-based Augmented Reality in order to explore new types of interaction, leveraging a spatial representation of virtual content. In particular, a method for developing both a mobile application, that can be used by people to see augmented content in the real world, and an editor application, which can be used to design and create MR experiences is proposed.

&ldquo;Merging Worlds&rdquo; Presentation

Navigation And Information Browsing Case Study

Marco Cavallo Thesis Presentation &ldquo;Merging Worlds: A Location-based Approach to Mixed Reality&rdquo;

EVL Summer Intern

Shreyas Mohan

This project&rsquo;s purpose was to investigate the ability of virtual reality to teach music. An application was developed for the CAVE2&trade; and Microsoft Hololens, and their ability to teach music effectively was analyzed. Both the applications teach the user how to play &rsquo;Mary Had a Little Lamb&rsquo;on the viola, and each virtual reality platform had itrsquo;s own benefits and drawbacks. An interface was created for both platforms that shows the user the correct bow direction, tempo, fingering, and sheet music. Additional features included adjustable tempo and a metronome that can be turned on and off. The application was helpful since the user could get visual cues from the graphics while still keeping an eye on his/her instrument. Future goals for the application include songs that can be uploaded by the user, microphone input, more instruments, and better graphics. The results of this research show that music instruction is a viable possibility in a virtual reality environment and will spark developers and musicians to work together to further explore this field. 
YouTube video of Microsoft Hololens version of this application: 
<iframe width="560" height="315" src="https://www.youtube.com/embed/Tv2SavmbDVg" frameborder="0" allowfullscreen></iframe> 
YouTube video of CAVE2 version of the application: 
<iframe width="560" height="315" src="https://www.youtube.com/embed/RZg9cfZYQqU" frameborder="0" allowfullscreen></iframe>

Shreyas Mohan in CAVE2 using the viola instruction virtual/augmented reality application he developed during his 2016 summer internship at EVL.

Music Instruction in a Virtual/Augmented Environment

EVL summer intern, Shreyas Mohan from Metea Valley High School, Aurora, IL presents the results of his efforts towards developing a robust music instruction application for use in virtual and augmented reality environments - the CAVE2 and Microsoft HoloLens systems. 
EVL Cyber-Commons, 2068 ERF 
Wednesday, August 10, 2016, 1:00 PM 
Shreyas developed an application for the CAVE2 and Microsoft HoloLens to investigate whether these technologies could be used effectively to teach music to a novice. Both applications taught the user &lsquo;Mary Had a Little Lamb&rsquo; on the viola, graphically representing the correct bow direction, tempo, fingering, and sheet music. Additional application features included adjustable tempo and metronome functionality. Combining graphical visual cues while seeing the physical instrument in a mixed reality environment proved to be a powerful method for independent music instruction.

Music Instructiton in VR/AR Presentation

HoloLens Interface

Music Instruction in a Virtual/Augmented Reality Environment (CAVE2&trade; and HoloLens)

This project&rsquo;s purpose was to investigate the ability of virtual reality to teach music. An application was developed for the CAVE2&trade; and Microsoft HoloLens, and their ability to teach music effectively was analyzed. Both the applications teach the user how to play &lsquo;Mary Had a Little Lamb&rsquo; on the viola, and each virtual reality platform had it&rsquo;s own benefits and drawbacks. An user interface was created for both platforms that shows the user the correct bow direction, tempo, fingering, and sheet music. Additional features included adjustable tempo and a metronome that can be turned on and off. The application was helpful since the user could get visual cues from the graphics while still keeping an eye on his/her instrument. Future goals for
the application include songs that can be uploaded by the user, microphone input, more instruments, and better graphics. The results of this research show that music instruction is a viable possibility in a virtual reality environment and will spark developers and musicians to work together to further explore this field.

Trackers placed on viola for CAVE2 version of the music instruction application.

Music Instruction in a Virtual/Augmented Reality Environment Using the CAVE2 and Microsoft HoloLens

This paper introduces a user-centered Augmented Reality (AR) approach for publishing 2D media archives as interactive content. We discuss the relevant technical considerations for developing an effective application for public outdoor AR experiences that leverage context-specific elements in a challenging, real-world environment. Specifically, we show how a classical marker-less approach can be combined with mobile sensors and geospatial information in order apply our knowledge of the surroundings to the experience itself. Our contributions provide the enabling technology for Chicago 0,0 Riverwalk, a novel app-based AR experience that superimposes historical imagery onto matching views in downtown Chicago, Illinois along an open, pedestrian waterfront located on the bank of the Chicago River. Historical photographs of sites along the river are superimposed onto buildings, bridges, and other architectural features through image-based AR tracking, providing a striking experience of the city’s history as rooted in extant locations along the river. 
Index Terms: H.5.1 [Information interfaces and presentation (e.g. HCI)]: Multimedia Information Systems-Artificial, augmented and virtual realities

A group of people testing the Riverwalk AR application with both smartphone and tablet devices along the Chicago River.

Newcity magazine is Chicago&rsquo;s only locally owned and operated cultural weekly, and has been covering Chicago&rsquo;s local scene for more than 25 years. It&rsquo;s current &ldquo;Best of Chicago&rdquo; issue just came out (Volume 31, No. 1359). The section on City Life lists many best things about Chicago, including &ldquo;Best place to get your geek on.&rdquo; Um, that would be the Electronic Visualization Laboratory at the University of Illinois at Chicago! The article is reproduced below. 
* * * * * * * * * * 
Best place to get your geek on 
The Electronic Visualization Laboratory (EVL) at UIC 
Cutting-edge research meets a massive wall made out of small screens that would make any hardcore videogame fan scream and a literal cave that opens up a highly immersive, surround-screen, surround-sound, virtual reality environment at the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago. The lab brings together academics, designers, engineers, artists and mad scientists using technologies that bend reality off into unimaginable directions. EVL is the place where visualization, virtual reality and real-time interactive computer graphics are born. As if that were not enough, it was here that the computer graphics for the first &ldquo;Star Wars&rdquo; film were created in 1976. How cool is that? 
UIC College of Engineering, Department of Computer Science, 842 W. Taylor 
Best of Chicago 2016

Best of Chicago 2016

Newcity magazine rates UIC/EVL as &ldquo;Best place to get your geek on&rdquo;

Node Kara is an audiovisual mixed reality installation by the artist Anıl Camci. It offers a body-based interaction using a 3D imaging of the exhibition space. With a blurred audiovisual scene in its steady-state, it invites visitors to interact with the multiplicity of layers it holds within. By moving in the exhibition space, the visitors can deblur both the sound and the visuals as they get immersed in the piece. Rather than a detection of discrete events or gestures, Node Kara relies on fluid interactions with continuous visual layers and stochastic audio. This way, the piece reacts to each visitor in a unique way. A video preview of Node Kara can be seen at <a href="https://vimeo.com/anilcamci/nodekara">https://vimeo.com/anilcamci/nodekara</a> (password: nodekara). 
Index Terms: H.5.1 [Information interfaces and presentation (e.g., HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; H.5.2 [Information Interfaces and Presentation]: User Interfaces-Auditory (non-speech) feedback.

Four layers of Node Kara superimposed onto each other with a representation of the visitor’s depth image highlighted.

Node Kara: An Audiovisual Mixed Reality Installation

October 2016 - Augmented reality (AR) is being used in a very novel and compelling way to engage Chicago residents and tourists into learning more about the city&rsquo;s history. John Russick of the Chicago History Museum teamed with Geoffrey Alan Rhodes at the School of the Art Institute (SAIC) to create Chicago 0,0 Riverwalk, a mobile AR application that superimposes historical imagery onto matching views of Chicago&rsquo;s downtown riverwalk. The first project focuses on a single block between N. LaSalle and Clark Streets, the site of the Eastland Disaster in 1915, in which 844 people lost their lives on a pleasure boat when it rolled over in the Chicago River. On October 19, 2016, journalist Jay Shefsky told the story of how AR to being used to chronicle the tragedy of the <a href="http://chicagotonight.wttw.com/2016/10/19/augmented-reality-app-offers-new-look-eastland-disaster">S.S. Eastland on an episode of WTTW Chicago Tonight television</a>. 
During the development of this application, UIC Computer Science (CS) professor Angus Forbes and his MS student Marco Cavallo, both affiliated with the Electronic Visualization Laboratory (EVL), worked with the SAIC faculty and students on technical considerations that have since been incorporated into the second version of the application software. EVL&rsquo;s expertise enabled better tracking and feature detection of real-world objects; better registration and orientation of the real-world environment with historical photographs; the use of a smart, intuitive interface to help users navigate in the real world and locate historical images; and, better image contrast, even on overcast days with adverse lighting conditions. Together, these technical efforts have contributed to better user experiences. 
In the WTTW story, John Russick of the Chicago History Museum notes the value of this AR application: &ldquo;One of the biggest challenges in communicating history is finding human emotional connections to very old stories … that draw us in a deeper, more meaningful way.&rdquo; Rather than passively view photographs or documentary films, AR enables users to experience history firsthand, in its relevant surroundings. 
<a href="http://chicago00.org/">Download more information and the Chicago 0,0 app for smartphones or tablets</a>. 
Marco Cavallo (UIC), Geoffrey Alan Rhodes (SAIC) and Angus Forbes (UIC) co-authored a poster paper <a href="https://www.evl.uic.edu/entry.php?id=2255">&ldquo;Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences,&rdquo;</a> that was presented at the 15th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), September 15-23, 2016, in Merida, Mexico.

Augmented Reality App Offers New Look at Eastland Disaster

Umwelt, the German word for environment, is a psychology term referring to the individual&rsquo;s mental image of the surrounding world. This subjective universe is shaped by an organism&rsquo;s unique perception of the environment around them and, therefore, differs from individual to individual. In other words, organisms experience the world through their senses, construct the mental model of their vicinity, and react upon it. Hence, this image never remains static - instead it is constantly refined, as a result of the dynamic nature of the world. Umwelt is closely related to the concept of Situation Awareness, that is the perception of environmental elements, comprehension of their nature, and projection of their near future status with respect to time and/or space. 
Sensory perception (exteroception) of the surrounding environment is, as a matter of fact, restricted to the limitations of biological senses. For example, although we consider humans with five fully developed senses (sight, hearing, touch, smell and taste) as able-bodied individuals, they are essentially blind to a multitude of information that constantly surrounds them. To put things into perspective, both human sight and hearing, can only perceive a fraction of information- part of the electromagnetic spectrum (light waves), and of the sound frequencies accordingly. However, as sensory systems differ among species, other creatures have an entirely different perceptual experience, as they are able to perceive a distinct spectrum of information. 
Despite the fact that our limited human senses restrict our Umwelt and, therefore, our Situation Awareness, Human Augmentics, referring to technologies that expand the capabilities and characteristics of humans, allows us to peek into an invisible &dlquo;other&drquo; world. Thus, some important questions arise: What would the consequences of an expanded perception of the
surrounding environment be, and how would that affect an individual&rsquo;s Situation Awareness? Since our Umwelt is limited by genetics, how would individuals attune with characteristics and capacities that lie beyond human range? Could Human Augmentics possibly expand an individual&rsquo;s Umwelt, leading to improved Situation Awareness? 
Jakob von Uexkull (1864 - 1944), a German biologist who worked on animal behavior studies, cybernetics of life, and muscular physiology, is attributed for coining the term Umwelt. Uexkull&rsquo;s Umwelt is the phenomenal world of a living animal, in which it exists and acts, and as a result is shaped according to the stimuli received from its senses. Due to the fact that individuals, even across the same species, sense the world differently, each animal creates its own unique Umwelt. 
Perception and, as a result, interaction with the world begins with the raw input of data from the environment through the sensory system. Anatomically, the sensory system is part of the nervous system and includes the sensory receptors that act as probes of the environment, parts of the brain that are used for sensory perception, and the neural pathways that connect the brain with the receptors. Notably, humans&rsquo; sensory system consists of five traditional senses: Sight, hearing, taste, smell and touch. The stimulation type and bandwidth of each sensory channel are limited to its characteristics, which in turn constrain the size of the Umwelt.
However not all living organisms possess the same qualitative or quantitative senses. Depending on the environment in which they have evolved, their sensory system may be entirely different than ours; some have more while others have less sensitive senses. Others have quite different ones. Bats and dolphins, for instance, have the ability to calculate orientation and position of other animals and objects through reflected beams of sound, an ability known as echolocation. Sharks and rays, on the other hand, can detect changes in the electric field of their vicinity, an ability known as electroreception. 
Situation Awareness 
It is a well-known fact that humans are capable of sensing only a small fraction of the data that constantly surrounds us. Magnetic fields, infrared light, ultrasound, cosmic rays, radiation are just a few of the things that are completely invisible to us. However, recent technological advances in specialized sensors, enable the detection of some of this information. For instance, geiger counters are used by hazmat personnel to assess radiation levels in nuclear accidents. These technologies provide feedback using one of our existing senses, usually visual or audio, in order to help workers safely perform daily tasks. 
Situation Awareness (SA) is the perception of environmental elements, comprehension of their nature, and projection of their near future status with respect to time and/or space. In other words, it is the awareness of the immediate vicinity, comprehension of the current state of the environment and ability to predict what may happen. Inputs from the sensory system are used by the individual in order to create and update their Situation Awareness state, and as a result, decisions depend on the accuracy of the perceived model. Situation Awareness is tightly associated with experience; somebody less experienced may miss information that a more experienced user will not. To put things into perspective, the hazmat personnel&rsquo;s (that we discussed above) Situation Awareness includes the geiger counter readings, comprehension of their meaning and ability to predict what will happen if they raise above a certain level. 
Despite the fact that sensory systems have been honed by millions of years of evolution, they are very limited to a multitude of data. We are blind to a plethora of information that exists around us, and fortunately most of it is harmless. We depend on our sensory system to interpret the world, and we use our Situation Awareness to comprehend the environment around us and act upon it. As a result, lack of information for a certain situation may lead to misjudgements and accidents. In fact 76% of aviation errors, happen because the pilot did not perceive one piece of the airplane&rsquo;s crucial information. To that end, another important question arises: Given that today&rsquo;s technology can sense some of this invisible information, can we expand the Umwelt and improve Situation Awareness? 
Research Questions 
Genetic defects, diseases or accidents can lead to sense disabilities or complete sense lose, which has been observed to both animals and humans. To that end, individuals learn over time to use the remaining of their senses in order to overcome a disability. Recent scientific discoveries have shown that while this new ability results in anatomical changes of the brain, the responsible regions remain the same. In other words, even though the sensory input channel has changed, the brain will still process the information in the same region|a phenomenon known as Sensory Substitution. 
As a result, Sensory Substitution devices use one or more of the remaining senses to increase an individual&rsquo;s Situation Awareness through a different sensory pathway. More specifically, tactile displays are able to translate external environmental information onto one&rsquo;s skin, through the use of haptics. While these systems are primarily designed for people with disabilities, their concepts can be extended to other sensory deprived individuals, who find themselves in no or limited visibility environments due to external, environmental reasons. For example, firefighters operating within smoky, low visibility hazardous environments or soldiers fighting in pitch black darkness, are more likely to have no Situation Awareness of their surrounding environment. 
The ever-dynamic nature of the world around us, however, makes navigating with a wearable tactile display, a quite challenging task. From feeling the vibrations onto one&rsquo;s skin, comprehending their meaning and correctly acting upon them, to getting inundated with tactile sensory overload, tactile displays have great potential, when communicating information in an eloquent
way. 
In this dissertation we will explore different environment-to-vibration mappings while assessing one&rsquo;s Situation Awareness. Furthermore, we seek to understand whether one's experience with a tactile display would have an effect on their Situation Awareness. Finally, we hypothesize that a one-size-fits-all wearable tactile display would not perform equally well in all environmental scenarios and therefore we are going to explore how different vibration configurations, patterns and environment-to-vibration mappings would affect one&rsquo;s navigation. All of the above are distilled into the following research questions: 
- Could tactile displays be used to increase Situation Awareness of visually deprived individuals? 
- To what extent would Situation Awareness be associated with experience when using a tactile display?

Positioning of Sensor Modules and Controller Box

Supporting Navigation with a Torso Wearable Tactile Display

JANUARY 24, 2019 - Daria Tsoupikova, associate professor in the UIC School of Design and an affiliated faculty member of the UIC Electronic Visualization Laboratory (EVL) which is part of the Computer Science (CS) department, recently received a $150,000 grant from the University of Illinois (UI) System&rsquo;s Presidential Initiative to Celebrate the Impact of the Arts and the Humanities. Given how important the arts and humanities are to education, research, scholarship, and public engagement, this newly-created initiative supports the visibility and excellence of the arts and humanities at the UI System&rsquo;s sister campuses. 
Tsoupikova&rsquo;s &ldquo;Transforming Storytelling: Multi-User Virtual Reality Theater for Collaborative Tele-Immersive Exploration&rdquo; is a collaboration with EVL Director of Research and CS associate professor Andrew Johnson, CS/EVL Senior Research Programmer Lance Long, CS/EVL PhD student Arthur Nishimoto, and Chicago-based theater director Jo Cattell, who recently received the Goodman Theatre&rsquo;s 2018 Michael Maggio Directing Fellowship. They will partner with the Goodman Theatre&rsquo;s Alice Rapoport Center for Education and Engagement, whose youth program works with Chicago&rsquo;s diverse demographics of middle-grade and teenage kids (ages 11-18). 
&ldquo;Transforming Storytelling&rdquo; will demonstrate how virtual-reality technology can be both an artistic medium to further immerse participants in live theater experiences and a storytelling medium. Project collaborators will develop multi-user theatrical performances that merge real and virtual worlds with the goal of encouraging stronger theater engagement with tech-savvy younger audiences. Chicago youth will attend performances at the Goodman Theatre and at UIC/EVL&rsquo;s CAVE2&trade;, allowing for immersive participation in the plot lines through avatars, hand-held devices and 3D platforms. 
While non-profit theater groups have great interest in incorporating new technologies in live performances, they don&rsquo;t necessarily have the financial support to do so. UIC&rsquo;s collaboration with the Goodman Theatre can demonstrate that live performance combined with virtual reality is a viable option for immersive theater. 
The UI System&rsquo;s Presidential Initiative selected 14 faculty projects from 53 proposals, which will share nearly $2-Million in funding over the next two years. Of the 14 proposals, all but one was allocated $100,000 or more and eight will receive $150,000 or more. By comparison, the average grant award in 2016 by the National Endowment for the Arts (NEA) was $25,000 and fewer than 40 NEA grants totaled more than $100,000, according to Grantmakers in the Arts. 
<a href="https://news.uillinois.edu/view/7815/738301">University of Illinois System News, January 24, 2019</a> 
<a href="https://cs.uic.edu/news-stories/uics-electronic-visiualization-lab-is-helping-reimagine-live-stage-performance/">UIC Computer Science Department News, April 25, 2019 
Chicago Tribune, January 25, 2019 
<a href="https://www.pressreader.com/usa/chicago-tribune/20190125/281638191417696">&ldquo;U. of I. grants $2M to bolster arts, humanities&rdquo;</a>

Young adults explore &ldquo;GenWorld,&rdquo; a virtual world previously developed by UIC CS/Design students Peter Hanula, Kyle Dalton, Morgan Osborn, and Christel Tiu as part of the Spring 2017 CS/Design Creative Coding class co-taught by Daria Tsoupikova

Tsoupikova&rsquo;s &ldquo;Transforming Storytelling: Multi-User Virtual Reality Theater for Collaborative Tele-Immersive Exploration&rdquo; Awarded

Since 2017, Leonardo commenced a quarterly recognition of exceptional peer reviewers noted for their in-depth and deeply constructive feedback on papers under consideration for publication. Among those recognized in the August 2019 edition of <a href="https://www.leonardo.info/blog/2019/07/22/recognition-of-leonardos-outstanding-peer-reviewers?utm_source=LNN+subscribers&utm_campaign=054f45e4b7-EMAIL_CAMPAIGN_2019_07_31_05_49&utm_medium=email&utm_term=0_59c5038641-054f45e4b7-116710661&mc_cid=054f45e4b7&mc_eid=c32370193e">Leonardo 52:4</a> is Daria Tsoupikova, associate professor in the UIC School of Design and an affiliated faculty member of the Electronic Visualization Laboratory. 
Tsoupikova's research and artwork focus on the development of virtual reality (VR) art projects and networked multi-user exhibitions for VR projection systems, such as the Cave Automatic Virtual Environment (CAVE and CAVE2). Her work lies at the crossroads of artistic and technological innovation and investigates computational creativity to design real-life applications that advance healthcare, education and social change.

Daria Tsoupikova Recognized Among Leonardo's Outstanding Peer Reviewers

Ph.D. Student Name: Arthur Nishimoto 
Committee: 
Prof. Andrew Johnson, Chair and Advisor 
Prof. Debaleena Chattopadhyay 
Prof. Steve Jones 
Prof. Wei Tang 
Prof. Jason Leigh (University of Hawaiʻi at Manoa) 
Date: 12/2/2019 
Time: 11 am - 1 pm 
Location: ERF 2068 
Abstract: 
Given the adoption of many variations of virtual and augmented reality devices such as CAVE/CAVE2, large virtual reality (VR) capable displays walls/domes, and a growing set of increasingly capable and affordable head-mounted displays (HMD) in both commercial and industrial applications, it has become apparent that not one platform will be appropriate for every use case scenario. HMDs are becoming increasingly more lightweight and powerful, but in collaborative settings there always will be a need for a shared display or environment to bring discussions together. 
The goal of this proposal is to leverage the best of current augmented reality head mounted display technologies: 360 degree field of regard visuals, rendering of graphics over the real world, and personalized views with the best of large display wall environments: high resolution and contrast, shared visual environment, and more accurate tracking. 
With fully immersive HMDs blocking our natural sense of communication and AR still limited by resolution and field of view, combining large VR display walls and AR may provide a middle ground to maximize the growing benefits of interacting together in virtual worlds.

Preliminary Examination Announcement: &ldquo;Collaborative Augmented Reality for Virtual Reality Display Wall Environments&rdquo;

October 1 - - EVL Director of Research Andrew Johnson has been named Interim Director of EVL following Maxine Brown&rsquo;s retirement from UIC. Johnson now assumes the responsibility of heading EVL - overseeing EVL&rsquo;s overall mission in education, research, development, and technology transfer. 
For over 23 years, Johnson has been a valued EVL faculty member (Professor of Computer Science) and researcher who has focused upon interaction and collaboration using advanced visualization displays and the application of those displays to enhance discovery and learning. 
His areas of expertise include visualization, virtual and augmented reality, natural language processing, visual analytics, and high-performance networking. 
Over the years, Johnson has been instrumental in the development of EVL instrumentation and systems, including the Continuum smart environment, CAVE2&trade;, LambdaVision high-resolution tiled displays, LambdaTable tabletop touch system, GeoWall portable geoscience display environment, PARIS augmented reality and ImmersaDesk virtual reality systems. He has overseen the development of numerous visualization and virtual reality applications across diverse domain disciplines from science and engineering to the humanities and art. Johnson is a highly regarded mentor and educator to EVL&rsquo;s graduate students and undergraduates alike. 
Johnson retains his position as Director of Research while leading EVL&rsquo;s future initiatives and innovations as Interim Director.

Andrew Johnson Named Interim Director, EVL

EVL alumnus Marco Cavallo was awarded with the UIC College of Engineering Outstanding Young Achievement Award in Computer Science for 2020. The award identifies alumni who, through their achievement in their field of engineering, have significantly enhanced the industry and who inspire and help to instill a sense of pride among the alumni, students, faculty and staff of UIC. Marco is among the 15 alumni award honorees in engineering and computer science who have used their UIC education to establish rewarding, productive, and high-impact careers.<bra> 
Marco currently works as a senior research engineer in Apple&rsquo;s technology development group, where he is in charge of creating data visualization tools and machine learning infrastructure to foster the development of new computer vision algorithms for augmented reality. 
He graduated with a dual master&rsquo;s degree in computer science and engineering from Politecnico di Milano in Italy and the University of Illinois Chicago, where he wrote a thesis on augmented reality while working at UIC&rsquo;s Electronic Visualization Laboratory. 
Congratulations Marco! 
See the award winner video on <a href="https://www.youtube.com/watch?v=Oskk1icFSvE&feature=youtu.be">YouTube</a>.

EVL and UIC Computer Science Alumnus Marco Cavallo Honored with Outstanding Young Achievement Award

The data entry task is one of the common office tasks performed in the seated position that can be assisted by augmented reality technology. The paper introduces an ongoing project to develop an AR-based data presentation interface to assist office workers&rsquo; data entry tasks. The interface developed in this project was designed to facilitate the transition between the two steps including reading data and typing the data into a computer system using a keyboard. Usability evaluation is conducted in a controlled experiment with human subjects to examine whether the AR-based data presentation interface improves data entry performance and mitigates office workers&rsquo; physical and mental workload. 
Keywords: Office work, augmented reality, data entry, human factors. 
Index Terms: Human-centered concepts [Human computer interaction (HCI)]: Interaction paradigms-Virtual reality; Human-centered concepts [Human computer interaction (HCI)]: Visualization-Empirical studies in visualization

Interface development process.

Using Augmented Reality to Assist Seated Office Workers&rsquo; Data Entry Tasks

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE&trade; and the LCD-based successor CAVE2&trade;. Each of these has distinct advantages and limitations based on how they&rsquo;re used. This work explores preserving the high resolution and sense of presence of CAVE2 environments in full stereoscopic mode by using a see-through augmented reality HMD to expand the user&rsquo;s field of regard beyond the physical display walls. In our explorative study, we found that in a visual search task in a stereoscopic CAVE2, the addition of the HoloLens to expand the field of regard did not hinder the performance or accuracy of the participant, but promoted more physical navigation which in post-study interviews participants felt aided in their spatial awareness of the virtual environment. 
CCS CONCEPTS: Human-centered computing, Human computer interaction (HCI); Mixed / augmented reality; Virtual reality. 
KEYWORDS; Augmented Reality; Virtual Reality; Display Wall; Immersive Displays; Presence

HoloLens view inside CAVE2

Extending Virtual Reality Display Wall Environments Using Augmented Reality

Construction activities for the new $117.8 million, 135,000-square-foot Computer Design Research and Learning Center to be located at 900 W. Taylor St. is underway. The CDRLC will support the Department of Computer Science&rsquo;s growing student enrollment with 16,000 square feet of classroom space for collaborative teaching and learning environments, as well as 35 labs for faculty research activities in the computer science and engineering disciplines. 
The CDRLC will be the new home for the Electronic Visualization Laboratory featuring the latest technological and computing innovations to support research in advanced visualization, networking and collaborative environments. 
Read more about the CDRLC groundbreaking in the <a href="https://today.uic.edu/uic-breaks-ground-on-new-computer-design-research-and-learning-center">July 15th issue of UIC Today.</a> 
See a live webcam of the building&rsquo;s progress via the <a href="https://webcampub.multivista.com/index.cfm?fuseaction=aPublicWebcam.embed&WebcamPublicEmbedUID=5D4DCF05-AB2B-4770-A090-D79BE467F958">Multivista Construction Documentation webcam.</a>

New Computer Design Research and Learning Center Breaks Ground

Following the premiere of &ldquo;Hummingbird&rdquo; as part of the Chicago&rsquo;s Tony Award-winning Goodman Theatre New Stages Festival in December 2021, the virtual reality theatrical experience was exhibited at the SIGGRAPH 2022 Conference Immersive Pavilion held in Vancouver, CA, August 8-11th. Team members enhanced the existing experience to allow for a stand-alone version running consecutively for 4 days to sell-out audiences/participants - augmenting interactivity, animations, videography as well as upgrades to the underlying software and VR systems. By all accounts the exhibition was extremely well received and a success. 
&ldquo;Hummingbird: Multi-User Virtual Reality Theater for Collaborative Tele-Immersive Exploration&rdquo; is a modern, innovative performance, merging live theater and interactive virtual reality by bringing a group of active participants into a shared space for a live performance. This project bridges art, science and live theater through a collaborative research effort between computer science and design faculty and students at the University of Illinois Chicago (UIC) Electronic Visualization Laboratory (EVL) and Chicago theater directors, actors, videographers and producers. &ldquo;Hummingbird&rdquo; is developed by Daria Tsoupikova (Professor, School of Design); Andrew Johnson (Professor, EVL, Computer Science); Jo Cattell (Goodman Theatre); Lance Long (Senior Research Programmer, EVL, Computer Science); Arthur Nishimoto (PhD Candidate, EVL, Computer Science); Sai Jyothula Priya (PhD Candidate, EVL, Computer Science).

Participants engaged in the &ldquo;Hummingbird&rdquo; virtual reality performance at S2022.

&ldquo;Hummingbird&rdquo; Exhibited at SIGGRAPH 2022 (S2022)

Hummingbird: is a modern, innovative performance merging live theater and interactive virtual reality by bringing a group of active participants into a shared space for a live performance. The performance premiered as part of Chicago&rsquo;s Tony Award-winning Goodman Theatre New Stages Festival showcasing innovative and ground-breaking theater works in December 2021. This project bridges art, science and live theater through a collaborative research effort between computer science and design faculty and students at the University of Illinois Chicago (UIC) Electronic Visualization Laboratory (EVL) and Chicago theater directors, actors, videographers and producers. Hummingbird&rsquo;s story celebrates courage and coming of age through the eyes of a gutsy teen who must outsmart her mother&rsquo's narcissistic boss and survive dangerous new technology in a live, immersive adventure. Hummingbird extends traditional live theater and makes virtual reality art accessible to a broader audience, demonstrating how virtual reality can transform theatrical storytelling. 
CCS CONCEPTS 
Applied computing - Arts and humanities; Performing arts; Human-centered computing - Collaborative and social computing; Collaborative and social computing theory, concepts and paradigms; Computer-supported cooperative work. 
KEYWORDS 
Virtual reality, Storytelling, Theater, Multi-user, Performance

&ldquo;Hummingbird&rdquo; being performed at EVL during pilot performances summer of 2021

Hummingbird: A Collaborative Live Theater and Virtual Reality Adventure

UIC Department of Computer Science 
Doctoral Admissions Webinar 
November 2nd 2022 @7 am-8:30am Chicago time (CDT) 
Webinar: <a href="https://tinyurl.com/uicwebinar2022">https://tinyurl.com/uicwebinar2022</a> 
See what the Department of Computer Science at the University of Illinois Chicago has to offer! Located in the beautiful city of Chicago, UIC is among the most culturally diverse universities in the United States, and home to one of the most innovative computer science departments in the United States. UIC is Chicago&rsquo;s top public research university, and was recently ranked in the top 20 of the best 100 universities worldwide that are under 50 years of age. Doctoral students in their first five years of enrollment are guaranteed a fully funded appointment that includes a tuition waiver, and a stipend. Students are also eligible for summer support. Our students get jobs in academia, industry, and entertainment upon graduation, including positions at top research labs and companies such as Google, Microsoft, Apple, Disney, and Amazon. 
During this webinar, UIC Computer Science faculty will talk about our research agenda, application fee waivers, the application process, application expectations in terms of GPA/GRE etc, and will take questions. A recording of the webinar will be made available on our Graduate Admissions page. 
Apply: <a href="https://cs.uic.edu/graduate/admissions/">https://cs.uic.edu/graduate/admissions/</a> 
Deadline: December 15th 2022 
See our flyer at: <a href="https://tinyurl.com/uic2022flyer">https://tinyurl.com/uic2022flyer</a> 
 <iframe width="420" height="315"src="https://www.youtube.com/embed/eNoBbCm1skM">
</iframe>

Interested in pursuing a PhD in Computer Science in the United States?

UIC <a href="https://cs.uic.edu/undergraduate/cs-and-design/">Engineering&rsquo;s News</a> promotes the new and very popular computer science + design undergraduate degree. UIC is the only public university offering this interdisciplinary degree, which addresses the growing trend and demand for creative technologists in the job market. The CSX program, as the computer science + design degree is also known will teach topics from augmented and virtual reality to 3D fabrication. 
To see more of the Engineering&rsquo; newsletter and activities featured as the Dean&rsquo;s list see: 
<a href="https://emails.uofi.uic.edu/newsletter/669921009.html">https://emails.uofi.uic.edu/newsletter/669921009.html</a>

Augmented Reality

Bibliography

pubs (10)

Related Topics

art (3)

people (1)

events (15)

research (2)

Contact