A Cyber-Commons is a technology-enhanced meeting room on a university campus that supports local and distance collaboration and promotes group-oriented problem solving. It is a next-generation computer science resource that relies upon advanced networking and multiple high-definition (HD) displays to transform the traditional computer lab / classroom filled with terminals - to a work environment that facilitates and encourages group collaboration. 

Envisioned by EVL director Jason Leigh and Computer Science faculty member Andy Johnson, the space is designed to connect students with advanced technologies and tools in virtual space with one another. The Cyber-Commons is a direct outgrowth of EVL&rsquo;s collaborative research focused on the development of technologies and tools for scientists linked via high-speed networks. 

The Cyber-Commons is an experimental space that will likely evolve as student needs change and technology advances. The current Cyber-Commons installation in the Computer Science department at UIC is equipped and networked to run HD videoconferencing and cabled to allow students to plug personal laptops into the HD displays. Students can now see and talk to remote collaborators in near life-size using an HD video conferencing system, and simultaneously share information to work on joint projects. 

The components of the Cyber-Commons can be altered and enhanced as needed; increasing network bandwidth and the number of HD displays could enable more participants and more interaction with high-resolution scientific images stored in distributed databases. 

The goal of the Cyber-Commons is to teach students how to collaborate - to solve problems together - both within a university and between universities. Rapid proliferation of Cyber-Commons to computer science departments at other institutions is anticipated over the next five years.

Students collaborating in EVL Cyber-Commons.

Cyber-Commons

Exploring big cities like Chicago can be a daunting experience when the tourist has to gather information from multiple sources regarding places, directions, transportation, safety, and weather.  We propose a large display solution (kiosks) which aims to encourage people to explore the city by showing cultural attractions, special events and other points of interest (POI) on a map along with multi-modal transportation options to take them to their destination. POI&rsquo;s and events are updated frequently based on the time and safety information. Kiosks are intended to be located throughout the city especially locations with the highest density of points of interest.

Screen shot of the &ldquo;Exploring Chicago&rdquo; application prototype.

Exploring Chicago

UIC Computer Science mentors two DREU undergraduates to work on summer research project 
JULY 27, 2017 - Undergraduate students Taylor Day, from Texas A&M University, College Station, and Mariko Kamiya, from Swarthmore University, were selected to be part of this summer&rsquo;s Distributed Research Experiences for Undergraduates (DREU) program, sponsored by the Computing Research Association Committee on the Status of Women in Computing Research (CRA-W) and the Coalition to Diversify Computing, with support from the National Science Foundation. The DREU program provides undergraduates with the opportunity to be directly involved in a research project and interact with graduate students and professors on a daily basis. 
DREU matched the students with faculty mentor Debaleena Chattopadhyay, an assistant professor in the UIC Computer Science Department and its Electronic Visualization Laboratory (EVL). Given her expertise in human-computer interaction, she assigned Day and Kamiya a project: to study smartphone ease of use by older adults. More specifically, how accessible are smartphone map applications for people aged 65 and older? And, is there a better way to design a mobile map application that considers the effects of old age, such as decreased motor skills, worsening eyesight, and limited cognitive processing skills? This is an area that is little studied, yet 42% of adults aged 65+ own a smartphone in the U.S. in 2017. 
In their 10-week summer internship, Day and Kamiya conducted a think-aloud user study with more than 12 older adults (for which they had applied for, and received, Institutional Review Board [IRB] approval), analyzed design requirements, learned web development, and prototyped a better application that integrated a probabilistic function to combat limited fine-motor skills during panning and zooming touchscreen maps. In what little spare time they had left, they even managed to explore Chicago a bit. 
Though Day and Kamiya gave a presentation of their findings to other EVL students, staff and faculty on the last day of their internship at UIC, they are far from done. Chattopadhyay has plans for them to write and submit a short paper to a professional conference. And, Day submitted a poster proposal to the Grace Hopper Celebration of Women in Computing conference on her mathematical contributions to the mobile application prototype, which was accepted. Subsequently, she applied to the CRA-W Grace Hopper Celebration Research Scholars Program for travel support, and will find out if she is accepted in mid-August. 
The DREU program requires its students to keep a Blog of their experiences. To learn more about Day&rsquo;s and Kamiya&rsquo;s experiences doing research at UIC and exploring the city of Chicago, see: 
<a href="https://tmacday.wordpress.com/">Taylor Day Blog</a> 
<a href="https://mkamiyadreu.wordpress.com/">Mariko Kamiya Blog</a> 
For more information on the DREU program, see: <a href="http://cra.org/cra-w/dreu/">http://cra.org/cra-w/dreu/</a>

DREU students Mariko Kamiya (left), Taylor Day (right) with faculty mentor Debaleena Chattopadhyay, UIC Computer Science/EVL assistant professor (center); project "Understanding Accessibility of Touchscreen Maps for Older Adults,” Summer 2017

Understanding Accessibility of Smartphone Map Applications for Older Adults

Just as NSF cyberinfrastructure produces greater volumes and varieties of data from data storage systems, online instrumentation and major computational resources like XSEDE and Blue Waters, advanced visualization instruments serve as the &ldquo;lenses&rdquo; of these data generators (similar to the telescopes and microscopes of yore), enabling researchers to peer into cyberspace to manage, access and analyze big data. 
Continuum is a cyberinfrastructure instrument that unifies high-resolution computer-enhanced group collaboration workspaces with arrays of ambient sensors, enabling users to collaborate with local and remote colleagues and their data more effectively by giving the room itself the ability to anticipate their needs, improve their interaction, and focus their time on discovery rather than the technology. Continuum is a digital assistant, benefiting global scientific collaboratories and providing a foundation for new computer science research. War rooms benefit collaboration because they support continuous interaction and awareness of team members, enable persistent information sharing, and encourage mobility among team members and information. A computer-enhanced war room, such as Continuum, which (1) connects distributed teams with their data, whether stored in the cloud or streamed from major computational resources over high-speed networks, (2) supports persistent digital artifacts, (3) enables ultra-high-resolution 2D and 3D stereoscopic digital media to be simultaneously viewed, and (4) supports ubiquitous and intuitive interaction, creates a powerful and easy-to-use information-rich environment in support of scientific discovery. 
Continuum is the culmination of the University of Illinois at Chicago’s (UIC) 20+ years experience developing interactive instruments, from the original CAVE in 1992, to the ultra-high-resolution LambdaVision tiled LCD wall in 2004, and the hybrid-reality CAVE2 in 2012. Each new generation of visualization instrumentation provided scientific communities with more advanced features Continuum development focuses on three areas: (1) transforming passive display walls that respond to user commands into attentive display walls that monitor the users in a room and anticipate their needs to improve their experience; (2) providing an alternative approach to constructing high-resolution visualization display walls using thin-border 4K passive stereo OLED displays that have impressive brightness, contrast, uniformity, off-axis viewing, and low reflectivity in brightly lit rooms; and (3) utilizing cloud computing to prototype future Continuum spaces that don’t require local high-end compute and display resources. Twelve funded research projects from local institutions, in Astronomy, Astrophysics, Biology,
Combustion, Geophysics, Neuroscience, Pathology, Physics, Psychiatry, and High Performance Computing, are poised to use Continuum for their data visualization needs. Continuum will open up new opportunities in computer science research at the intersection of large-scale data visualization, human-computer interaction, and high-speed networking. Continuum will also directly support 10 classes taught in the UIC Computer Science, Art and Design, and Biomedical Science departments.

Artist's rendering of Continuum space.

MRI - Development of Continuum: A Virtualized Attentive Environment for Amplified Collaboration

Although visualization design models exist in the literature in the form of higher-level methodological frameworks, these models do not present a clear methodological prescription for the domain characterization step. This work presents a framework and end-to-end model for requirements engineering in problem-driven visualization application design. The framework and model are based on the activity-centered design paradigm, which is an enhancement of human-centered design. The proposed activity-centered approach focuses on user tasks and activities, and allows an explicit link between the requirements engineering process with the abstraction stage-and its evaluation-of existing, higher-level visualization design models. In a departure from existing visualizationdesign models, the resulting model: assigns value to a visualization based on user activities; ranks user tasks before the user data; partitions requirements in activity-related capabilities and nonfunctional characteristics and constraints; and explicitly incorporates the user workflows into the requirements process. A further merit of this model is its explicit integration of functional specifications, a concept this work adapts from the software engineering literature, into the visualization design nested model. A quantitative evaluation using two sets of interdisciplinary projects supports the merits of the activity-centered model. The result is a practical roadmap to the domain characterization step of visualization design for problem-driven data visualization. Following this domain characterization model can help remove a number of pitfalls that have been identified multiple times in the visualization design literature. 
Index Terms - Design studies, Tasks and requirements analysis, Visualization models, Domain characterization, Activity-centered design, Functional specifications 
<iframe src="https://player.vimeo.com/video/230835822" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
<a href="https://vimeo.com/230835822">[VIS17 Preview] Activity-Centered Domain Characterization for Problem-Driven Scientific Visualization</a> from <a href="https://vimeo.com/vgtcommunity">VGTCommunity</a> on <a href="https://vimeo.com">Vimeo</a>.

Activity-Centered Domain Characterization for Problem-Driven Scientific Visualization

The <a href="http://hci.cs.uic.edu/">Human-Computer Interaction (HCI) Laboratory</a> at the University of Illinois at Chicago (UIC) is led by assistant professor Debaleena Chattopadhyay of the Department of Computer Science. The HCI Lab designs computer technologies to transform human experiences and deliver positively impactful societal change. 
UIC HCI is an affiliate laboratory of the Electronic Visualization Laboratory (EVL) providing a human centered approach to the development of tools and technologies related to advanced visualization, virtual reality, visual analytics and networked collaboration. 
The HCI Lab&rsquo;s mission is to support individual and social activities and improve everyday life, designing, building and evaluating ubiquitous computing technologies. The design of these systems draws from research on how people work, play, and communicate in day-to-day life. 
The UIC HCI Lab is located on the UIC campus at: 
Science & Engineering Laboratory West Room 1032 
900 West Taylor Street 
Chicago, IL 60607

UIC Human-Computer Interaction (HCI) Laboratory

Drug-drug interaction (DDI) alerts safeguard patient safety during medication prescribing, but are often ignored by physicians. Despite attempts to improve the usability of such alerts, physicians still mistrust the relevance of simplistic computerized warnings to support complex medical decisions. By building on prior fieldwork, this paper evaluates novel designs of trust - eliciting cues in DDI alerts. A sequential mixed-method study with 70 physicians examined what trust cues improve compliance, promote reflection, and trigger appropriate actions. In a survey, 52 physicians rated the likelihood of compliance and usefulness of redesigned alerts. Based on these findings, alerts were assessed in a scenario-based simulation with 18 physicians prescribing medications in 6 patient scenarios. Our results show that alerts embodying expert endorsement, awareness of prior actions, and peer advice were less likely to be overridden than current alerts, and promoted reflection, monitoring, or order modifications - thus building towards greater attention to patient safety. 
Keywords: healthcare information systems; health informatics; clinical alerts; trust in socio-technical systems; interaction design; decision-making 
<ahref="https://doi.org/10.1093/iwc/iwx020">doi:10.1093/iwc/iwx020</a>

DDI alerts were displayed in an experimental setting that simulated an inpatient meeting room where physicians use a workstation to prescribe medications. Interruptions were simulated using a second computer and screen interactions recorded using Camtasia

Design and Evaluation of Trust - Eliciting Cues in Drug - Drug Interaction Alerts

Traditional models of slideware assume one presenter controls attention through slide navigation and pointing while a passive audience views the action. This paradigm limits group interactions, curtailing opportunities for attendees to use slides to participate in a collaborative discourse. However, as slideware permeates contexts beyond simple one-to-many presentations, there are growing efforts to shift the dynamics of collocated interactions. Technologies exist to shift the one-to-many information control paradigm to variations that extend functions to multiple attendees. But there is limited detailed research on how to design such multi-person attentional control and facilitate collocated interactions without disrupting existing work practices. We report on a detailed naturalistic case of using presentation in a design meeting, where participants used Office Social, an experimental slideware technology that enabled open access to shared interaction with slides across multiple devices. We explore how the design of Office Social supported informal collaboration. Our video-based analysis reveals how the orderly structures ofconversational turn-taking and bodily conduct were used in conjunction with theaffordances of the socio-technical ecosystem to organize collective activity. We suggest that supporting collocated interactions should take account of the existing conversational methods for achieving orderly collaboration rather than superimposing prescriptive technological methods of order. 
Keywords: Presentation interactivity; Interaction design; Audience participation; Collocated interaction; Collaboration; Open systems; Design meeting 
<a href="https://doi.org/10.1080/07370024.2017.1388170">doi.org/10.1080/07370024.2017.1388170</a>

Office Social is designed as a PowerPoint add-in (A) and a companion smartphone app (B). When a slideshow is launched, the slide on the laptop screen is mirrored on the presenter’s smartphone and the public display.

Beyond presentation: Shared slideware control as a resource for collocated collaboration

The uncanny valley hypothesis predicts that an entity appearing almost human risks eliciting cold, eerie feelings in viewers. Categorization-based stranger avoidance theory identifies the cause of this feeling as categorizing the entity into a novel category. This explanation is doubtful because stranger is not a novel category in adults; infants do not avoid strangers while the category stranger remains novel; infants old enough to fear strangers prefer photographs of strangers to those more closely resembling a familiar person; and the uncanny valley&rsquo;s characteristic eeriness is seldom felt when meeting strangers. We repeated our original experiment with a more realistic 3D computer model and found no support for categorization-based stranger avoidance theory. By contrast, realism inconsistency theory explains cold, eerie feelings elicited by transitions between instances of two different, mutually exclusive categories, given that at least one category is anthropomorphic: Cold, eerie feelings are caused by prediction error
from perceiving some features as features of the first category and other features as features of the second category. In principle, realism inconsistency theory can explain not only negative evaluations of transitions between real and computer modeled humans but also between different vertebrate species. 
Keywords: Anthropomorphism; Computer animation; Face perception; Novelty; Stranger avoidance 
<a href="http://dx.doi.org/10.1016/j.cognition.2016.09.001">dx.doi.org/10.1016/j.cognition.2016.09.001</a>

Stimuli ratings of subjective realism, eeriness, warmth, and familiarity are plotted against their fraction of real as are percentage categorized as real and response time.

Categorization-based stranger avoidance does not explain the uncanny valley effect (2016)

Computer-modeled characters resembling real people sometimes elicit cold, eerie feelings. This effect, called the uncanny valley, has been attributed to uncertainty about whether the character is human or living or real. Uncertainty, however, neither explains why
anthropomorphic characters lie in the uncanny valley nor their characteristic eeriness. We propose that realism inconsistency causes anthropomorphic characters to appear unfamiliar, despite their physical similarity to real people, owing to perceptual narrowing. We further propose that their unfamiliar, fake appearance elicits cold, eerie feelings, motivating threat avoidance. In our experiment, 365 participants categorized and rated objects, animals, and humans whose realism was manipulated along consistency reduced and control transitions. These data were used to quantify a Bayesian model of categorical perception. In hypothesis testing, we found reducing realism consistency did not make objects appear less familiar, but only animals and humans, thereby eliciting cold, eerie feelings. Next, structural equation models elucidated the relation among realism inconsistency (measured objectively in a two-dimensional Morlet wavelet domain inspired by the primary visual cortex), realism, familiarity, eeriness, and warmth. The fact that reducing realism consistency only elicited cold, eerie feelings toward anthropomorphic characters, and only when it lessened familiarity, indicates the role of perceptual narrowing in the uncanny valley. 
doi:10.1167/16.11.7

Objects, animals, and humans constitute the low, intermediate, and high anthropomorphism groups, respectively. The right half of a photograph of each entity is shown beside the left half of its 3-D computer model.

Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley

BS Graduate, 2022 Research Experience Undergraduate (2018-2021)

Kenan Arica

Undergraduate Researcher (REU), BS 2018

Kayla Condrella

Pollution exposure assessment at the population level is an established enterprise for environmental scientists and public health officials - but efforts to help individuals monitor and track their personal pollution exposures have just begun to garner research interest. Self-tracking pollution exposure is challenging for several reasons, including current limitations in sensor size, accuracy, and cost, frequent calibration requirements, and that people&rsquo;s daily activities often interfere with data quality in wearable sensing. The goal of this research is to develop a human-centered computing framework for the emerging field of personal pollution exposure assessment. To that aim, in this position paper, we propose a Bayesian approach to combine environmental sensing data from different spatiotemporal resolutions, such as from citywide national monitoring stations, neighborhood-wide lightweight sensing nodes, and personal wearables. 
Author Keywords: Environmental sensing; pollution exposure profiling; self-tracking; pollution monitoring.

The phone application of myCityMeter, lookup (a), personal exposure profiles (b), and suggested actions (c).

Toward a Bayesian Approach for Self- Tracking Personal Pollution Exposures

Recent epidemiological studies suggest that age-related cognitive decline - particularly, the stages between normal cognitive changes in aging and early dementia - is adversely affected by environmental exposures, such as long-term air pollution and traffic noise. Although monitoring outdoor air pollution is now commonplace, and smart home solutions for monitoring indoor air quality is becoming prevalent, ways that the elderly can record long-term environmental exposures and adopt healthy lifestyle changes are little explored. We present myCityMeter, a pollution exposure management tool for older adults and their caregivers. myCityMeter measures the pollutants shown to be associated with cognitive impairment in older adults: PM2.5 and ambient noise. Using a set of neighborhood-level stationary and personal mobile sensors, myCityMeter helps users to monitor their environmental exposures, know potential exposures when planning activities, journal cognitive performances, and take day-to-day actions to avoid the environmental risk factors for early dementia. 
Author Keywords: Older adults; pollution exposure management; pollution monitoring; cognitive impairment; environmental sensing; pm2.5; noise; array of things.

An overview of myCityMeter, a personal pollution exposure monitoring tool that collects data from three monitoring sources.

myCityMeter: Helping Older Adults Manage the Environmental Risk Factors for Cognitive Impairment

Pi: Barbara Di Eugenio; Co-I: Debaleena Chattopadhyay 
A growing body of research has found that patients who actively manage their health have better outcomes, and patient engagement has become a central tenet in recent healthcare approaches. However, it has also been shown that many patients lack knowledge of their disease and medications, particularly as concerns chronic health conditions, whose exacerbation causes about two thirds of hospitalizations every year in the USA. Discharge instructions after hospitalization are a critical tool to engage patients in their self-care, but are often unsuccessful. Our hypothesis is that, to engage patients at discharge, a personalized hospitalization summary should be provided, that integrates physician and nurse documentation, and is informed by the patient's level of engagement in self-care and their socio-cultural perspective. 
Our transformative pilot work on physician and nurse documentation has identified an additional obstacle to the effective flow of information from professionals to patients: vocabulary differences between physicians and nurses. By leveraging the Unified Medical Language System Metathesaurus, we found no common synonyms between physician and nurse documentations for 75% of patients. Further, 86% of terms patients use are simple compared to only 15% for physicians. The lack of common language among patients, physicians and nurses motivates including all perspectives to make instructions meaningful for patients. Our hypothesis is that a better informed patient population who understands the connections between their illness, hospital experience, and discharge instructions will be more empowered and more engaged in their chronic disease management. Our innovation is to automatically create an easy-to-understand, personalized hospitalization summary that integrates events described in the EHR; and that is responsive to the patient's needs and preferences, and their activation framework (PAF) level. We pursue four specific aims: 1. To investigate socio-cultural patterns of patient perceptions of illness via a socio linguistic analysis of patient interviews; 2. To integrate multi-faceted information about a patient, as provided by physicians and nurses; 3a. To develop MyPHA, a novel application program, which will automatically generate the personalized summaries and deliver them to the patient on a tablet. MyPHA will be informed by aims 1 and 2, and will be grounded in Natural Language Generation technology; 3b. To evaluate MyPHA with multiple cycles of formative evaluation with all stakeholders, and a summative evaluation with patients. 
Whereas our pilot work has focused on heart failure, the techniques we will develop can easily be adapted to the management of other chronic health conditions, in particular for patients with cancer and cancer survivors. In addition, cancer and heart failure can co-occur, especially in older patients, and several cancer treatments can lead to the development of heart failure. The software program MyPHA will be released ooen source to the research communitv to enable raoid customization to other chronic conditions. 
Awarded Amount: $1,495,213

NIH: R01: SCH: INT: MyPHA: Automatically generating personalized accounts of inpatient hospitalization

OCTOBER 18, 2018 - &ldquo;Herstory: Digital Innovations Symposium and Book Signing,&rdquo; celebrates the book &ldquo;New Media Futures (NMF): The Rise of Women in the Digital Arts.&rdquo; The Symposium will take place at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UIUC), 1205 West Clark, Room 1030, Urbana, IL, on October 26, 2018, from 8:30am - 2:00pm. 
The NMF chronicles 22 women who went to school or worked at UIUC, University of Illinois at Chicago (UIC), and the School of the Art Institute of Chicago, in the 1980s and beyond, during a pivotal time for the digital arts media in the Midwest. Nine of the 22 women profiled in the book are PhD/MS/MFA/BFA graduates and/or employees of the UIC Electronic Visualization Laboratory (EVL), which, at the time, was a formal collaboration between the UIC Computer Science department and UIC School of Art and Design. 
Thirteen of the book&rsquo;s contributors will attend the Symposium - including the woman who designed the first graphical web browser, the first female gaming company founder, and the woman who helped build the first room-sized virtual reality room environment - and will give TED-inspired Lightning Talks, and meet with attendees. They will discuss their passions, digital innovations and their herstories - both formally in short presentations and informally in small group breakouts - to inspire and mentor a young and diverse audience. Seven of the speakers have an affiliation with EVL: Maxine Brown, Carolina Cruz-Neira, Margaret Dolinsky, Copper Giloth, Dana Plepys, Mary Rasmussen and Barbara Sykes. 
For more information about the Symposium, see the flyer below and <a href="http://avl.ncsa.illinois.edu/events/herstory-digital-innovations-symposium">go to the website.</a> 
<a href="https://www.evl.uic.edu/entry.php?id=2369">More information about the NMF book.</a>

UIC Electronic Visualization Laboratory Graduates and Staff Participate in Symposium &ldquo;Herstory: Digital Innovations&rdquo;

OCTOBER 22, 2018 - UIC EVL/MFA graduate JD Pirtle recently started his own company, <a href="https://www.depthandlight.com/">Depth and Light</a> which is an education, technology, and design studio+consultancy. Its mission is to help students, teachers, and learners of all ages feel confident using traditional and emerging technology for research, inquiry, and creativity. 
JD, as an EVL/MFA student, helped develop both the &ldquo;20 Foot Virtual Canvas&rdquo; multi-touch paint system and the &ldquo;Mega Tenori-on&rdquo; virtual sound instrument for EVL&rsquo;s large-scale tiled display wall, called Cyber-Commons. 
After graduating UIC, EVL retained JD to complete development of the CAVE2 sound library. He was then hired by Chicago&rsquo;s Catherine Cook School (preschool - 8th grade) as Director of Innovation, and promptly created IDEA (Innovation, Design, Engineering, Art) Lab, a school-wide, cross-divisional program that included 3D printers, a laser cutter/engraver, a large interactive display wall running EVL/LAVA&rsquo;s SAGE2 software, robotics, sewing equipment, and traditional shop tools. He ran several successful IDEA:TE Transforming Education workshops, too, targeted for preschool - 12th grade educators to discuss how best to integrate traditional and emerging technologies into their curriculum. 
Now JD brings us Depth and Light. As the website says &ldquo;Technology is magical, but it's not magic.&rdquo; JD is certainly a wizard behind the curtain, and we wish him luck in his new venture!

JD Pirtle presenting at the IDEA:TE 2015 workshop

UIC EVL/MFA Graduate JD Pirtle Starts Depth and Light Consultancy Firm

MS, 2019

Mirko Mantovani

This article provides a 25 year-long perspective on Immersive Analytics through the lens of first-in-kind technological advancements introduced at the Electronic Visualization Laboratory, University of Illinois at Chicago, along with the challenges and lessons learned from multiple Immersive Analytics projects. 
The Electronic Visualization Lab (EVL) at the University of Illinois at Chicago has been at the forefront of Virtual Reality research since 1992, when it introduced the CAVE&trade; Automatic Virtual Environment, the first projection-based virtual reality system in the world. Since then, the lab has developed a wide range of immersive systems and technologies to be used for visualization and visual analysis tasks, such as PARIS&trade;, ImmersaDesk&trade;, CAVE2&trade;, and SAGE2&trade;. These technologies paved the way for the resurgence in virtual reality in the past few years, and lead to the introduction of Immersive Analytics - which investigates how new interaction
and display technologies can be used to support analytical reasoning by immersing the users in data.

CAVE2 immersive exploration of dark matter formation in the universe. While little information can be shown on a typical desktop display, the high-resolution tiled environment lets analysts examine details without losing the context of the larger dataset.

Immersive Analytics Lessons from the Electronic Visualization Laboratory: a 25 Year Perspective

BS Graduate, 2023 Research Experience Undergraduate (REU) 2019-2023

Zhongyi Chen

Research Experience Undergraduate (REU), 2019

Sumayya Siddiqui

Research Experience Undergraduate (REU)

Szymon Popowski

Research has shown that compliance to interactive training in tasks such as movement therapy following injuries, stroke and major surgeries is sub-optimal. Efforts are being made to make the rehabilitation process a more engaging experience to motivate patients to adhere to therapy. The use of games and virtual reality has gained interest among researchers and clinicians as it may cost less than clinical therapy and can be made portable for use at home. However, without the encouragement of a physical therapist, motivating patients may be difficult. In this paper, we propose a virtual slot machine game and hypothesize that it can potentially make individuals increase time devoted to therapy. Since this game requires minimal skill for gameplay, patients can simply reach a target to trigger a reel spin. We propose that the anticipation and uncertainty of this game will capture patients’ interest and motivate them to play longer. This preliminary study with able-bodied individuals may be used to tune the testing designs for future efforts on patient populations that need to repetitively practice arm movements. 
Keywords - rehabilitation, exercises, games, slot machines

A user testing the game and the interface.

Virtual Slots Game for Rehabilitation Exercises

PhD Candidate: Anthony Perritano 
Date: Friday, September 6th, 2019 
Time: 10:30 am CT 
Location: Electronic Visualization Laboratory (EVL), Room 2068 ERF (Continuum) 
Committee: 
Tom Moher (Chair) 
Andrew Johnson 
Chris Kanich 
Joel Brown, Biology 
James Slotta, University of Toronto 
Yvonne Rogers, University College London 
Abstract: 
An important research question concerns the nature of Human-Computer Interaction (HCI) methods and supports for improving students&rsquo; access to, application of, and general interactions with the community knowledge in a learning community approach. This thesis investigates a novel approach to supporting classroom learning communities through the use of embodied interaction and ambient visualizations. Specifically, community knowledge is embedded within the physical space of the classroom, with the aim of mediating opportunistic inter-group interactions, instigated through proximity and shared artifacts. This approach entails decomposing the community knowledge-base into a collection of independent thematic sub-stores, and then conceptually distributing those sub-stores to mapped, demarcated locations around the classroom, called &ldquo;Knowledge Places.&rdquo; This necessitates physical movement among and proximity to those places in order for students to contribute to or otherwise access their peers&rsquo; contributions to the emerging knowledge-base. This investigation was embedded within a larger NSF-funded study and was enacted over the course of a semester within a sixth-grade classroom as part of the regular life science curriculum (e.g., food webs, ecosystems).

Students Working at Knowledge Place

PhD Thesis Defense Announcement: &ldquo;Knowledge Places: Embedding Knowledge in the Space of the Classroom&rdquo;

This research investigates a novel approach to supporting classroom learning communities through the use of proxemic interaction and ambient visualizations. Specifically, community knowledge is embedded within the physical space of the classroom, with the aim of mediating opportunistic inter-group interactions, instigated through proximity and shared artifacts. This approach entails decomposing the community knowledge-base into a collection of independent thematic sub-stores, and then conceptually distributing those sub-stores to mapped, demarcated locations around the classroom, called &ldquo;Knowledge Places&rdquo;. This necessitates physical movement among and proximity to those places in order for students to contribute to or otherwise access their peers&rsquo; contributions to the emerging knowledge-base. The present research studies the materialization of Knowledge Places over the course of ten weeks within a sixth-grade life science curriculum, with topics of food webs and ecosystems. 
Keywords: Embodied Interaction, proxemic Interaction, ambient visualization, RFID, indoor location, Ubiquitous Computing; Internet Of Things; IOT 
ISBN: 978-1-4503-6690-8

Students composing contributions at ecosystems on group tablets.

Knowledge Places: Embedding Knowledge in the Space of the Classroom

This paper describes the design and pilot enactment of an instructional unit for elementary school students, Hunger Games, which centers on development of learner understandings of animal foraging behavior. Inspired by traditional teaching practices employing physical simulations, within the unit students engage in an embodied enactment of foraging using stuffed animals (with embedded RFID tags) as tangible avatars to represent their foraging among food patches (with camouflaged RFID readers) distributed around a classroom. Displays situated near the food patches provide students with information regarding the energy gain as the forage in the environment. A two-period pilot enactment of the unit demonstrated the feasibility of the design for classroom use, evidenced the development of affective relationships between learners and avatars, and afforded the emergence of unanticipated behaviors that promoted new questions about the science phenomena. The results suggest provisional support for the effectiveness of the unit as a science learning environment. 
Keywords: Indoor location, RFID, visualization, game theory, education, learning 
ISBN: 978-1-4503-2635-3

Students checking on their total energy gain over the course of a foraging bout.

Back to the Future: Embodied Classroom Simulations of Animal Foraging

We describe a visual computing approach to radiation therapy (RT) planning, based on spatial similarity within a patient cohort. In radiotherapy for head and neck cancer treatment, dosage to organs at risk surrounding a tumor is a large cause of treatment toxicity. Along with the availability of patient repositories, this situation has lead to clinician interest in understanding and predicting RT outcomes based on previously treated similar patients. To enable this type of analysis, we introduce a novel topology-based spatial similarity measure, T-SSIM, and a predictive algorithm based on this similarity measure. We couple the algorithm with a visual steering interface that intertwines visual encodings for the spatial data and statistical results, including a novel parallel-marker encoding that is spatially aware. We report quantitative results on a cohort of 165 patients, as well as a qualitative evaluation with domain experts in radiation oncology, data management, biostatistics, and medical imaging, who are collaborating remotely. 
Keywords: Biomedical and Medical Visualization, Spatial Techniques, Visual Design, High-Dimensional Data 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Screen shot of visual interface for cohort-based radiation therapy (RT) prediction results.

Cohort-based T-SSIM Visual Computing for Radiation Therapy Prediction and Exploration

PhD Graduate, 2025

Carla Floricel

Background: In virtual reality (VR) applications such as games, virtual training, and interactive neurorehabilitation, one can employ either the first-person user perspective or the third-person perspective to perceive the virtual environment; however, applications rarely offer both perspectives for the same task. We used a targeted-reaching task in a large-scale virtual reality environment (N = 30 healthy volunteers) to evaluate the effects of user perspective on the head and upper extremity movements, and on user performance. We further evaluated how different cognitive challenges would modulate these effects. Finally, we obtained the user-reported engagement level under the different perspectives. 
Results: We found that first-person perspective resulted in larger head movements (3.52 ± 1.3m) than the third-person perspective (2.41 ± 0.7m). First-person perspective also resulted in more upper-extremity movement (30.08 ± 7.28m compared to 26.66 ± 4.86m) and longer completion times (61.3 ± 16.4s compared to 53 ± 10.4s) for more challenging tasks such as the &ldquo;flipped mode&rdquo;, in which moving one arm causes the opposite virtual arm to move. We observed no significant effect of user perspective alone on the success rate. Subjects reported experiencing roughly the same level of engagement in both first-person and third-person perspectives (F(1.58) = 0.9, P = .445). 
Conclusion: User perspective and its interaction with higher-cognitive load tasks influences the extent of movement and user performance in a virtual theater environment, and may influence the choice of the interface type (first or third person) in immersive training depending on the user conditions and exercise requirements. 
Keywords: User perspective, Engagement, CAVE2

View of the CAVE2, displaying the third-person perspective of a task. 72 high-resolution displays are arranged in 18 columns in a circular fashion, providing an area of 320 degrees of screen space with a resolution of 36 Megapixels for each user eye.

User perspective and higher cognitive task-loads influence movement and performance in immersive training environments

Ph.D. Student: Jillian Aurisano 
Date: Tuesday, September 17, 2019 
Time: 11:00 am 
Location: Room 3036, Engineering Research Facility 
Committee: 
Andrew Johnson (Chair) 
Debaleena Chattopadhyay 
Barbara Di Eugenio 
G. Elisabeta Marai 
Rick Stevens (Argonne National Laboratory, University of Chicago) 
Abstract: 
Analysis of large, complex datasets stands to benefit from environments that permit users to view and juxtapose many views of data. Large, high-resolution environments are capable of showing many related views of data, but interaction with these views poses significant challenges in visual and interaction design. In this talk, I will present work toward &ldquo;multi-view interactions&rdqo; that enable users to create, organize and act on many views at once, through multi-modal speech+gesture queries in large flexible canvas environments. The goal is the enable users to rapidly and efficiently generate sets of views in support of multi-view analysis tasks, organize these views to reflect changing analysis goals, and operate on sets of views collectively, rather than individually, to efficiently reach large portions of the &lsquo;data+attribute space&rsquo;. I will implement and evaluate this approach within two domains and two user communities: City of Chicago data for informed citizens and deep learning predictions data for precision medicine in cancer.

Preliminary Examination Announcement: &ldquo;Multi-view, multi-modal speech+gesture interaction in large display environments&rdquo;

In head and neck (HN) cancer treatment, radiotherapy (RT) plans assigned to a given patient depend in large measure on the tumor location with respect to surrounding organs at risk (OARs). With the emergence of Big Data repositories, RT data from patients with
similar tumor locations could be used to predict the RT plan for a new patient, as opposed to clinicians or institutional memory alone. 
We propose an automated technique to evaluate RT plan similarity between HN cancer patients who have previously received radiotherapy, based on similarities in their tumor location with respect to OARs. We use this technique to predict and evaluate the RT dose distribution across organs in a new HN patient, using similar patients as a prior. 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Cohort-Based Spatial Similarity Can Predict Radiotherapy Dose Distribution

Imaging is an integral tool in radiation oncology, starting from initial diagnosis to follow-up post-therapy. In head and neck
cancer treatment, toxicity and tumor control depend on the tumor locational geometry and systematic predictable spread of disease to
affected lymph nodes (LN) levels. However, a rigorous methodology for integrating imaging spatial information into treatment outcome
prediction models remains an unmet need. We hypothesized that integrating tumor and LN spatial information derived from routine diagnostic images can aid in prediction model development for both oncologic outcomes and post-therapy toxicity. 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Correlating Toxicity Outcomes with Spatial Patterns of Lymph Node Metastasis for Oropharyngeal Cancer Patients

PhD Graduate Student

Sanjana Srabanti

PhD Candidate

Abhinav Kumar

SAGE (the Scalable Adaptive Graphics Environment) and its successor SAGE2 (the Scalable Amplified Group Environment) are operating systems for managing content across wideband display environments. This paper documents the prevalent usage patterns of SAGE-enabled display walls in support of the e-Science enterprise, based on nearly 15 years of observations of the SAGE community. These patterns will help guide e-Science users and cyberinfrastructure developers on how best to leverage large tiled display walls, and the types of software services that could be provided in the future. 
Keywords: tiled display wall, immersive analytics, visualization, human-centered computing, computer-supported cooperative work

The e-Science Research, Development and Training Enterprise.

Usage Patterns of Wideband Display Environments In e-Science Research, Development and Training

SAGE2 is a user-centered platform for small groups or large distributed teams to access digital media datasets from various sources and juxtapose, share and investigate content on tiled display walls. SAGE2 builds on SAGE (Scalable Adaptive Graphics Environment), developed with prior NSF funding. SAGE2 is having a transformative effect on data exploration, visualization and collaboration, making cyberinfrastructure more accessible to end-users, both in the laboratory and in the classroom. 
<a href="https://doi.org/10.6084/m9.figshare.12837638">2020 Poster: https://doi.org/10.6084/m9.figshare.12837638</a> 
<a href="https://doi.org/10.6084/m9.figshare.12837638">2020 PPT: https://doi.org/10.6084/m9.figshare.12837695</a>

SAGE2 Scalable Amplified Group Environment (poster 2020)

Ph.D. Student: Abeer Alsaiari 
Committee: 
Prof. Andrew Johnson, Chair and Advisor 
Prof. Joseph Michaelis 
Prof. Wei Tang 
Prof. Steve Jones (Department of Communication) 
Prof. Khairi Reda (Indiana University-Purdue University Indianapolis) 
Date: Monday, March 9th 
Time: 1:30 pm 
Location: ERF 3036 
Abstract: 
There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. This new thrust of multi-device interfaces supports new models for complex collaboration scenarios, and have great potential to support analysts in their data analysis by utilizing each device&rsquo;s capabilities. However, there are some challenges inherently associated with visual data analysis in multi-device environments (MDE). As the analytical process is underway, analytical components become scattered among different devices and displays. Building a mental model of the analysis flow can render the analytical process more challenging as it would be difficult to track many visualizations and therefore difficult for analysts to keep track of all the prior analyses. The primary goal of my thesis research is to understand the characterization of the analysis process, with regard to the use and formation around devices, and to support the design with visualization of the search space, which will allow users to efficiently explore their data.

Preliminary Examination Announcement: &ldquo;Supporting Collaborative Visual Data Analysis in Multi-device Environment&rdquo;

After graduating with his bachelor&rsquo;s degree in astrophysics from the University of Illinois at Urbana-Champaign, Ryan Nishimoto began his pursuit of blending his interest in astrophysics and computer science. He liked the Electronic Visualization Lab (EVL) and heard good things about the computer science graduate program. 
Upon acceptance into the graduate program, Ryan selected a coursework-only option for his degree. This afforded him the opportunity to participate in a number of project-based classes through EVL. One project, he collaborated with Adler Planetarium astronomer Aaron Gellar, and astronomer Mark SubbaRao, who also directs Adler&rsquo;s Space Visualization Laboratory, on virtual reality and displays to be used as educational tools in the museum. In addition, Ryan received a Research Assistantship under Professor Andy Johnson to work on developing virtual reality applications for patient rehabilitation with the Shirley Ryan AbilityLab. 
These experiences helped round out Ryan&rsquo;s graduate education for which upon graduating with his master&rsquo;s degree in Computer Science, he will begin a yearlong rotational program with Optum Pharma Inc., a consultancy company specializing in pharmaceuticals and healthcare insurance. 
<a href="https://cs.uic.edu/news-stories/get-to-know-computer-science-graduate-student-ryan-nishimoto/">View the complete Computer Science Department article featuring Ryan Nishimoto.</a>

EVL Research Assistant Ryan Nishimoto Featured as Computer Science Grad

This paper describes results from an observational, exploratory study of visual data exploration in a large, multi-view, flexible canvas environment. Participants were provided with a set of data exploration sub-tasks associated with a local crime dataset and were instructed to pose questions to a remote mediator who would respond by generating and organizing visualizations on the large display. We observed that participants frequently posed requests to cast a net around one or several subsets of the data or a set of data attributes. They accomplished this directly and by utilizing existing views in unique ways, including by requesting to copy and pivot a group of views collectively and posing a set of parallel requests on target views expressed in one command. These observed actions depart from multi-view flexible canvas environments that typically provide interfaces in support of generating one view at a time or actions that operate on one view at a time. We describe how participants used these &ldquo;cast-a-net&rdquo; requests for tasks that spanned more than one view and describe design considerations for multi-view environments that would support the observed multi-view generation actions. 
Index Terms: Human-centered computing - Empirical studies in visualization 
This work was supported initially by NSF award IIS 1445751 and currently by NSF award CNS 1625941. 
<a href="https://doi.org/10.1111/cgf.13976">https://doi.org/10.1111/cgf.13976</a>

A participant generates 15 views of her data. In the final request, the participant references a set of three views and poses a request to copy and pivot these views collectively to four new subsets of the data. The final result is a grid of views.

Many At Once: Capturing Intentions to Create And Use Many Views At Once In Large Display Environments

Mir Masood Ali

Researchers at University of Hawai&rsquo;i at Mānoa, University of Illinois Chicago, and Virginia Tech were awarded a $5 million National Science Foundation grant to synergize two complementary technologies - large-scale data visualization and artificial intelligence - to create the Smart Amplified Group Environment (SAGE3) open-source software. 
SAGE, soon to be on its third iteration as SAGE3, is the most widely used big-data visualization and collaboration software in the world. 
SAGE and SAGE2 are software to enable data-rich collaboration on high-resolution display walls. SAGE2 moved SAGE into cloud computing and SAGE3 ushers in the inclusion of artificial intelligence. 
Principal investigator Jason Leigh is a computer and information science professor at University of Hawai&rsquo;i at Mānoa and the inventor of SAGE. SAGE is software to enable teams of collaborators to work together with data in the form of data visualizations. 
SAGE3&rsquo;s novel interactive artificial intelligence methods will learn from human analytic activity and use the knowledge gained to assist people in synthesizing relevant data connections that lead to new hypotheses and findings, explained Chris North, a professor of computer science in the College of Engineering at Virginia Tech and a co-principal investigator. 
Scientists analyzing their data in SAGE3 will collaborate with each other and with artificial intelligence through large interactive visualization spaces, such as multi-monitor workstations, tiled display walls, and virtual reality headsets. 
Leigh said the applications for the SAGE3 technology are endless. &ldquo;It will be tremendously useful to enable evidence-based response during natural disasters and events like the current COVID-19 pandemic,&rdquo; Leigh said. 
He created SAGE in collaboration with researchers from the University of Illinois Chicago 16 years ago. They&rsquo;ve since further developed SAGE and SAGE2 with 15 years of research funding by the National Science Foundation. 
&ldquo;This new funding provides us with an opportunity to take SAGE to the next level - providing smart end-user services to amplify users&rsquo; productivity, making commercial and open-source artificial intelligence solutions available that can autonomously and transparently analyze data while continually learning and improving through user interactions,&rdquo; said Maxine Brown, director of the Electronic Visualization Laboratory at the University of Illinois Chicago. Brown has been involved with SAGE since its inception. 
&ldquo;SAGE3 also supports resource orchestration services that integrates these smart services with applications, workflows, and visualizations and collaboration services, and makes it easy for users to connect to local and remote display walls and computational systems and to securely run reproducible work models,&rdquo; explained Luc Renambot, Electronic Visualization Laboratory associate research professor and co-architect with Leigh of the SAGE software since its inception. 
&ldquo;As a &rsquo;co-pilot&rsquo; to individuals or teams who are trying to examine large datasets, SAGE3 will provide a discovery pathway from data sources to human insights and knowledge, which is currently a gap in the e-Science pipeline that the National Science Foundation has called the &rsquo;&ldquo;missing middle,&rsquo;&rdquo; said Mahdi Belcaid, assistant professor at University of Hawai&rsquo;i at Mānoa and the artificial intelligence co-principal investigator on the project. 
According to the researchers, SAGE3 will make artificial intelligence technologies broadly accessible, not just a privilege for the technically savvy. North said the project will conduct user-centered design research studies to ensure usability of this novel combination of data visualization and artificial intelligence in SAGE3. 
Through seed funds provided by Virginia Tech&rsquo;s Institute for Creativity, Arts, and Technology and the Hawai&rsquo;i Data Science Institute, North had the opportunity to work side-by-side with Leigh in the Laboratory for Advanced Visualization & Applications at the University of Hawai&rsquo;i during the spring 2020 semester. This in-person exchange, and numerous video-teleconferences with the Electronic Visualization Laboratory, was essential to helping plan and initiate the five-year SAGE3 collaboration, noted North. SAGE3 will serve the Institute&rsquo;s missions of bringing together computer scientists, domain scientists, engineers, artists, and designers to tackle some of the world&rsquo;s most complex challenges. 
The user base goes well beyond scientists. Leigh said he sees SAGE3 helping teachers use and share artificial intelligence concepts to future generations of students. And for those who work in creative media, such as filmmakers and video game designers, Leigh envisions it being a highly visual tool to brainstorm new ideas and analyze all forms of digital media to see how they relate to each other, which ones were most popular and why. 
Written by Jenise L. Jacques, Virginia Tech 
For further information or queries, contact Maxine Brown <a href="mailto:maxine@uic.edu">maxine@uic.edu</a>. 
Also, see <a href="https://cs.uic.edu/news-stories/data-visualization-gets-artificial-intelligence-boost-with-5-million-nsf-grant/">UIC&rsquo;s Computer Science News Story</a>.

An example of SAGE2 at work, using cloud-based and web-browser technologies in order to enhance data intensive co-located and remote collaboration.

Announcing SAGE3: Data visualization gets artificial intelligence boost with $5 million NSF grant

Multi-device environments present new opportunities for collaborative visual data analysis and sense-making by utilizing each device&rsquo;s strengths and capabilities. However, one of the associated challenges with visual data analysis in multi-device environments is the sharing of visual components across devices. We present a framework developed on top of SAGE2 platform for cross-device collaborative visual data exploration. As part of our framework, we contribute to the concept of rapid development and assembling of visualizations that can span multiple devices of different modalities. It provides the users with an environment for visualization compositions that delegate the rendering to the target device, allowing them to augment their large display workspace with portable devices for further exploration territories. Facilitated by its intuitive visualization composition pipeline, users with no programming skills such as data analysts can enhance their analytical scope with no coding barriers. We describe the framework, its implementation with a use case, and the rationale behind its design. 
Keywords: Collaboration, Multi-device environment, Visual Data Analysis

PolyVis: Cross-Device Framework for Collaborative Visual Data Analysis

There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. However, multi-device ecologies pose several challenges as information and tasks are scattered among separate devices and displays. To understand the challenges associated with multi-device environments for information visualization, we performed an exploratory study designed to examine how users employ different tools to perform different kinds of activities in approaching a visual analysis task. Previous work examined three factors (users, tools, and tasks) independently. We study the synthesis of these factors. To do this, we adopted a hybrid analysis approach that focuses on three different aspects: users, tools, and tasks. We believe this analysis will help us identify associated challenges and better inform design goals in developing multi-device tools for visual data analysis. 
Keywords: Collaborative visual analytics, exploratory analysis, multi-device ecology

Illustration of the study setup.

Towards Understanding Collaborative Visual Data Analysis in Multi-Device Environments

Supporting exploratory visual data analysis is essential when multiple analysts collaborate using multiple devices. Yet, we still have no full understanding of how the iterative process of analysis unfolds in complex settings. In this paper, we present the results from an exploratory study where six groups of three participants performed a collaborative visual data analysis task in a complex multi-user multi-device environment. We found that the course of the analysis happens at two levels. Within each level, we observed a set of exploration patterns. We present a categorization of the analysis structure in such a complex environment and discuss the implications of device affordances on this categorization. We also discuss this categorization in relation to the current structural assumptions of exploratory visual analysis. 
Index Terms: Human-centered Computing, Empirical studies in visualization

Evaluating Strategies of Exploratory Visual Data Analysis in Multi Device Environments

The VisSnippets system is designed to facilitate effective collaborative data exploration. VisSnippets leverages SAGE2 middleware that enables users to manage the display of digital media content on large displays, thereby providing collaborators with a high-resolution common workspace. Based in JavaScript, VisSnippets provides users with the flexibility to implement and/or select visualization packages and to quickly access data in the cloud. By simplifying the development process, VisSnippets removes the need to scaffold and integrate interactive visualization applications by hand. Users write reusable blocks of code called &ldquo;snippets&rdquo; for data retrieval, transformation, and visualization. By composing dataflows from the group&rsquo;s collective snippet pool, users can quickly execute and explore complementary or contrasting analyses. By giving users the ability to explore alternative scenarios, VisSnippets facilitates parallel work for collaborative data exploration leveraging large-scale displays. We describe the system, its design and implementation, and showcase its flexibility through two example applications. 
CCS Concepts: Human-centered computing, Visualization systems and tools; Collaborative and social computing systems and tools. 
Keywords: information visualization, visual data science, collaborative visual analytics

An interactive transportation dashboard for SF Muni bus data.

VisSnippets: A Web-Based System for Impromptu Collaborative Data Exploration on Large Displays

Research Experience Undergraduate

Eseroghene Omene

PI: Andrew Johnson (UIC/EVL); co-PIs: Barbara DiEugenio (UIC/CS), Moira Zellner (Norheastern University) 
Collaborator: Jason Leigh (University of Hawaii Manoa) 
The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. 
This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users&rsquo; current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., &ldquo;that pie chart about electricity consumption&rdquo;). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users&rsquo; work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users&rsquo; experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Articulate+ - A Conversational Interface for Democratizing Visual Analysis

UIC Computer Science Professors Andy Johnson (PI) and Barbara Di Eugenio (co-PI) have been awarded a $333,000 NSF grant entitled &ldquo;CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democratizing Visual Analysis&rdquo; project period October 2020 to September 2022. Dr. Moira Zellner, now of Northeastern University will also participate as co-PI on this grant. This project is part of a larger overall award of $500,000 that also includes Jason Leigh at University of Hawaii at Manoa. 
&ldquo;Articulate+&rdquo; continues EVL&rsquo; research in Natural Language Processing and Visualization also sponsored by NSF for the last 5+ years. See <a href="https://www.evl.uic.edu/entry.php?id=2522">Articulate+ - A Conversational Interface for Democratizing Visual Analysis</a> for more information on this project. 
NSF Award Number (FAIN): 2007257

NSF Grant &ldquo;CHS: Small: Collaborative Research: Articulate+&rdquo; Awarded

October 19, 2020 - - Fabio Miranda joins the EVL faculty as an assistant professor with the Department of Computer Science. Miranda interests include developing techniques that allow for the interactive visual analysis of large-scale data, combining methods from visualization, data management, machine learning, and computer graphics. Of particular concern is how visual data analytics can help address different problems cities face by integrating data on different resolutions and from different sources. 
Miranda&rsquo;s research in data science, human-computer interaction and visualization include: visual data analysis, data structures and indexing techniques for big data visualization, urban analytics, harnessing data sets of different types (images, land use information, 3D building geometry, etc.) to address concrete city problems, and machine learning. He has worked closely with domain experts from different fields, from urban planning to occupational therapy, and the outcome of these collaborations include not only research published in leading visualization, database, HCI, and AI venues, but also systems that were made available to experts in academia, industry, and government agencies. 
Miranda received his Ph.D. from New York University (2018).

Fabio Miranda Joins EVL Faculty

We developed a collaborative virtual environment for architectural planning that facilitates groups of people to work together using three different interfaces. These interfaces enable users to interact with the scene with varying levels of immersion and different interaction modalities. We conducted a user study to gauge the general usability of the system and to understand how the different interfaces affect the group work. In this paper we present the architecture of the system along with its different interfaces. We also present the user study results and the insights we gained from the study. 
Keywords: group work, collaborative environments

Two users interact with the 3D interface

A Hybrid Collaborative Virtual Environment with Heterogeneous Representations for Architectural Planning

October 1 - - EVL Director of Research Andrew Johnson has been named Interim Director of EVL following Maxine Brown&rsquo;s retirement from UIC. Johnson now assumes the responsibility of heading EVL - overseeing EVL&rsquo;s overall mission in education, research, development, and technology transfer. 
For over 23 years, Johnson has been a valued EVL faculty member (Professor of Computer Science) and researcher who has focused upon interaction and collaboration using advanced visualization displays and the application of those displays to enhance discovery and learning. 
His areas of expertise include visualization, virtual and augmented reality, natural language processing, visual analytics, and high-performance networking. 
Over the years, Johnson has been instrumental in the development of EVL instrumentation and systems, including the Continuum smart environment, CAVE2&trade;, LambdaVision high-resolution tiled displays, LambdaTable tabletop touch system, GeoWall portable geoscience display environment, PARIS augmented reality and ImmersaDesk virtual reality systems. He has overseen the development of numerous visualization and virtual reality applications across diverse domain disciplines from science and engineering to the humanities and art. Johnson is a highly regarded mentor and educator to EVL&rsquo;s graduate students and undergraduates alike. 
Johnson retains his position as Director of Research while leading EVL&rsquo;s future initiatives and innovations as Interim Director.

Andrew Johnson Named Interim Director, EVL

MS Student

Aastha Saraf

Join us in congratulating EVL faculty and School of Design professor Daria Tsoupikova who has been named the SIGGRAPH 2021 Art Papers Chair. Each year, hundreds of computer graphics researchers, professionals and technologists contribute their scientific innovations, technical achievements, and artistic accomplishments to the SIGGRAPH Art Papers venue. This is a juried and peer reviewed event, which comprises an unparalleled, collaborative experience that is one of the world&rsquo;s largest and most influential events at the intersection of art, science, and technology. 
Find out more about all of the SIGGRAPH Programs and join us online next summer for <a href="https://s2021.siggraph.org/">SIGGRAPH 2021</a>.

Daria Tsoupikova Named SIGGRAPH 2021 Art Papers Chair

EVL alumnus Marco Cavallo was awarded with the UIC College of Engineering Outstanding Young Achievement Award in Computer Science for 2020. The award identifies alumni who, through their achievement in their field of engineering, have significantly enhanced the industry and who inspire and help to instill a sense of pride among the alumni, students, faculty and staff of UIC. Marco is among the 15 alumni award honorees in engineering and computer science who have used their UIC education to establish rewarding, productive, and high-impact careers.<bra> 
Marco currently works as a senior research engineer in Apple&rsquo;s technology development group, where he is in charge of creating data visualization tools and machine learning infrastructure to foster the development of new computer vision algorithms for augmented reality. 
He graduated with a dual master&rsquo;s degree in computer science and engineering from Politecnico di Milano in Italy and the University of Illinois Chicago, where he wrote a thesis on augmented reality while working at UIC&rsquo;s Electronic Visualization Laboratory. 
Congratulations Marco! 
See the award winner video on <a href="https://www.youtube.com/watch?v=Oskk1icFSvE&feature=youtu.be">YouTube</a>.

EVL and UIC Computer Science Alumnus Marco Cavallo Honored with Outstanding Young Achievement Award

National Institutes of Health: National Cancer Institute has awarded UIC funding to continue research in head and neck cancer therapy. UIC PI, EVL faculty and Computer Science associate professor Liz Marai in conjunction with co-PI Xinhua Zhang are joined by collaborators from the University of Texas MD Anderson Cancer Center (Dave Fuller, PI) and the University of Iowa (Guadalupe Canahuate, PI) to develop a longitudinal decision support tool to assist clinicians in selecting treatments that balance survival and side-effects responses to cancer therapy. The project will lead to better insights into the biological and therapeutic factors related to cancer survival and toxicity, allowing personalization of care at the individual level. 
Title: Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy 
PI: G. Elisabeta Marai, Co-PI: Xinhua Zhang 
Sponsor: National Institutes of Health: National Cancer Institute 
Dates: March 1, 2021 - Feb 28, 2026 
Amount: $2,797,868 
Grant Number: 1 R01 CA258827-01

NIH Awards &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo;

Cancers that depend on the spatial location of the disease affect all ethnicities, and are diagnosed each year in large numbers in the United States, leading to large, rich repositories of patient data. This project develops a longitudinal decision support tool to assist clinicians in selecting treatments that balance survival and side-effects responses to cancer therapy. The project will lead to better insights into the biological and therapeutic factors related to cancer survival and toxicity, allowing personalization of care at the individual level. 
This research is sponsored by the National Institutes of Health: National Cancer Institute, and led by EVL faculty and Computer Science associate professor Liz Marai (PI) and co-PI Xinhua Zhang, along with collaborators Guadalupe Canahuate at University of Iowa, and Dave Fuller at University of Texas, MD Anderson Cancer Center.

Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy

Undergrad Researcher, BS 2020

Jonathan Vega

Alumnus Jonathan Vega (BS 2020, Computer Science) presented his experiences and research as an undergrad at EVL entitled &ldquo;Visualization of parameters from multiple numeral simulations&rdquo; as part of the UIC Impact & Research Week. Jonathan&rsquo;s research, under the supervision of EVL associate professor Georgeta-Elisabeta Marai focused upon learning how to visualize a data set for intuitive analysis. Specifically, Jonathan used contrail data to evaluate particle temperature over a given range of time with the goal of better understanding the impact of contrails on global warming. This was of particular interest to Jonathan as he is a member of the Colorado River Indian Tribe with firsthand experience of global warning on the Colorado River. The tribe relies upon the River for everything from food to Tribal traditions and ceremonies. Research has shown that the River flow has slowed dues to global warming with negative impact. Jonathan&rsquo;s efforts not only resulted in a useful visualization tool, but his achievements were also recognized with the UIC Chancellor&rsquo;s Undergraduate Research Award. 
Jonathan&rsquo;s work contributed to research under NSF Award #CBET 1854815 &ldquo;Highperformance Computing and Data-driven Modeling of Aircraft Contrails&rdquo; PI Roberto Paoli (Mechanical and Industrial Engineering); co-PI G.Elisabeta Marai (Computer Science).

CURA Presentation, Jonathan Vega,4/7/21

2D Visualization of Particle&rsquo;s Temperature

Research Experience and Visualization of Parameters from Multiple Numerical Simulations

EVL faculty and Computer Science assistant professor Wei Tang has been working with Motorola to use computer vision to improve cell phone photos taken in low-light conditions. Dr. Tang&rsquo;s research focuses on the integration of deep neural networks with conventional image processing techniques to address complex computer vision problems. Specifically, Tang and his graduate student researchers are working on methods to deblur low-light cell phone images, to return a crisp, clean version of the picture. The process includes providing the neural network a series of images - both blurry and clean - to create a loss function, or learning objective, to identify the preferred images. The resulting deblurred images are then provided to the Motorola team to brighten the low-light photos for highly improved results. 
Tang&rsquo;s research interests in computer vision, pattern recognition and machine learning can also be applied to augmented reality and intelligent robotics applications. Stay tuned for updates in Tang's research and collaborations. 
Read the related <a href="https://cs.uic.edu/news-stories/wei-tang-using-computer-vision-to-improve-cell-phone-photos-taken-in-low-light-conditions/">CS e-newsletter</a> article by Andrea Poet for more detail.

Wei Tang&rsquo;s Machine Learning Researh Featured in UIC Computer Science e-News

EVL faculty and computer science (CS) associate professor Liz Marai in collaboration with co-investigator, CS assistant professor Xinhua Zhang are developing a computational artificial intelligence system to serve as a decision support tool for oncologists prescribing treatment for head and neck cancer. The research entitled &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo; is supported by a $2.8 million grant from the National Institutes of Health NIH grant #R01CA258827 with co-investigators at the University of Texas MD Anderson Cancer Center and the University of Iowa. The goal is to develop a scalable system based on real cancer patient data that can help guide physicians in how to treat specific patients. 
Marai and Zhang are using the computational resources at Electronic Visualization Laboratory to develop their cancer treatment AI. Guadalupe Canahuate of the University of Iowa and Dr. David Fuller of the University of Texas are co-investigators on the NIH grant. 
See the full press release on <a href="https://today.uic.edu/maximizing-cancer-survival-minimizing-treatment-side-effects-with-ai">UIC Today</a> by Jacqueline Carey.

Collaborators examine a large dataset in EVL&rsquo;s CAVE2

Maximizing cancer survival, minimizing treatment side effects with AI

EVL hosts the pilot performances of &ldquo;HUMMINGBIRD&rdquo; the multiuser theatrical work that merges real and virtual worlds, engaging tech-savvy youth with a highly interactive virtual reality experience. This innovative project combines elements of traditional theater with virtual storytelling where participants collaborate to traverse virtual worlds and accomplish goals while being led by actors throughout their experience. 
EVL, the Department of Computer Science, and the College of Architecture, Design, and the Arts faculty and students in collaboration with award-winning Immersive Theatre Director Jo Cattell are organizing the pilot performance on June 18-20. This ambitious project involves a large team of faculty, graduate and undergraduate students, Goodman Theater Directors, actors, producers, a professional music composer, sound engineer, and many other contributors. This project is supported by the University of Illinois Presidential Award for Art and Humanities, the Office of the Vice Chancellor for Research Creative Activity Award, and the National Endowment for the Arts. 
The project team consists of School of Design professor, Daria Tsoupikova, award-winning theater director Jo Cattell, application developers and PhD candidates Arthur Nishimoto and Sai Priy Jyouthua, professor and EVL interim director Andrew Johnson, and project network engineering lead Lance Long. The theater team and actors include Denise Savas (stage manager), Devonte Washington (actor/assistant stage manager), Stephanie Shum (lead actor), Liviu Passare (videographer). 
Pilot performances are by invitation only. Stay tuned for the upcoming premiere this fall hosted by EVL and Goodman Theatre&rsquo;s Alice Rapport Center.

Pilot Performances of &ldquo;HUMMINGBIRD&rdquo; Collaborative Virtual Reality for Tele-Immersive Interactive Theater

Since receiving an <a href="http://sage3.sagecommons.org/">NSF award for SAGE3</a>, the Smart Amplified Group Environment, in May 2020, the SAGE3 Team has been hard at work designing and developing SAGE3. An alpha version of the SAGE3 remote collaboration platform is being developed, and the Team will give a Zoom-based SAGE3 presentation and demonstration as part of the CineGrid organization&rsquo;s Summer Community Forum series. All are welcome to attend! 
Jason Leigh and Luc Renambot will be presenting next-generation SAGE3 along with the rest of the Team. Jason and Luc are CineGrid founding members and SAGE/SAGE2/SAGE3 inventors and architects. The CineGrid organization previously focused on the use of advanced networks for digital media processing, analysis, transformation and preservation, but more recently it is focusing on Artificial Intelligence (AI) research and its relevance to digital media. 
SAGE3 is being designed to integrate AI technologies with applications, visualization workflows and collaboration services. SAGE3 takes advantage of major technological and sociological changes that have occurred since SAGE2 was introduced in 2013. Notably, web browsers and languages are maturing; scientific communities are relying on notebooks; science workloads incorporate AI programs and programmable infrastructure; and, given the pandemic, the workplace is changing to hybrid office/home environments that require not just video-teleconferencing but data-driven tools for largely distributed groups. 
All are welcome to attend the CineGrid Community Forum. There will be two SAGE3 sessions to accommodate international audiences; both sessions will present the same content, though the audience and questions may vary. 
* Tuesday, June 22 at 4:30pm - 6:00pm PDT 
* Wednesday, June 23 at 10am - 11:30am PDT 
Registration is required for each session, but there is no cost. To register, <a href="https://www.eventbrite.com/e/cinegrid-community-forum-june-tickets-156005222821">click here</a>. 
This event has been recorded and available on the <a href="https://www.youtube.com/watch?v=PyE09TxwhPE">CineGrid Community Forum archive website</a>. 
SAGE3 is being developed by the Laboratory for Advanced Visualization and Applications (LAVA) and the Hawaii Data Science Institute (HI-DSI) at the University of Hawaiʻi at Mānoa (UHM), the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC), and the Sanghani Center for AI and Data Analytics (Sanghani) at Virginia Tech (VT). SAGE3 receives major funding from the U.S. National Science Foundation, awards #2004014 (UHM), #2003800 (UIC) and #2003387 (VT). SAGE, SAGE2 and SAGE3 are trademarks of the University of Illinois Board of Trustees.

Jason Leigh, UHM, is the primary principal investigator and inventor of SAGE/SAGE2/SAGE3.

Introducing SAGE3 - Zoom event hosted by the CineGrid Community Forum

Visualization, while an effective tool for identifying patterns and insights, requires expert knowledge due to challenges faced when translating user queries to visual encodings. Research has shown that using a natural language interface (NLI) is effective for these challenges because the user can simply talk to a computer capable of producing the graphs directly. In this paper, we discuss our intelligent assistant which processes speech and hand pointing gestures while also dealing with any number of visualizations on a large screen display. Evaluation of the system shows that it is capable of quickly producing visualizations. It also particularly effective
at responding to less ambiguous queries, while in certain cases can handle ambiguous or complex queries.

The System Pipeline

Intelligent Assistant for Exploring Data Visualizations

Our goal is to develop an intelligent assistant to support users explore data via visualizations. We have collected a new corpus of conversations, CHICAGO-CRIME-VIS, geared towards supporting data visualization exploration, and we have annotated it for a variety of features, including contextualized dialogue acts. In this paper, we describe our strategies and their evaluation for dialogue act classification. We highlight how thinking aloud affects interpretation of dialogue acts in our setting and how to best capture that information. A key component of our strategy is data augmentation as applied to the training data, since our corpus is inherently small. We ran experiments with the Balanced Bagging Classifier (BAGC), Condiontal Random Field (CRF), and several Long Short Term Memory (LSTM) networks, and found that all of them improved compared to the baseline (e.g., without the data augmentation pipeline). CRF outperformed the other classification algorithms, with the LSTM networks showing modest improvement, even after obtaining a performance boost from domain-trained word embeddings. This result is of note because training a CRF is far less resource-intensive than training deep learning models, hence given a similar if not better performance, traditional methods may still be preferable in order to lower resource consumption. 
Keywords: Dialogue, Corpus, Statistical and Machine Learning Methods

Contextualized actionable requests segment relevant user utterances in each turn of the conversation.

Augmenting Small Data to Classify Contextualized Dialogue Acts for Exploratory Visualization

The data entry task is one of the common office tasks performed in the seated position that can be assisted by augmented reality technology. The paper introduces an ongoing project to develop an AR-based data presentation interface to assist office workers&rsquo; data entry tasks. The interface developed in this project was designed to facilitate the transition between the two steps including reading data and typing the data into a computer system using a keyboard. Usability evaluation is conducted in a controlled experiment with human subjects to examine whether the AR-based data presentation interface improves data entry performance and mitigates office workers&rsquo; physical and mental workload. 
Keywords: Office work, augmented reality, data entry, human factors. 
Index Terms: Human-centered concepts [Human computer interaction (HCI)]: Interaction paradigms-Virtual reality; Human-centered concepts [Human computer interaction (HCI)]: Visualization-Empirical studies in visualization

Interface development process.

Using Augmented Reality to Assist Seated Office Workers&rsquo; Data Entry Tasks

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE&trade; and the LCD-based successor CAVE2&trade;. Each of these has distinct advantages and limitations based on how they&rsquo;re used. This work explores preserving the high resolution and sense of presence of CAVE2 environments in full stereoscopic mode by using a see-through augmented reality HMD to expand the user&rsquo;s field of regard beyond the physical display walls. In our explorative study, we found that in a visual search task in a stereoscopic CAVE2, the addition of the HoloLens to expand the field of regard did not hinder the performance or accuracy of the participant, but promoted more physical navigation which in post-study interviews participants felt aided in their spatial awareness of the virtual environment. 
CCS CONCEPTS: Human-centered computing, Human computer interaction (HCI); Mixed / augmented reality; Virtual reality. 
KEYWORDS; Augmented Reality; Virtual Reality; Display Wall; Immersive Displays; Presence

HoloLens view inside CAVE2

Extending Virtual Reality Display Wall Environments Using Augmented Reality

Background - Clinical decision support (CDS) tools - with easily understood and actionable information, at the point of care - are needed to help registered nurses (RNs) make evidence-based decisions. Not clear are the optimal formats of CDS tools. Thorough, preclinical testing is desirable to avoid costly errors associated with premature implementation in electronic health records. 
Objective - To determine the feasibility of the protocol for designed to compare multiple CDS formats, and evaluate effects of numeracy and graph literacy on RN adoption of best practices and care-planning time in a simulated environment. 
Methods - In this pilot study, 60 RNs were randomly assigned to one of four CDS conditions (control; text; text+graph; text+table) and asked to adjust the plan of care for two patient scenarios over three shifts. A total of 14 best practices were identified for the two patients and sent as suggestions with evidence to the three CDS groups. Best practice adoption rates, care-planning time, and their relationship to the RN&rsquo;'s numeracy and graph literacy scores were assessed. 
Results - CDS groups had a higher adoption rate of best practices (p < .001) across all shifts and decreased care-planning time in shifts two (p = .01) and three (p = .02) compared to the control group. Higher numeracy and graph literacy were associated with shorter care-planning times under text+table (p = .05) and text+graph conditions (p = .01). No significant differences were found between the three CDS groups on adoption rate and care-planning time. 
Discussion - This pilot study demonstrates the feasibility of our protocol. Findings show preliminary evidence that CDS improves the efficiency and effectiveness of care-planning decisions, and that the optimal format may depend on individual RN characteristics. We recommend a study with sufficient power to compare different CDS formats, and assess the impact of potential covariates on adoption rates and care-planning time. 
Keywords: care plans; decision support; standardized nursing terminologies; usability

Nursing care plan examples within the three different S-HANDS CDS prototypes.

Toward meaningful care plan clinical decision support: Feasibility and effects of a simulated pilot study

Congratulations to Daria Tsoupikova in collaboration with award winning theater director Jo Cattell and their team, including application developers - EVL PhD candidates Arthur Nishimoto and Sai Priya, Computer Science professor / EVL interim director Andrew Johnson, and project network engineering lead Lance Long! The theater team and actors include Denise Savas (stage manager), Devonte Washington (actor/assistant stage manager), Stephanie Shum (lead actor), Liviu Passare (videographer). 
The following story appears in the <a href="http://digitaledition.chicagotribune.com/infinity/article_share.aspx?guid=2f768537-b6cd-42bd-af70-f5fa419357fe">Chicago Tribune, June 22, 2021</a>. 
Erratum: The article lists funding sources but incorrectly identifies the amounts of funding received. Hummingbird received the UI Presidential Award for Art and Humanities in 2019 for $150,000, the UIC Office of the Vice Chancellor for Research Creative Activity award in in 2020 for $10,000, and the National Endowment for the Arts in 2021 for $20,000. 
* * * * * * * * * * * * * * * 
Virtual-reality play: 
UIC debuts &rsquo;Hummingbird,&rsquo; allowing audience to participate in the story 
BY DARCEL ROCKETT CHICAGO TRIBUNE 
June 22, 2021 
Live theaters have yet to reopen as much of the state has, but when they do, take comfort in the fact that there are people working on taking the traditional theater-going experience to the next level. 
A group of designers and developers from the University of Illinois at Chicago&rsquo;s Electronic Visualization Laboratory in collaboration with award-winning theater director Jo Cattell, are making theater more participatory with virtual reality. The goal was realized this past weekend, June 18-20, when a variety of people ranging from preteen to retirement age were invited to partake in the immersive play &ldquo;Hummingbird.&rdquo; 
&ldquo;The whole idea here is moving in space and doing things together,&rdquo; said Daria Tsoupikova, a professor in the School of Design at UIC. &ldquo;You’re not just passive observer, you&rsquo;re actively participating.&rdquo; 
The approximately half-hour event took place in the Engineering Research building with a group of willing participants, including this Chicago Tribune reporter. Upon entering, each guest dons a lab coat and fills out a questionnaire about their soul: How old is it? When&rsquo;s the last time it was fed/replenished? After name tags are given, the group is introduced to a main character in the play, Aya, the teenage daughter of Demi, a Gerard Corp. employee. Aya is a bit antsy, but speaks her own mind; she is our guide along the experience. 
Participants follow Aya into a room where everyone is outfitted with their own Oculus Quest, an untethered virtual-reality headset with embedded tracking and touch controllers that enable multi-user interaction. At this time, the group is introduced to Gerard, a more contemporary version of Max Headroom, with better makeup, blond wig and more sass. Gerard is a floating head that talks back to Aya in real time. Gerard built Gerard Corp. to create Hummingbird technology, and he captured Demi, the talented scientist and mom of Aya. 
What follows is a collaborative team effort to escape Gerard&rsquo;s world of Zen gardens, flashlight-esque stalactites, moving statues, talking fauna and rocks by finding clues to lead us out and save Aya&rsquo;s mom. There&rsquo;s underground work, underwater work and even aerial work. As avatars, we all have different colors and stick-figure faces. The only thing that doesn&rsquo;t change is the participants&rsquo; voices. We all can talk to one another to communicate, do tasks together and problem solve. The colors, the sound design and camaraderie with strangers keeps everyone engaged with the play and one another. 
The project was designed for Goodman Theater New Stages 2020, showcasing experimental and groundbreaking theater works. The goal: to have participants attend tele-immersive, real-time performances running concurrently at the Goodman Theatre&rsquo;s Alice Rapport Center and at UIC, allowing for untethered social interaction and collaborative exploration of the plot lines through avatars. 
&ldquo;It&rsquo;s supposed to be totally immersive, it&rsquo;s supposed to be in real time, and the group at the Goodman and the group in here, they meet with each other and collaboratively explore the virtual space in co-located events, but inside virtual reality,&rdquo; Tsoupikova said. 
It&rsquo;s a meeting of art, science and live theater, where professional actors from the Goodman join with UIC faculty and students from the Department of Computer Science, School of Design and the Electronic Visualization Laboratory. The lab is known around the world for having introduced CAVE VR theater to the world In 1992 and in 2012, the CAVE2 VR theater. 
Tsoupikova says &ldquo;Hummingbird&rdquo; is to be conducted in more than one location simultaneously. The production targets youth, ages 11 and older, in under-served Chicago communities who lack access to the performing arts within their schools. 
&ldquo;We want to engage underrepresented Chicago youth - a tech-savvy younger audience through the use of
technology to engage them in the theater, because the theater attendance rates among youth are dropping,&rdquo; Tsoupikova said. &ldquo;We really want to show them that this is the future of storytelling, this is how it can be, and the theaters that productions through collaborative design like this is possible.&rdquo; 
According to Tsoupikova, the multi-user theatrical performance was conceptualized in 2018 and supported by a UI Presidential Award for Art and Humanities and an Office of the Vice Chancellor for Research Creative Activity award. A 2019 six-figure grant from the National Endowment of the Arts also helped with the development of the technology, creation of virtual scenes, scripted audience tracks, stage sets and professional actors. With the Goodman being the first major nonprofit Chicago theater to return to live performances on July 30, Tsoupikova hopes &ldquo;Hummingbird&rdquo; will open at Goodman Theatre this fall. 
&ldquo;My son hasn&rsquo;t been able to play with anyone, given the pandemic for 15 months, and for him to be so excited to be in a room with four other kids and suddenly to be interacting with people was like a dream come true,&rdquo; said &ldquo;Hummingbird&rdquo; director and writer Cattell. &ldquo;That was something nice to see, and (to) be able to give him a controlled environment where he can play with people again. I hope that continues - kids being really excited to be in a situation where it’s a different type of playground, it&rsquo;s a technical playground, because that&rsquo;s what we&rsquo;re really trying to create: a playground for everyone to play in.&rdquo;

Genevieve Thiers participates Sunday in &ldquo;Hummingbird&rdquo; at the University of Illinois at Chicago.

Virtual-reality play: UIC debuts &rsquo;Hummingbird,&rsquo; allowing audience to participate in the story

Although grief is often associated with death, receiving a diagnosis of a terminal disease for oneself or a loved one also unleashes an onslaught of emotions for the individual and caregivers. Sorrow, grief, fear, anger, and anxiety may fill the time between diagnosis and loss. Rando defines anticipatory grief as &ldquo;the phenomenon encompassing the processes of mourning, coping, interaction, planning, and psychosocial reorganization that are stimulated and begun in part in response to the awareness of the impending loss of a loved one (death) and in the recognition of associated losses in the past, present, and future.&rdquo; Though its topical presence in the literature has increased over the past half-century, the continued discrepant findings highlight the importance of further research into the outcomes of those experiencing anticipatory grief. The purpose of our study was to examine the outcomes of hospitalized end-of-life patients with anticipatory grief. 
Globally, 8 million people die of cancer every year. Coupled with the 7.7 million new annual diagnoses of dementia worldwide as well as the millions who lose family members to other terminal diseases, the apparent impact of anticipatory grief on the lives of the patient and caregivers is colossal. Research has been contradictory in its findings regarding of the long-term effects of anticipatory grief. Some argue that anticipatory grief may lead to a prolonged grief reaction, whereas some argue that it aids recovery from the grief process after death, and others that it is not related to post-death bereavement.5 One cited contributory reason to the contradictory findings is the lack of an operational definition. An underutilized solution is the NANDA International's (NANDA-I, formerly North American Nursing Diagnosis Association International) operational definition of Anticipatory Grieving, which defines the concept as an action state of engagement in grief work before the death or loss occurs (Table 1), instead of as a non-gerundial noun, anticipatory grief, which is used in much of the literature. In this article, we use Anticipatory Grieving (AG) to represent the NANDA-I diagnosis, and anticipatory grief to represent the concept, which is similar to but not necessarily conforming to the defining characteristics of AG (Table 1). Researchers have demonstrated that nurses, well-trained in the use of clinical decision support, accurately use standardized nursing terms and measures for nursing diagnoses, interventions, and outcomes in the electronic health record (EHR), which makes it possible to evaluate the impact of care provided to patients with AG. 
Keywords: electronic health record; nursing; anticipatory grieving; palliative care; grief

Outcomes for End-of-Life Patients with Anticipatory Grieving: Insights from Practice with Standardized Nursing Terminologies within an Interoperable Internet-based Electronic Health Record

To determine acceptability, usefulness, and ease of use for four nursing clinical decision support interface prototypes. In a simulated hospital environment, 60 registered nurses (48 female; mean age=33.7 10.8; mean years of experience=8.1 9.7) participated in a randomized study with four study groups. Measures included acceptability, usefulness, and ease of use scales. Mean scores were high for acceptability, usefulness, and the ease of use for all four groups. Inexperienced participants (<1 year) reported higher perceived ease of use (p=.05) and perceived usefulness (p=.01) than those with ≥1 year experience. Participants completed the protocol and reported that all four interfaces, including the control (HANDS), were acceptable, easy to use, and useful. Further study is warranted before clinical implementation within the electronic health record. 
Keywords: clinical decision support; practice-based evidence; electronic health record; end-of-life care; interface usability; simulation

Four screen shots (A–D) show the HANDS control group interface that was used to orient the participant to the major features of the care planning system.

Acceptability of clinical decision support prototypes for a nursing electronic health record to facilitate palliative care outcomes.

PURPOSE 
To develop methods for rapid and simultaneous design, testing, and management of multiple clinical decision support (CDS) features to aid nurse decision-making. 
METHODS 
We used quota sampling, think-aloud and cognitive interviews, and deductive and inductive coding of synchronized audio-video data and archival libraries. 
FINDINGS 
Our methods and organizational tools allowed us to rapidly improve the usability, understandability, and usefulness of CDS in a generalizable sample of practicing nurses. 
CONCLUSIONS 
The method outlined allows the rapid integration of nursing terminology-based electronic health record data into routine workflow and holds strong potential for improving patient outcomes. 
IMPLICATIONS FOR NURSING PRACTICE 
The methods and organizational tools for the development of multiple CDS system features can be used to translate knowledge into practice.

The Hands Ecosystem

Toward a More Robust and Efficient Usability Testing Method of Clinical Decision Support for Nurses Derived from Nursing Electronic Health Record Data

Empirical evaluation methods for visualizations have traditionally focused on assessing the outcome of the visual analytic process as opposed to characterizing how that process unfolds. There are only a handful of methods that can be used to systematically study how people use visualizations, making it difficult for researchers to capture and characterize the subtlety of cognitive and interaction behaviors users exhibit during visual analysis. To validate and improve visualization design, it is important for researchers to be able to assess and understand how users interact with visualization systems under realistic scenarios. This article presents a methodology for modeling and evaluating the behavior of users in exploratory visual analysis. We model visual exploration using a Markov chain process comprising transitions between mental, interaction, and computational states. These states and the transitions between them can be deduced from a variety of sources, including verbal transcripts, videos and audio recordings, and log files. This model enables the evaluator to characterize the cognitive and computational processes that are essential to insight acquisition in exploratory visual analysis and reconstruct the dynamics of interaction between the user and the visualization system. We illustrate this model with two exemplar user studies and demonstrate the qualitative and quantitative analytical tools it affords. 
Keywords: Exploratory visual analysis, visual exploration, evaluation, insight-based evaluation

Modeling and evaluating user behavior in exploratory visual analysis.

In this paper, we present SAGE2, a software framework that enables local and remote collaboration on Scalable Resolution Display Environments (SRDE). An SRDE can be any configuration of displays, ranging from a single monitor to a wall of tiled flat-panel displays. SAGE2 creates a seamless ultra-high resolution desktop across the SRDE. Users can wirelessly connect to the SRDE with their own devices in order to interact with the system. Many users can simultaneously utilize a drag-and-drop interface to transfer local documents and show them on the SRDE, use a mouse pointer and keyboard to interact with existing content that is on the SRDE and share their screen so that it is viewable to all. SAGE2 can be used in many configurations and is able to support many communities working with various types of media and high-resolution content, from research meetings to the creative session to education. 
SAGE2 is browser-based, utilizing a web server to host content, WebSockets for message passing and HTML with JavaScript for rendering and interaction. Recent web developments, with the emergence of HTML5, have allowed browsers to use advanced rendering techniques without requiring plug-ins (canvas drawing, WebGL 3D rendering, native video player, etc.). One major benefit of browser-based software is that there are no installation requirements for users and it is inherently cross-platform. A user simply needs a web browser on the device he/she wishes to use as an interaction tool for the SRDE. This lowers considerably the barrier of entry to engage in meaningful collaboration sessions. 
Keywords: Scalable Resolution Display Environments, Window manager, Web-based, Collaboration, Multi-user interaction, Large-scale displays, Application development

Co-located collaborative session using SAGE2 - ten users interacting with a SRDE tiled wall.

SAGE2: a Collaboration Portal for Scalable Resolution Displays

Our long-term goal is to ensure nurse clinical decision support (CDS) works as intended before full deployment in clinical practice. As part of a broader effort, this pilot explores factors influencing acceptance/non-acceptance of 8 CDS suggestions displayed through selecting a blinking red button in an electronic health record (EHR) based nursing plan of care software prototype. A diverse sample of 21 nurses participated in this high fidelity clinical simulation experience and completed a questionnaire to assess reasons for accepting/not accepting the CDS suggestions. Of 168 total suggestions displayed during the experiment (8 for each of the 21 nurses), 123 (73.2%) were accepted and 45 (26.8%) were not accepted. The mode number of acceptances by nurses was 7 of 8 with only 2 of 21 nurses accepting all. The main reason for CDS acceptance was the nurse’s belief that the suggestions were good for the patient (n=100%) with other features being secondarily reinforcing. Reasons for non-acceptance were less clear, with under half of the subjects indicating low confidence in the evidence. This study provides preliminary evidence that high-quality simulation and targeted questionnaires about specific CDS selections offer a cost-effective means for testing before full deployment in clinical practice. 
Keywords:</strrong> electronic health records; nursing care plans; standardization; clinical decision support systems; simulation lab

Nursing care plan examples within the three different HANDS prototypes.

Use of Simulation to Study Nurses&rsquo; Acceptance and Nonacceptance of Clinical Decision Support Suggestions

We present findings of a comparative study of numeracy and graph literacy in a representative group of 60 practicing nurses. The paper focuses on a fundamental concern related to the effectiveness of numeric information displayed in various features in the electronic health record during clinical workflow. Our findings suggest the need to consider numeracy and graph literacy when presenting numerical information as well as the potential for tailoring numeric display types to individual&rsquo;s cognitive strengths. 
Keywords: Clinical decision making; electronic health records; graphical literacy; numeracy; nurses

Numeracy and Graph Literacy Means and Standard Deviations by Nurse Characteristics

Nurses&rsquo; Numeracy and Graphical Literacy: Informing Studies of Clinical Decision Support Interfaces

The purpose of this concurrent mixed-methods study was to 1) examine the factors pediatric intensive care unit nurses consider when assessing and intervening for children who report severe pain and to 2) determine the effect of child behavior and diagnosis on the nurses' pain ratings and intervention choices for written and virtual human vignettes. Quantitative and qualitative results substantiated that despite recommendations to use self-report, many PICU nurses use behavior as the primary indicator to assess and treat pain, even when a child is old enough to articulate pain intensity and there is sufficient cause for pain to be present. 
Keywords: Acute pain; Hospitalized child; Knowledge use in pain care; Patient simulation; Pediatric intensive care; Pediatric nurse.

Univariate Differences in Pain Ratings and Morphine Doses Between Behaviors

PICU Nurses&rsquo; Pain Assessments And Intervention Choices for Virtual Human And Written Vignettes

The popularity of large-scale, high-resolution display walls, as visualization endpoints in eScience infrastructure, is rapidly growing. These displays can be connected to distributed computing resources over high-speed network, providing effective means for researchers to visualize, interact with, and understand large volumes of datasets. Typically large display walls are built by tiling multiple physical displays together and running a tiled display wall required a cluster of computers. With the advent of advanced graphics hardware, a single computer can now drive over a dozen displays, thereby greatly reducing the cost of ownership and maintenance of a tiled display wall system. This in turn enables a broader user base to take advantage of such technologies. Since tiled display walls are also well suited to collaborative work, users tend to launch and operate multiple applications simultaneously. To ensure that applications maintain a high degree of responsiveness to the users even under heavy use loads, the display wall must now ensure that the limited system resources are prioritized to maximize interactivity rather than thread-level fair sharing or overall job-completion throughput. In this paper, we present a new resource scheduling scheme that is specifically designed to prioritize responsiveness in collaborative large display wall environments where multiple users can interact with multiple applications simultaneously. We evaluate our scheduling scheme with a user study involving groups of users interacting simultaneously on a tiled display wall with multiple applications. Results show that our scheduling framework provided a higher frame rate for applications, which led to a significantly higher user performance (approx. 25%) in a target acquisition test when compared against the traditional operating system scheduling scheme. 
Keywords: Scheduling, Human factors, Algorithms, Interactive systems, Distributed graphics

A single machine-driven tiled-display wall running SAGE at the Electronic Visualization Laboratory in the University of Illinois at Chicago. The 20′ by 6′ display wall is made up of 18 LCD panels with a total resolution approximately 17 megapixels.

Multiuser-Centered Resource Scheduling for Collaborative Display Wall Environments

Introduction: As virtual experiences are increasingly used in health care training and research, it is important that adequate processes are applied for developing valid scenarios. We describe the development and validation of virtual human (VH) vignettes, computer-generated scenarios with animated patients and clinical information, for a mixed-methods study regarding nurses&rsquo; assessment and intervention choices for critically ill children&rsquo;s pain. 
Methods: We followed the case development and review process for high-fidelity simulation case scenarios, including the use of validated written vignettes and content experts. Forty nurses described their pain assessment and intervention choices for the newly derived VH vignettes and completed a pain questionnaire. Nurses&rsquo; reports of VH vignette consistency with their professional experience and recognition of VH facial expressions were evaluated to establish face validity. Their pain ratings for the VH and written (questionnaire) vignettes were evaluated for convergent validity. Qualitative content analysis, descriptive statistics, correlations, and paired t-tests were used. 
Results: Most nurses (68.4%) supported vignette consistency with their professional experience. Facial expression recognition was 98.4%. Smiling children&rsquo;s pain was rated significantly lower than grimacing children in both VH and written vignettes. Pain was rated significantly lower for grimacing children in the VH vignettes than the written vignettes. Virtual human vignette pain ratings were strongly correlated with their written counterparts. 
Conclusions: This process was effective for developing VH vignettes that demonstrated good face validity with participants and convergent validity with written vignettes. Virtual human vignettes may be useful in studying the influence of facial actions on nurses' choices for children&rsquo;s pain assessment and treatment.

Formation of base head models. Examples of photographs of two of the four children (left, top and bottom) used to develop the base head models for the virtual human vignettes.

Development and Validation of a Virtual Human Vignette to Compare Nurses Assessment and Intervention Choices for Pain in Critically Ill Children

Pain management of end of life patients (EOL) (n=596 episodes) is examined using statistical and data mining processes of the HANDS database of care plans coded with NANDA-I, NOC, and NIC (NNN) terminologies. HANDS episode data (episode =care plans updated at every handoff on a patient while staying on a hospital unit) were gathered in 8 units located in 4 different health care facilities (total episodes = 40,747; EOL episodes = 1,425) over two years. Results show the multiple discoveries such as EOL patients with hospital stays (< 72 hrs.) are less likely (p<0.005) to meet the pain relief goals compared to EOL patients with longer hospital stays. The study demonstrates a major benefit of systematically integrating NNN into electronic health records. 
Keywords: Data mining; plan of care; electronic health record; end of life hospital care; pain

Human Computer Interaction (HCI)

Bibliography

pubs (76)

Related Topics

people (29)

events (66)

research (10)

Contact