This collaborative research project led by the colleagues at the University of Delaware aims to develop and advance tools for using image-data appearing in scientific publications, in addition to text, in order to support beneficial, targeted access to the biomedical literature. 
UIC's principal investigator, Computer Science associate professor, G. Elisabeta Marai will be responsible for the image analysis algorithms to distinguish between gel and plate images, the image analysis algorithms to distinguish between Electronic Microscopy vs Fluorescent Microscopy vs Light Microscopy and X-Rays, and the design of the visual interface to present results to end users. She will lead the design, development and implementation of the feature extraction and selection tools and techniques related to this research project.

Incorporating Image-based Features into Biomedical Document Classification

COMPaaS DLV, COMposable Platform as a Service Instrument for Deep Learning & Visualization, was purchased from Liqid, Inc., with NSF funds. Its 4th-generation computer architecture, composable infrastructure, treats computer components - computer processing units (CPUs), graphical processing units (GPUs), networking, memory and storage - as pools of resources. It is: 
<ul><li>Highly flexible. Each application defines the resources it needs, and the operating system composes, or combines, resources on the fly.</li> 
<li>Scalable. As more infrastructure is added, it’s auto-integrated with the existing infrastructure and becomes part of the pool of capacity.</li> 
<li>High-throughput. System components are interconnected with a high-speed internal fabric, enabling Big Data to quickly move among CPU, GPU, memory and storage at optimum speeds with little to no bottlenecks.</ul></li> 
Funded in part by NSF award #CNS-1828265.

Visual Data Science: Advanced Cyberinfrastructure Systems (poster session)

Ph.D. Student: Abeer Alsaiari 
Committee: 
Prof. Andrew Johnson, Chair and Advisor 
Prof. Joseph Michaelis 
Prof. Wei Tang 
Prof. Steve Jones (Department of Communication) 
Prof. Khairi Reda (Indiana University-Purdue University Indianapolis) 
Date: Monday, March 9th 
Time: 1:30 pm 
Location: ERF 3036 
Abstract: 
There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. This new thrust of multi-device interfaces supports new models for complex collaboration scenarios, and have great potential to support analysts in their data analysis by utilizing each device&rsquo;s capabilities. However, there are some challenges inherently associated with visual data analysis in multi-device environments (MDE). As the analytical process is underway, analytical components become scattered among different devices and displays. Building a mental model of the analysis flow can render the analytical process more challenging as it would be difficult to track many visualizations and therefore difficult for analysts to keep track of all the prior analyses. The primary goal of my thesis research is to understand the characterization of the analysis process, with regard to the use and formation around devices, and to support the design with visualization of the search space, which will allow users to efficiently explore their data.

Preliminary Examination Announcement: &ldquo;Supporting Collaborative Visual Data Analysis in Multi-device Environment&rdquo;

Maxine Brown (UIC PI), Andrew Johnson (UIC co-PI), Luc Renambot (UIC co-PI) 
Jason Leigh (University of Hawaii Manoa PI) Christopher North (Virginia Polytechnic PI) 
NSF Award Number: 2003800 
Award Period of Performance: 05/15/2020 - 04/30/2025 
Total Amount Obligated: $2,250,000 
UIC Computer Science Electronic Visualization Lab (EVL) Director Maxine Brown, PI, co-PIs UIC CS Professor Andy Johnson and UIC CS Research Associate Professor Luc Renambot, and Senior Personnel EVL Senior Research Programmer Lance Long are awarded a $2.25 million dollar National Science Foundation grant entitled &ldquo;Collaborative Research: CSSI Frameworks: SAGE3: Smart Amplified Group Environment for Harnessing the Data Revolution.&rdquo; The grant runs from May 2020 to April 2025. It&rsquo;s part of an overall $5 million project, that also includes a team from University of Hawaii Manoa led by Jason Leigh and a team from Virginia Polytechnic led by Christopher North. 
ABSTRACT 
The Big Data revolution necessitates the use of sophisticated tools such as Artificial Intelligence (AI) and Data Visualization to harness the sheer volume, velocity and variety of datasets that are becoming the norm. However, it is the research community that must make sense of the data being amassed, so cyberinfrastructure must extend to people. SAGE3 (Smart Amplified Group Environment) puts the &ldquo;human in the loop&rdquo; by providing scientists with an intuitive framework that integrates state-of-the-art AI technologies with applications, workflows, smart visualizations and collaboration services to help them access, share, explore and analyze their data, come to conclusions, and make decisions with greater speed, accuracy, comprehension and confidence. 
SAGE3 builds upon the NSF-funded first-generation SAGE (Scalable Adaptive Graphics Environment) and second-generation SAGE2 (Scalable Amplified Group Environment), which are today&rsquo;s de facto operating systems to manage Big Data on tiled display walls. Its development focuses on two fundamental components: AI-enhanced &ldquo;smart&rdquo; services and advanced computing resource orchestration to support reproducible work models for secure collaborative work. SAGE3 amplifies user productivity, providing them with commercially available and open-source AI solutions, which autonomously and transparently analyze data while continually learning and improving through user interactions. SAGE3 makes AI technologies broadly accessible, not just a privilege for the technically savvy. SAGE3 further democratizes AI by using Data Visualization to help interpret and explain AI models so users better understand how AI came to its decisions, which engenders user trust and can help identify potentially prejudiced or biased models. 
SAGE3 augments every step of the scientific discovery enterprise - from quickly summarizing large data, to finding trends and similarities or anomalies among one or more linked datasets, to communicating findings to scientists, public policy and government officials, and the general public, to educating the next-generation workforce. Ultimately, it is the scientists and future scientists who must Harness the Big Data revolution to solve the nation&rsquo;s grand challenge problems that will benefit society as a whole - from studying the diversity of life on Earth, to understanding the Earth and its systems from satellite imagery of its poles, to developing response scenarios for natural disasters such as landslides and pandemics that impact the citizens and economies of the world.

UIC Computer Science/EVL awarded $2.25 Million from NSF for SAGE3

UIC PI: Andrew Johnson, co-PI: Luc Renambot 
University of Hawaii Manoa PI: Jason Leigh 
Virginia Polytechnic PI: Chirstopher North 
NSF Award Number: 2003800 
Award Period of Performance: 05/15/2020 - 04/30/2025 
Total Amount Obligated: $2,250,000 
The Big Data revolution necessitates the use of sophisticated tools such as Artificial Intelligence (AI) and Data Visualization to harness the sheer volume, velocity and variety of datasets that are becoming the norm. However, it is the research community that must make sense of the data being amassed, so cyberinfrastructure must extend to people. SAGE3 (Smart Amplified Group Environment) puts the &ldquo;human in the loop&rdquo; by providing scientists with an intuitive framework that integrates state-of-the-art AI technologies with applications, workflows, smart visualizations and collaboration services to help them access, share, explore and analyze their data, come to conclusions, and make decisions with greater speed, accuracy, comprehension and confidence. 
SAGE3 builds upon the NSF-funded first-generation SAGE (Scalable Adaptive Graphics Environment) and second-generation SAGE2 (Scalable Amplified Group Environment), which are today&rsquo;s de facto operating systems to manage Big Data on tiled display walls. Its development focuses on two fundamental components: AI-enhanced &ldquo;smart&rdquo; services and advanced computing resource orchestration to support reproducible work models for secure collaborative work. SAGE3 amplifies user productivity, providing them with commercially available and open-source AI solutions, which autonomously and transparently analyze data while continually learning and improving through user interactions. SAGE3 makes AI technologies broadly accessible, not just a privilege for the technically savvy. SAGE3 further democratizes AI by using Data Visualization to help interpret and explain AI models so users better understand how AI came to its decisions, which engenders user trust and can help identify potentially prejudiced or biased models. 
SAGE3 augments every step of the scientific discovery enterprise - from quickly summarizing large data, to finding trends and similarities or anomalies among one or more linked datasets, to communicating findings to scientists, public policy and government officials, and the general public, to educating the next-generation workforce. Ultimately, it is the scientists and future scientists who must Harness the Big Data revolution to solve the nation&rsquo;s grand challenge problems that will benefit society as a whole - from studying the diversity of life on Earth, to understanding the Earth and its systems from satellite imagery of its poles, to developing response scenarios for natural disasters such as landslides and pandemics that impact the citizens and economies of the world. 
<a href="https://youtu.be/kvNSLGcs5KE">View a video of SAGE3 being demonstrated</a> on EVL&rsquo;s YouTube channel.

SAGE3: Smart Amplified Group Environment for Harnessing the Data Revolution

Led by EVL Director Maxine Brown, a large team of research faculty and staff from UIC&rsquo;s College of
Engineering applied for and received a Major Research Instrumentation (MRI) grant from the National Science Foundation (NSF) for approximately $1-million to acquire a deep learning and visualization infrastructure. 
The design utilizes composable infrastructure architecture from <a href="https://www.liqid.com/">Liqid</a>. The computer&rsquo;s components (traditional processor, GPU, storage, and networking) are pooled so that different applications with different workflows can be run simultaneously, with each configuring the resources it requires almost instantaneously, at any time. 
&ldquo;EVL resources are traditionally very tailored to meet the requirements of specific applications. For example, on any given day, EVL may address everything from computational fluid dynamics, ecology and neuroimaging to natural language processing, urban engineering, robotics, visualization and visual analytics,&rdquo; said Lance Long, EVL Senior Research Programmer. &ldquo;Liqid&rsquo;s composable infrastructure architecture is interesting because it allows us the flexibility to redefine cyberinfrastructure programmatically based on the applications&rsquo; requirements. This enables the EVL team to create a data science platform that presents a full-stack hardware and software solution to the challenges faced by our researchers trying to deploy and scale reproducible experiments.&rdquo;

EVL&rsquo;s COMPaaS DLV system

Case Study: UIC Utilizes Liqid Composable Infrastructure for Uneven Applications in Scientific Research

In October 2018, the UIC Electronic Visualization Laboratory (EVL) received an equipment grant from the National Science Foundation (NSF #1828265) for a GPU-based system for College of Engineering research and education. Since that time, EVL staff has surveyed vendors, analyzed UIC faculty&rsquo;s application requirements, and then designed, purchased, received and installed its GPU-based system from Liqid. EVL refers to its new computing system as Composable Platform as a Service Instrument for Deep Learning & Visualization (COMPaaS DLV). 
<a href="https://www.liqid.com/">Liqid</a> is a start-up company specializing in systems integration (and has major industrial partners such as Dell, Intel, Nvidia and Mellanox), who currently sells its next-generation &ldquo;composable infrastructure&rdquo; computer systems primarily to data centers. Working with UIC faculty, it became apparent that this advanced architecture could address their diverse application requirements as their data moves through the computational pipeline (e.g., data gathering, filtering, computing, storing). 
EVL worked with Liqid to educate them about how its product could be tailored to meet the needs of the academic research market, helping them realize that science and engineering research is a potential market for their product. Working together, Liqid has configured a system to suit the needs of EVL&rsquo;s research focus. 
Liqid has published a <a href="https://www.evl.uic.edu/entry.php?id=2495">case study, &ldquo;UIC Utilizes Liqid Composable Infrastructure for Uneven Applications in Scientific Research&rdquo;</a> that explores EVL&rsquo;s use of COMPaaS DLV.

Liqid publishes a case study on how EVL is using their GPU-based system COMPaaS DLV

This paper describes results from an observational, exploratory study of visual data exploration in a large, multi-view, flexible canvas environment. Participants were provided with a set of data exploration sub-tasks associated with a local crime dataset and were instructed to pose questions to a remote mediator who would respond by generating and organizing visualizations on the large display. We observed that participants frequently posed requests to cast a net around one or several subsets of the data or a set of data attributes. They accomplished this directly and by utilizing existing views in unique ways, including by requesting to copy and pivot a group of views collectively and posing a set of parallel requests on target views expressed in one command. These observed actions depart from multi-view flexible canvas environments that typically provide interfaces in support of generating one view at a time or actions that operate on one view at a time. We describe how participants used these &ldquo;cast-a-net&rdquo; requests for tasks that spanned more than one view and describe design considerations for multi-view environments that would support the observed multi-view generation actions. 
Index Terms: Human-centered computing - Empirical studies in visualization 
This work was supported initially by NSF award IIS 1445751 and currently by NSF award CNS 1625941. 
<a href="https://doi.org/10.1111/cgf.13976">https://doi.org/10.1111/cgf.13976</a>

A participant generates 15 views of her data. In the final request, the participant references a set of three views and poses a request to copy and pivot these views collectively to four new subsets of the data. The final result is a grid of views.

Many At Once: Capturing Intentions to Create And Use Many Views At Once In Large Display Environments

Precision medicine seeks to tailor therapy to the individual patient, based on statistical correlates from patients who are similar to the one under consideration. These correlates can and should go beyond genetics, and in general, beyond tabular or array data that can be easily represented computationally and compared. For example, in many types of cancer, cancer treatment and toxicity depend in large measure on the spatial disease spread - e.g., metastasizes to regional lymph nodes in head and neck cancer. However, there is currently a lack of methodology for integrating spatial information when considering patient similarity. We present a novel modeling methodology for the comparison of cancer patients within a cohort, based on the spatial spread of the lymph nodes affected in each patient. The method uses a topological map, bigrams, and hierarchical clustering to group patients based on their similarity. We compare this approach against a nonspatial (categorical) similarity approach where patients are binned solely by their affected nodes. We present similarity results on a 582 head and neck cancer patient cohort, along with two visual abstractions for analysis of the results, and we present clinician feedback. Our novel methodology partitions a patient cohort into clinically meaningful groups more susceptible to treatment side-effects. Such spatially-aware similarity approaches can help maximize the effectiveness of each patient&rsquo;s treatment. 
Funding: NIH NCI-R01CA214825, NIH NCI-R01CA2251, and NIH NLM-R01LM012527

Topological map and graph representation.

A Spatial Neighborhood Methodology for Computing and Analyzing Lymph Node Carcinoma Similarity in Precision Medicine

PhD Graduate Student

Mir Masood Ali

EVL PhD candidate Juan Trelles Trabucco has been recognized among the UIC Computer Science Department&rsquo;s 2020 Outstanding Graduate TAs. This award typically recognizes graduate TAs for their work over an entire year, but given Juan&rsquo;s stellar performance during the spring semester, and especially during the pandemic, Juan is being duly recognized for his contributions to courses CS426 and CS529. 
&ldquo;Juan&rsquo;s TA performance in CS426 was beyond spectacular. The course had 60 students enrolled and made use of a variety of technologies. Yet with Juan&rsquo;s help, we had a flawless transition to online teaching, when the pandemic lockout was instituted.&rdquo; 
CS associate professor Elisabeta Marai was the instructor for both courses. CS426: Video Game Design and Implementation covers tools and techniques for the design and implementation of 3D interactive video games. CS529: Visual Data Science is an introduction to key design principles and techniques for interactively visualizing and analyzing data in data science. 
In addition, to the usual TA assignments of tutoring students, grading papers, etc. assisting in the transition of these courses to an online virtual platform posed additional challenges, and makes Juan truly deserving of this recognition. 
Please join us in congratulating Juan!

EVL PhD Candidate Juan Trelles Trabucco Receives CS 2020 Outstanding TA Award

Researchers at University of Hawai&rsquo;i at Mānoa, University of Illinois Chicago, and Virginia Tech were awarded a $5 million National Science Foundation grant to synergize two complementary technologies - large-scale data visualization and artificial intelligence - to create the Smart Amplified Group Environment (SAGE3) open-source software. 
SAGE, soon to be on its third iteration as SAGE3, is the most widely used big-data visualization and collaboration software in the world. 
SAGE and SAGE2 are software to enable data-rich collaboration on high-resolution display walls. SAGE2 moved SAGE into cloud computing and SAGE3 ushers in the inclusion of artificial intelligence. 
Principal investigator Jason Leigh is a computer and information science professor at University of Hawai&rsquo;i at Mānoa and the inventor of SAGE. SAGE is software to enable teams of collaborators to work together with data in the form of data visualizations. 
SAGE3&rsquo;s novel interactive artificial intelligence methods will learn from human analytic activity and use the knowledge gained to assist people in synthesizing relevant data connections that lead to new hypotheses and findings, explained Chris North, a professor of computer science in the College of Engineering at Virginia Tech and a co-principal investigator. 
Scientists analyzing their data in SAGE3 will collaborate with each other and with artificial intelligence through large interactive visualization spaces, such as multi-monitor workstations, tiled display walls, and virtual reality headsets. 
Leigh said the applications for the SAGE3 technology are endless. &ldquo;It will be tremendously useful to enable evidence-based response during natural disasters and events like the current COVID-19 pandemic,&rdquo; Leigh said. 
He created SAGE in collaboration with researchers from the University of Illinois Chicago 16 years ago. They&rsquo;ve since further developed SAGE and SAGE2 with 15 years of research funding by the National Science Foundation. 
&ldquo;This new funding provides us with an opportunity to take SAGE to the next level - providing smart end-user services to amplify users&rsquo; productivity, making commercial and open-source artificial intelligence solutions available that can autonomously and transparently analyze data while continually learning and improving through user interactions,&rdquo; said Maxine Brown, director of the Electronic Visualization Laboratory at the University of Illinois Chicago. Brown has been involved with SAGE since its inception. 
&ldquo;SAGE3 also supports resource orchestration services that integrates these smart services with applications, workflows, and visualizations and collaboration services, and makes it easy for users to connect to local and remote display walls and computational systems and to securely run reproducible work models,&rdquo; explained Luc Renambot, Electronic Visualization Laboratory associate research professor and co-architect with Leigh of the SAGE software since its inception. 
&ldquo;As a &rsquo;co-pilot&rsquo; to individuals or teams who are trying to examine large datasets, SAGE3 will provide a discovery pathway from data sources to human insights and knowledge, which is currently a gap in the e-Science pipeline that the National Science Foundation has called the &rsquo;&ldquo;missing middle,&rsquo;&rdquo; said Mahdi Belcaid, assistant professor at University of Hawai&rsquo;i at Mānoa and the artificial intelligence co-principal investigator on the project. 
According to the researchers, SAGE3 will make artificial intelligence technologies broadly accessible, not just a privilege for the technically savvy. North said the project will conduct user-centered design research studies to ensure usability of this novel combination of data visualization and artificial intelligence in SAGE3. 
Through seed funds provided by Virginia Tech&rsquo;s Institute for Creativity, Arts, and Technology and the Hawai&rsquo;i Data Science Institute, North had the opportunity to work side-by-side with Leigh in the Laboratory for Advanced Visualization & Applications at the University of Hawai&rsquo;i during the spring 2020 semester. This in-person exchange, and numerous video-teleconferences with the Electronic Visualization Laboratory, was essential to helping plan and initiate the five-year SAGE3 collaboration, noted North. SAGE3 will serve the Institute&rsquo;s missions of bringing together computer scientists, domain scientists, engineers, artists, and designers to tackle some of the world&rsquo;s most complex challenges. 
The user base goes well beyond scientists. Leigh said he sees SAGE3 helping teachers use and share artificial intelligence concepts to future generations of students. And for those who work in creative media, such as filmmakers and video game designers, Leigh envisions it being a highly visual tool to brainstorm new ideas and analyze all forms of digital media to see how they relate to each other, which ones were most popular and why. 
Written by Jenise L. Jacques, Virginia Tech 
For further information or queries, contact Maxine Brown <a href="mailto:maxine@uic.edu">maxine@uic.edu</a>. 
Also, see <a href="https://cs.uic.edu/news-stories/data-visualization-gets-artificial-intelligence-boost-with-5-million-nsf-grant/">UIC&rsquo;s Computer Science News Story</a>.

An example of SAGE2 at work, using cloud-based and web-browser technologies in order to enhance data intensive co-located and remote collaboration.

Announcing SAGE3: Data visualization gets artificial intelligence boost with $5 million NSF grant

Multi-device environments present new opportunities for collaborative visual data analysis and sense-making by utilizing each device&rsquo;s strengths and capabilities. However, one of the associated challenges with visual data analysis in multi-device environments is the sharing of visual components across devices. We present a framework developed on top of SAGE2 platform for cross-device collaborative visual data exploration. As part of our framework, we contribute to the concept of rapid development and assembling of visualizations that can span multiple devices of different modalities. It provides the users with an environment for visualization compositions that delegate the rendering to the target device, allowing them to augment their large display workspace with portable devices for further exploration territories. Facilitated by its intuitive visualization composition pipeline, users with no programming skills such as data analysts can enhance their analytical scope with no coding barriers. We describe the framework, its implementation with a use case, and the rationale behind its design. 
Keywords: Collaboration, Multi-device environment, Visual Data Analysis

PolyVis: Cross-Device Framework for Collaborative Visual Data Analysis

There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. However, multi-device ecologies pose several challenges as information and tasks are scattered among separate devices and displays. To understand the challenges associated with multi-device environments for information visualization, we performed an exploratory study designed to examine how users employ different tools to perform different kinds of activities in approaching a visual analysis task. Previous work examined three factors (users, tools, and tasks) independently. We study the synthesis of these factors. To do this, we adopted a hybrid analysis approach that focuses on three different aspects: users, tools, and tasks. We believe this analysis will help us identify associated challenges and better inform design goals in developing multi-device tools for visual data analysis. 
Keywords: Collaborative visual analytics, exploratory analysis, multi-device ecology

Illustration of the study setup.

Towards Understanding Collaborative Visual Data Analysis in Multi-Device Environments

Supporting exploratory visual data analysis is essential when multiple analysts collaborate using multiple devices. Yet, we still have no full understanding of how the iterative process of analysis unfolds in complex settings. In this paper, we present the results from an exploratory study where six groups of three participants performed a collaborative visual data analysis task in a complex multi-user multi-device environment. We found that the course of the analysis happens at two levels. Within each level, we observed a set of exploration patterns. We present a categorization of the analysis structure in such a complex environment and discuss the implications of device affordances on this categorization. We also discuss this categorization in relation to the current structural assumptions of exploratory visual analysis. 
Index Terms: Human-centered Computing, Empirical studies in visualization

Evaluating Strategies of Exploratory Visual Data Analysis in Multi Device Environments

EDITOR’S NOTE: The Professional University of Information and Management for Innovation (iU) announces a partnership with 7 international universities, including the University of Illinois at Chicago&rsquo;s (UIC) Electronic Visualization Laboratory (EVL). More specifically, Osamu Kamatani, Dean of iU&rsquo;s Faculty of Information and Management for Innovation, previously worked for Japan&rsquo;s NTT Network Innovation Laboratories, which had a 20-year research collaboration with UIC/EVL focused on advanced networking and collaboration. NTT was an early adopter of EVL&rsquo;s SAGE visualization and collaboration software and was involved in EVL-related advanced networking projects. iU will adopt SAGE2 and SAGE3, and will participate in the Global Research Platform (GRP), which is being organized by UIC/EVL, University of California San Diego (UCSD) and Northwestern University. The MOU formalizes and strengthens iU&rsquo;s collaborations with UIC/EVL. 
* * * * * * * * * * * * * * * * 
iU of Japan announces partnership with 7 international universities 
Friday, August 21, 2020 - The Professional University of Information and Management for Innovation (&ldquo;iU&rdquo;) commenced its first year of operation on April 1st 2020, and has now reached comprehensive partnership agreements with the following seven global prominent schools: The University of California, San Diego (UCSD); The University of Illinois at Chicago (UIC); The University of Sheffield; Raffles University Malaysia; The National University of Singapore; Nikola Tesla Graduate School in the U.K.; and the Federal University of Technology Akure (FUTA), Ondo State, Nigeria, with ongoing efforts for additional international collaborations. 
iU will continue to actively collaborate with major universities in Japan and abroad, with the aim to realize a &ldquo;Global University Network,"&rdquo; representing an intellectual hub for the IT and business industries centered around iU. 
Specifically, in addition to the exchange of students and faculty members and the implementation of joint research, the objectives are as follows: 
　1. A &ldquo;passport&rdquo; system that allows classes at partner universities and recognition of course credits 
　2. Joint development for research of new ventures and new products within a joint strategic special zone 
　3. Establishment of a special visa zone for international students 
　4. International collaborative research on the fields of eSports, super education, super sports, anime, otaku, etc. 
　5. Entrepreneurship support and Human Capital development 
The &ldquo;Global University Network&rdquo; advocated by iU, includes alliances with overseas universities. Through collaboration with each institution, iU will strive to develop and promote international joint lectures, joint experiments, implementation of various IT technologies, and joint research through exchanges of students and faculty from partner universities. iU will also work closely with prominent companies in Japan and overseas that support these activities. 
<a href="https://www.i-u.ac.jp/news/detail.php?id=199">Read the entire press release.</a> 
For Further Information, please contact: 
PR division 
Professional University of Information and Management for Innovation 
TEL: +81356551555 E-mail: <a href="mailto:info@i-u.ac.jp">info@i-u.ac.jp</a> 
For UIC/EVL collaboration information, contact Maxine Brown: <a href="mailto:maxine@uic.edu">maxine@uic.edu</a>

iU of Japan announces partnership with 7 international universities

Graph convolutional networks (GCNs) have been applied to 3D human pose estimation (HPE) from 2D body joint detections and
have shown encouraging performance. One limitation of the vanilla graph convolution is that it models the relationships between neighboring nodes via a shared weight matrix. This is suboptimal for articulated body modeling as the relations between different body joints are different. The objective of this paper is to have a comprehensive and systematic study of weight sharing in GCNs for 3D HPE. We first show there are two different ways to interpret a GCN depending on whether feature transformation occurs before or after feature aggregation. These two interpretations lead to five different weight sharing methods and three more variants can be derived by decoupling the self-connections with other edges. We conduct extensive ablation study on these weight sharing methods under controlled settings and obtain new conclusions that will benefit the community.

Qualitative results of our approach on Human 3.6M

A Comprehensive Study of Weight Sharing in Graph Networks for 3D Human Pose Estimation

Graph convolutional networks (GCNs) have been applied to 3D human pose estimation (HPE) from 2D body joint detections and have demonstrated promising performance. However, since the vanilla graph convolution is performed on the one-hop neighbors of each node, it is unable to capture the long-range dependencies between body joints. They can help reduce the uncertainty caused by occlusion or depth ambiguity. To resolve this issue, we propose a high-order GCN for 3D HPE. Its core building block, termed a high-order graph convolution, aggregates features of nodes at various distances. As a result, the network can model a wide range of interactions among body joints. Furthermore, we investigate different methods to fuse those multi-order features and compare how they affect the performance. Experimental results demonstrate the effectiveness of the proposed approach.

Comparison between the vanilla graph convolution and the high-order graph convolution performed on a skeleton graph.

High-order Graph Convolutional Networks for 3D Human Pose Estimation

Research Experience Undergraduate

Eseroghene Omene

PI: Andrew Johnson (UIC/EVL); co-PIs: Barbara DiEugenio (UIC/CS), Moira Zellner (Norheastern University) 
Collaborator: Jason Leigh (University of Hawaii Manoa) 
The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. 
This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users&rsquo; current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., &ldquo;that pie chart about electricity consumption&rdquo;). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users&rsquo; work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users&rsquo; experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Articulate+ - A Conversational Interface for Democratizing Visual Analysis

UIC Computer Science Professors Andy Johnson (PI) and Barbara Di Eugenio (co-PI) have been awarded a $333,000 NSF grant entitled &ldquo;CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democratizing Visual Analysis&rdquo; project period October 2020 to September 2022. Dr. Moira Zellner, now of Northeastern University will also participate as co-PI on this grant. This project is part of a larger overall award of $500,000 that also includes Jason Leigh at University of Hawaii at Manoa. 
&ldquo;Articulate+&rdquo; continues EVL&rsquo; research in Natural Language Processing and Visualization also sponsored by NSF for the last 5+ years. See <a href="https://www.evl.uic.edu/entry.php?id=2522">Articulate+ - A Conversational Interface for Democratizing Visual Analysis</a> for more information on this project. 
NSF Award Number (FAIN): 2007257

NSF Grant &ldquo;CHS: Small: Collaborative Research: Articulate+&rdquo; Awarded

October 19, 2020 - - Fabio Miranda joins the EVL faculty as an assistant professor with the Department of Computer Science. Miranda interests include developing techniques that allow for the interactive visual analysis of large-scale data, combining methods from visualization, data management, machine learning, and computer graphics. Of particular concern is how visual data analytics can help address different problems cities face by integrating data on different resolutions and from different sources. 
Miranda&rsquo;s research in data science, human-computer interaction and visualization include: visual data analysis, data structures and indexing techniques for big data visualization, urban analytics, harnessing data sets of different types (images, land use information, 3D building geometry, etc.) to address concrete city problems, and machine learning. He has worked closely with domain experts from different fields, from urban planning to occupational therapy, and the outcome of these collaborations include not only research published in leading visualization, database, HCI, and AI venues, but also systems that were made available to experts in academia, industry, and government agencies. 
Miranda received his Ph.D. from New York University (2018).

Fabio Miranda Joins EVL Faculty

October 1 - - EVL Director of Research Andrew Johnson has been named Interim Director of EVL following Maxine Brown&rsquo;s retirement from UIC. Johnson now assumes the responsibility of heading EVL - overseeing EVL&rsquo;s overall mission in education, research, development, and technology transfer. 
For over 23 years, Johnson has been a valued EVL faculty member (Professor of Computer Science) and researcher who has focused upon interaction and collaboration using advanced visualization displays and the application of those displays to enhance discovery and learning. 
His areas of expertise include visualization, virtual and augmented reality, natural language processing, visual analytics, and high-performance networking. 
Over the years, Johnson has been instrumental in the development of EVL instrumentation and systems, including the Continuum smart environment, CAVE2&trade;, LambdaVision high-resolution tiled displays, LambdaTable tabletop touch system, GeoWall portable geoscience display environment, PARIS augmented reality and ImmersaDesk virtual reality systems. He has overseen the development of numerous visualization and virtual reality applications across diverse domain disciplines from science and engineering to the humanities and art. Johnson is a highly regarded mentor and educator to EVL&rsquo;s graduate students and undergraduates alike. 
Johnson retains his position as Director of Research while leading EVL&rsquo;s future initiatives and innovations as Interim Director.

Andrew Johnson Named Interim Director, EVL

For the Supercomputing (SC20) Conference this year, a research community partnership was formed to design an international networking testbed using existing facilities, with several international extensions. Demonstrations and experiments were conducted prior to the conference with results discussed during a SC20 virtual forum. &ldquo;The Forum For Experiments and Demonstrations Showcasing Innovations In Large-Scale, Data-Intensive-Science Networking&rdquo; was organized by Research Consortium for Data-Intensive Science Networking to communicate: a) the motivation behind these experiments and demonstrations, b) their design, c) technologies used, d) results and e) future research in these areas. 
EVL senior research programmer, Lance Long participated to discuss EVL&rsquo;s research in Composable Infrastructure, specifically MRI: Acquisition of a Composable Platform as a Service Instrument for Deep Learning & Visualization (COMPaaS DLV) funded under NSF Award #CNS-1828265. Items discussed included the benefits of composable infrastructure, implementation for deep learning in an academic research environment, operating systems, dynamic workflows, orchestration middleware, current use cases and future work. 
Other participants included: Ciena Research Lab, NASA Goddard Space Flight Center (GSFC), Naval Research Laboratory, iCAIR/Northwestern, StarLight, UCSD, FNAL, CalTech, SURFnet.

Composable Infrastructure for Data Science

Lance Long Presents EVL&rsquo;s Composable Infrastructure Research at SC20

Advances in data collection in radiation therapy have led to an abundance of opportunities for applying data mining and machine learning techniques to promote new data-driven insights. In light of these advances, supporting collaboration between machine learning experts and clinicians is important for facilitating better development and adoption of these models. Although many medical use-cases rely on spatial data, where understanding and visualizing the underlying structure of the data is important, little is known about the interpretability of spatial clustering results by clinical audiences. In this work, we reflect on the design of visualizations for explaining novel approaches to clustering complex anatomical data from head and neck cancer patients. These visualizations were developed, through participatory design, for clinical audiences during a multi-year collaboration with radiation oncologists and statisticians. We distill this collaboration into a set of lessons learned for creating visual and explainable spatial clustering for clinical users. 
Keywords: Data Clustering and Aggregation, Life Sciences, Collaboration, Mixed Initiative Human-Machine Analysis, Guidelines 
YouTube Video: <a href="https://youtu.be/UM8-_MN3a2Y">https://youtu.be/UM8-_MN3a2Y</a>

(A) Lymph nodes overlaid over a diagram of the neck. (B) Example graphs of diseased nodes for 2 individual patients (datapoint representation). (C) Example consensus graph for 1 cluster (cluster representation).

Explainable Spatial Clustering: Leveraging Spatial Data in Radiation Oncology

Microscopy images are pervasive in biomedical research publications, where images obtained through various microscopy modalities (light, fluorescence, scanning, transmission) are often used to describe and summarize experiments and contributions. Hence, there is growing interest in automatically identifying these microscopy images’ modality and utilizing this knowledge in automated search tools. However, identifying microscopy images poses challenges due to a lack of extensive collections of labeled images. We describe and evaluate two alternative approaches to microscopy image classification. In the first approach, we progressively fine-tuned layers of ResNet models. The second approach uses shallow variants of ResNet networks, where we leverage the outputs from previous convolutional blocks. We compare these results against a Support Vector Machine (SVM)-based baseline. Our results show that fine-tuning specific layers yields better results than fine-tuning the whole model. Furthermore, shallower variants produce competitive results when compared to the entire fine-tuned model. 
Index Terms: microscopy, image classification, deep learning

Four microscopy images from the ImageCLEF dataset showcasing different modalities: (a) Electron microscopy (DMEL), (b) Transmission microscopy (DMTR), (c) Light microscopy (DMLI), and (d) Fluorescence microscopy (DMFL).

Modality-Classification of Microscopy Images Using Shallow Variants of Deep Networks

EVL faculty and Computer Science associate professor G. Elisabeta Marai (MPI) along with H. Shatkay (MPI) at the University of Delaware have been awarded supplemental funding for COVID-19 related research to their NIH research project entitled &ldquo;Uncovering Clinical Evidence in COVID-19 Publications: An Integrated Search via Text &amp; Images&rdquo;. This supplement expands the work done under current R01 funding &ldquo;Incorporating Image-based Features into Biomedical Document Classification&rdquo;, to support discovery within the CORD-19 collection -- a large, continually updated collection of more than 60,000 scientific articles pertaining to COVID-19. This supplement will create a comprehensive platform for retrieving and displaying well-focused, need-targeted medical information within COVID-19 biomedical publications. Our approach is unique compared to other text-mining methods because it leverages the image content in the dataset. 
Funding: NIH R01 NLM Covid-19 supplement to Parent Grant: 1R01LM012527 (&ldquo;Incorporating Image-based Features into Biomedical Document Classification&rdquo;)

EVL Faculty G.E. Marai Awarded NIH COVID-19 Supplement

EVL faculty member and computer science associate professor Liz Marai has been appointed to co-chair the full papers session at the 23rd edition of EuroVis 2021. The EuroVis conference objective is to create and strengthen connections between visualization researchers and practitioners, to foster collaborations, and to draw more researchers into the rapidly growing area of visualization research. EuroVis has an expanded its scope to include all areas of visualization towards more wide-spread visibility and impact. 
EuroVis 2021 will be hosted by the University of Zurich in collaboration with FAU Erlangen-Nuremberg and ETH Zurich. EuroVis is the annual Visualization Conference organized by the Eurographics Working Group on Data Visualization. This year&rsquo;s conference will be held virtually June 14 - 18, 2021. 
Liz joins Rita Borgo, King&rsquo;s College London, UK and Tatiana von Landesberger, University of Rostock, Germany to co-chair the full papers session focusing on bu not limited to visualization techniques, systems, applications & design studies, evaluation & empirical research, and theory. 
Congratulations, Liz on this prestigious role and your tireless work as an ambassador for visualization research!

Associate Professor Liz Marai co-Chairs Full Papers Session for EuroVis21

National Institutes of Health: National Cancer Institute has awarded UIC funding to continue research in head and neck cancer therapy. UIC PI, EVL faculty and Computer Science associate professor Liz Marai in conjunction with co-PI Xinhua Zhang are joined by collaborators from the University of Texas MD Anderson Cancer Center (Dave Fuller, PI) and the University of Iowa (Guadalupe Canahuate, PI) to develop a longitudinal decision support tool to assist clinicians in selecting treatments that balance survival and side-effects responses to cancer therapy. The project will lead to better insights into the biological and therapeutic factors related to cancer survival and toxicity, allowing personalization of care at the individual level. 
Title: Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy 
PI: G. Elisabeta Marai, Co-PI: Xinhua Zhang 
Sponsor: National Institutes of Health: National Cancer Institute 
Dates: March 1, 2021 - Feb 28, 2026 
Amount: $2,797,868 
Grant Number: 1 R01 CA258827-01

NIH Awards &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo;

Cancers that depend on the spatial location of the disease affect all ethnicities, and are diagnosed each year in large numbers in the United States, leading to large, rich repositories of patient data. This project develops a longitudinal decision support tool to assist clinicians in selecting treatments that balance survival and side-effects responses to cancer therapy. The project will lead to better insights into the biological and therapeutic factors related to cancer survival and toxicity, allowing personalization of care at the individual level. 
This research is sponsored by the National Institutes of Health: National Cancer Institute, and led by EVL faculty and Computer Science associate professor Liz Marai (PI) and co-PI Xinhua Zhang, along with collaborators Guadalupe Canahuate at University of Iowa, and Dave Fuller at University of Texas, MD Anderson Cancer Center.

Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy

MS Candidate: Elisa Tardini 
Date: Wednesday, March 10, 2021 
Time: 9:30 am CT 
Location: <a href="https://uic.zoom.us/j/83263055512?pwd=UXdKZkRiNmsxR08vMFo3ZzU1VGQ3Zz09">Zoom Link</a> 
Meeting ID: 832 6305 5512 
Passcode: UPut2ew9 
Committee: Xinhua Zhang, G. Elisabeta Marai, Pier Luca Lanzi (Politecnico di Milano) 
Abstract: 
In recent years deep meta-reinforcement learning has extended the applicability of reinforcement learning algorithms: by integrating recurrent networks, trained models have the ability to quickly adapt to new unseen environments without the need for further backpropagation. We propose a novel recursive deep meta-reinforcement learning approach which enables the model of each decision of the sequential process to learn from and adapt to unseen circumstances by recursively integrating the feedback of the models of other decisions in the process. We apply this approach to a dataset of 3-step chemo-radiotherapeutic and surgical treatment of head and neck cancer patients, proving its ability to optimally handle previously unseen patient&rsquo;s (and physician&rsquo;s) preferences on survival and toxicity outcomes.

MS Thesis Defense Announcement: &ldquo;Recursive Meta-Reinforcement Learning for Personalized Sequential Dynamic Treatment Policies&rdquo;

MS Graduate, 2021

Elisa Tardini

Alumnus Jonathan Vega (BS 2020, Computer Science) presented his experiences and research as an undergrad at EVL entitled &ldquo;Visualization of parameters from multiple numeral simulations&rdquo; as part of the UIC Impact & Research Week. Jonathan&rsquo;s research, under the supervision of EVL associate professor Georgeta-Elisabeta Marai focused upon learning how to visualize a data set for intuitive analysis. Specifically, Jonathan used contrail data to evaluate particle temperature over a given range of time with the goal of better understanding the impact of contrails on global warming. This was of particular interest to Jonathan as he is a member of the Colorado River Indian Tribe with firsthand experience of global warning on the Colorado River. The tribe relies upon the River for everything from food to Tribal traditions and ceremonies. Research has shown that the River flow has slowed dues to global warming with negative impact. Jonathan&rsquo;s efforts not only resulted in a useful visualization tool, but his achievements were also recognized with the UIC Chancellor&rsquo;s Undergraduate Research Award. 
Jonathan&rsquo;s work contributed to research under NSF Award #CBET 1854815 &ldquo;Highperformance Computing and Data-driven Modeling of Aircraft Contrails&rdquo; PI Roberto Paoli (Mechanical and Industrial Engineering); co-PI G.Elisabeta Marai (Computer Science).

CURA Presentation, Jonathan Vega,4/7/21

2D Visualization of Particle&rsquo;s Temperature

Research Experience and Visualization of Parameters from Multiple Numerical Simulations

EVL faculty and computer science (CS) associate professor Liz Marai in collaboration with co-investigator, CS assistant professor Xinhua Zhang are developing a computational artificial intelligence system to serve as a decision support tool for oncologists prescribing treatment for head and neck cancer. The research entitled &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo; is supported by a $2.8 million grant from the National Institutes of Health NIH grant #R01CA258827 with co-investigators at the University of Texas MD Anderson Cancer Center and the University of Iowa. The goal is to develop a scalable system based on real cancer patient data that can help guide physicians in how to treat specific patients. 
Marai and Zhang are using the computational resources at Electronic Visualization Laboratory to develop their cancer treatment AI. Guadalupe Canahuate of the University of Iowa and Dr. David Fuller of the University of Texas are co-investigators on the NIH grant. 
See the full press release on <a href="https://today.uic.edu/maximizing-cancer-survival-minimizing-treatment-side-effects-with-ai">UIC Today</a> by Jacqueline Carey.

Collaborators examine a large dataset in EVL&rsquo;s CAVE2

Maximizing cancer survival, minimizing treatment side effects with AI

Since receiving an <a href="http://sage3.sagecommons.org/">NSF award for SAGE3</a>, the Smart Amplified Group Environment, in May 2020, the SAGE3 Team has been hard at work designing and developing SAGE3. An alpha version of the SAGE3 remote collaboration platform is being developed, and the Team will give a Zoom-based SAGE3 presentation and demonstration as part of the CineGrid organization&rsquo;s Summer Community Forum series. All are welcome to attend! 
Jason Leigh and Luc Renambot will be presenting next-generation SAGE3 along with the rest of the Team. Jason and Luc are CineGrid founding members and SAGE/SAGE2/SAGE3 inventors and architects. The CineGrid organization previously focused on the use of advanced networks for digital media processing, analysis, transformation and preservation, but more recently it is focusing on Artificial Intelligence (AI) research and its relevance to digital media. 
SAGE3 is being designed to integrate AI technologies with applications, visualization workflows and collaboration services. SAGE3 takes advantage of major technological and sociological changes that have occurred since SAGE2 was introduced in 2013. Notably, web browsers and languages are maturing; scientific communities are relying on notebooks; science workloads incorporate AI programs and programmable infrastructure; and, given the pandemic, the workplace is changing to hybrid office/home environments that require not just video-teleconferencing but data-driven tools for largely distributed groups. 
All are welcome to attend the CineGrid Community Forum. There will be two SAGE3 sessions to accommodate international audiences; both sessions will present the same content, though the audience and questions may vary. 
* Tuesday, June 22 at 4:30pm - 6:00pm PDT 
* Wednesday, June 23 at 10am - 11:30am PDT 
Registration is required for each session, but there is no cost. To register, <a href="https://www.eventbrite.com/e/cinegrid-community-forum-june-tickets-156005222821">click here</a>. 
This event has been recorded and available on the <a href="https://www.youtube.com/watch?v=PyE09TxwhPE">CineGrid Community Forum archive website</a>. 
SAGE3 is being developed by the Laboratory for Advanced Visualization and Applications (LAVA) and the Hawaii Data Science Institute (HI-DSI) at the University of Hawaiʻi at Mānoa (UHM), the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC), and the Sanghani Center for AI and Data Analytics (Sanghani) at Virginia Tech (VT). SAGE3 receives major funding from the U.S. National Science Foundation, awards #2004014 (UHM), #2003800 (UIC) and #2003387 (VT). SAGE, SAGE2 and SAGE3 are trademarks of the University of Illinois Board of Trustees.

Jason Leigh, UHM, is the primary principal investigator and inventor of SAGE/SAGE2/SAGE3.

Introducing SAGE3 - Zoom event hosted by the CineGrid Community Forum

Visualization, while an effective tool for identifying patterns and insights, requires expert knowledge due to challenges faced when translating user queries to visual encodings. Research has shown that using a natural language interface (NLI) is effective for these challenges because the user can simply talk to a computer capable of producing the graphs directly. In this paper, we discuss our intelligent assistant which processes speech and hand pointing gestures while also dealing with any number of visualizations on a large screen display. Evaluation of the system shows that it is capable of quickly producing visualizations. It also particularly effective
at responding to less ambiguous queries, while in certain cases can handle ambiguous or complex queries.

The System Pipeline

Intelligent Assistant for Exploring Data Visualizations

Our goal is to develop an intelligent assistant to support users explore data via visualizations. We have collected a new corpus of conversations, CHICAGO-CRIME-VIS, geared towards supporting data visualization exploration, and we have annotated it for a variety of features, including contextualized dialogue acts. In this paper, we describe our strategies and their evaluation for dialogue act classification. We highlight how thinking aloud affects interpretation of dialogue acts in our setting and how to best capture that information. A key component of our strategy is data augmentation as applied to the training data, since our corpus is inherently small. We ran experiments with the Balanced Bagging Classifier (BAGC), Condiontal Random Field (CRF), and several Long Short Term Memory (LSTM) networks, and found that all of them improved compared to the baseline (e.g., without the data augmentation pipeline). CRF outperformed the other classification algorithms, with the LSTM networks showing modest improvement, even after obtaining a performance boost from domain-trained word embeddings. This result is of note because training a CRF is far less resource-intensive than training deep learning models, hence given a similar if not better performance, traditional methods may still be preferable in order to lower resource consumption. 
Keywords: Dialogue, Corpus, Statistical and Machine Learning Methods

Contextualized actionable requests segment relevant user utterances in each turn of the conversation.

Augmenting Small Data to Classify Contextualized Dialogue Acts for Exploratory Visualization

Background - Clinical decision support (CDS) tools - with easily understood and actionable information, at the point of care - are needed to help registered nurses (RNs) make evidence-based decisions. Not clear are the optimal formats of CDS tools. Thorough, preclinical testing is desirable to avoid costly errors associated with premature implementation in electronic health records. 
Objective - To determine the feasibility of the protocol for designed to compare multiple CDS formats, and evaluate effects of numeracy and graph literacy on RN adoption of best practices and care-planning time in a simulated environment. 
Methods - In this pilot study, 60 RNs were randomly assigned to one of four CDS conditions (control; text; text+graph; text+table) and asked to adjust the plan of care for two patient scenarios over three shifts. A total of 14 best practices were identified for the two patients and sent as suggestions with evidence to the three CDS groups. Best practice adoption rates, care-planning time, and their relationship to the RN&rsquo;'s numeracy and graph literacy scores were assessed. 
Results - CDS groups had a higher adoption rate of best practices (p < .001) across all shifts and decreased care-planning time in shifts two (p = .01) and three (p = .02) compared to the control group. Higher numeracy and graph literacy were associated with shorter care-planning times under text+table (p = .05) and text+graph conditions (p = .01). No significant differences were found between the three CDS groups on adoption rate and care-planning time. 
Discussion - This pilot study demonstrates the feasibility of our protocol. Findings show preliminary evidence that CDS improves the efficiency and effectiveness of care-planning decisions, and that the optimal format may depend on individual RN characteristics. We recommend a study with sufficient power to compare different CDS formats, and assess the impact of potential covariates on adoption rates and care-planning time. 
Keywords: care plans; decision support; standardized nursing terminologies; usability

Nursing care plan examples within the three different S-HANDS CDS prototypes.

Toward meaningful care plan clinical decision support: Feasibility and effects of a simulated pilot study

Efficient RDF, graph-based queries are becoming more pertinent based on the increased interest in data analytics and its intersection with large, unstructured but connected data. Many commercial systems have adopted distributed RDF graph systems in order to handle increasing dataset sizes and complex queries. This paper introduces a distribute graph approach to pre-processing linked data. Instead of traversing the memory graph, our system indexes pre-processed join elements that are organized in a graph structure. We analyze the Dbpedia data-set (derived from the Wikipedia cor-pus) and compare our access method to the graph traversal access approach which we also devise. Results show from our experiments that the distributed, pre-processed graph approach to accessing linked data is faster than the traversal approach over a specific range of linked queries. 
CCS CONCEPTS: Computing methodologies, Distributed algorithms 
KEYWORDS: RDF, High Performance Computing; Distributed Algorithms

(a) Graph-store algorithm Graph construction and neighbor times over 1M, .5M triple dataset.

A Distributed Graph Approach for Pre-processing Linked RDF Data Using Supercomputers

Empirical evaluation methods for visualizations have traditionally focused on assessing the outcome of the visual analytic process as opposed to characterizing how that process unfolds. There are only a handful of methods that can be used to systematically study how people use visualizations, making it difficult for researchers to capture and characterize the subtlety of cognitive and interaction behaviors users exhibit during visual analysis. To validate and improve visualization design, it is important for researchers to be able to assess and understand how users interact with visualization systems under realistic scenarios. This article presents a methodology for modeling and evaluating the behavior of users in exploratory visual analysis. We model visual exploration using a Markov chain process comprising transitions between mental, interaction, and computational states. These states and the transitions between them can be deduced from a variety of sources, including verbal transcripts, videos and audio recordings, and log files. This model enables the evaluator to characterize the cognitive and computational processes that are essential to insight acquisition in exploratory visual analysis and reconstruct the dynamics of interaction between the user and the visualization system. We illustrate this model with two exemplar user studies and demonstrate the qualitative and quantitative analytical tools it affords. 
Keywords: Exploratory visual analysis, visual exploration, evaluation, insight-based evaluation

Modeling and evaluating user behavior in exploratory visual analysis.

Our long-term goal is to ensure nurse clinical decision support (CDS) works as intended before full deployment in clinical practice. As part of a broader effort, this pilot explores factors influencing acceptance/non-acceptance of 8 CDS suggestions displayed through selecting a blinking red button in an electronic health record (EHR) based nursing plan of care software prototype. A diverse sample of 21 nurses participated in this high fidelity clinical simulation experience and completed a questionnaire to assess reasons for accepting/not accepting the CDS suggestions. Of 168 total suggestions displayed during the experiment (8 for each of the 21 nurses), 123 (73.2%) were accepted and 45 (26.8%) were not accepted. The mode number of acceptances by nurses was 7 of 8 with only 2 of 21 nurses accepting all. The main reason for CDS acceptance was the nurse’s belief that the suggestions were good for the patient (n=100%) with other features being secondarily reinforcing. Reasons for non-acceptance were less clear, with under half of the subjects indicating low confidence in the evidence. This study provides preliminary evidence that high-quality simulation and targeted questionnaires about specific CDS selections offer a cost-effective means for testing before full deployment in clinical practice. 
Keywords:</strrong> electronic health records; nursing care plans; standardization; clinical decision support systems; simulation lab

Nursing care plan examples within the three different HANDS prototypes.

Use of Simulation to Study Nurses&rsquo; Acceptance and Nonacceptance of Clinical Decision Support Suggestions

We present findings of a comparative study of numeracy and graph literacy in a representative group of 60 practicing nurses. The paper focuses on a fundamental concern related to the effectiveness of numeric information displayed in various features in the electronic health record during clinical workflow. Our findings suggest the need to consider numeracy and graph literacy when presenting numerical information as well as the potential for tailoring numeric display types to individual&rsquo;s cognitive strengths. 
Keywords: Clinical decision making; electronic health records; graphical literacy; numeracy; nurses

Numeracy and Graph Literacy Means and Standard Deviations by Nurse Characteristics

Nurses&rsquo; Numeracy and Graphical Literacy: Informing Studies of Clinical Decision Support Interfaces

The purpose of this concurrent mixed-methods study was to 1) examine the factors pediatric intensive care unit nurses consider when assessing and intervening for children who report severe pain and to 2) determine the effect of child behavior and diagnosis on the nurses' pain ratings and intervention choices for written and virtual human vignettes. Quantitative and qualitative results substantiated that despite recommendations to use self-report, many PICU nurses use behavior as the primary indicator to assess and treat pain, even when a child is old enough to articulate pain intensity and there is sufficient cause for pain to be present. 
Keywords: Acute pain; Hospitalized child; Knowledge use in pain care; Patient simulation; Pediatric intensive care; Pediatric nurse.

Univariate Differences in Pain Ratings and Morphine Doses Between Behaviors

PICU Nurses&rsquo; Pain Assessments And Intervention Choices for Virtual Human And Written Vignettes

The popularity of large-scale, high-resolution display walls, as visualization endpoints in eScience infrastructure, is rapidly growing. These displays can be connected to distributed computing resources over high-speed network, providing effective means for researchers to visualize, interact with, and understand large volumes of datasets. Typically large display walls are built by tiling multiple physical displays together and running a tiled display wall required a cluster of computers. With the advent of advanced graphics hardware, a single computer can now drive over a dozen displays, thereby greatly reducing the cost of ownership and maintenance of a tiled display wall system. This in turn enables a broader user base to take advantage of such technologies. Since tiled display walls are also well suited to collaborative work, users tend to launch and operate multiple applications simultaneously. To ensure that applications maintain a high degree of responsiveness to the users even under heavy use loads, the display wall must now ensure that the limited system resources are prioritized to maximize interactivity rather than thread-level fair sharing or overall job-completion throughput. In this paper, we present a new resource scheduling scheme that is specifically designed to prioritize responsiveness in collaborative large display wall environments where multiple users can interact with multiple applications simultaneously. We evaluate our scheduling scheme with a user study involving groups of users interacting simultaneously on a tiled display wall with multiple applications. Results show that our scheduling framework provided a higher frame rate for applications, which led to a significantly higher user performance (approx. 25%) in a target acquisition test when compared against the traditional operating system scheduling scheme. 
Keywords: Scheduling, Human factors, Algorithms, Interactive systems, Distributed graphics

A single machine-driven tiled-display wall running SAGE at the Electronic Visualization Laboratory in the University of Illinois at Chicago. The 20′ by 6′ display wall is made up of 18 LCD panels with a total resolution approximately 17 megapixels.

Multiuser-Centered Resource Scheduling for Collaborative Display Wall Environments

Pain management of end of life patients (EOL) (n=596 episodes) is examined using statistical and data mining processes of the HANDS database of care plans coded with NANDA-I, NOC, and NIC (NNN) terminologies. HANDS episode data (episode =care plans updated at every handoff on a patient while staying on a hospital unit) were gathered in 8 units located in 4 different health care facilities (total episodes = 40,747; EOL episodes = 1,425) over two years. Results show the multiple discoveries such as EOL patients with hospital stays (< 72 hrs.) are less likely (p<0.005) to meet the pain relief goals compared to EOL patients with longer hospital stays. The study demonstrates a major benefit of systematically integrating NNN into electronic health records. 
Keywords: Data mining; plan of care; electronic health record; end of life hospital care; pain

User Interface of an example decision-making assistant based on knowledge discovery using data mining of Nursing Care Plans

Data Mining Nursing Care Plans of End of Life Patients: A Case Study to Improve Healthcare Decision Making

Objective: To test the hypothesis that Hands-on Automated Nursing Data System (HANDS) &ldquo;big picture summary&rdquo; can be implemented uniformly across diverse settings, and result in positive registered nurse (RN) and plan of care (POC) data outcomes across time. 
Design: In a longitudinal, multisite, full test study, a representative convenience sample of eight medical-surgical units from four hospitals (one university, two large community, and one small community) in one Midwestern state implemented the HANDS intervention for 24 (four units) or 12 (four units) months. 
Measurements: (a) RN outcomes-percentage completing training, satisfaction with standardized terminologies, perception of HANDS usefulness, POC submission compliance rate. (b) POC data outcomes-validity (rate of optional changes/episode); reliability of terms and ratings; and volume of standardized data generated. 
Results: One hundred percent of the RNs who worked on the eight study units successfully completed the required standardized training; all units selected participated for the entire 12- or 24-month designated period; compliance rates for POC entry at every patient hand-off were 78-92%; reliability coefficients for use of the standardized terms and ratings were moderately strong; the pattern of optional POC changes per episode declined but remained reasonable across time; and the nurses generated a database of 40,747 episodes of care. 
Limitations: Only RNs and medical-surgical units participated. 
Conclusion: It is possible to effectively standardize the capture and visualization of useful &ldquo;big picture&rdquo; healthcare information across diverse settings. Findings offer a viable alternative to the current practice of introducing new health information layers that ultimately increase the complexity and inconsistency of information for frontline users.

The HANDS Ecosystem.

Maintaining a consistent big picture: Meaningful Use of a Web-based POC EHR System

As the human body skeleton can be represented as a sparse graph, it is natural to exploit graph convolutional networks (GCNs) to model the articulated body structure for 3D human pose estimation (HPE). However, a vanilla graph convolutional layer, the building block of a GCN, only models the local relationships between each body joint and their neighbors on the skeleton graph. Some global attributes, e.g., the action of the person, can be critical to 3D HPE, especially in the case of occlusion or depth ambiguity. To address this issue, this paper introduces a new 3D HPE framework by learning global pose features in GCNs. Specifically, we add a global node to the graph and connect it to all the body joint nodes. On one hand, global features are updated by aggregating all body joint features to model the global attributes. On the other hand, the feature update of each body joint depends on not only their neighbors but also the global node. Furthermore, we propose a heterogeneous multi-task learning framework to learn the local and global features. While each local node regresses the 3D coordinate of the corresponding body joint, we force the global node to classify an action category or learn a low-dimensional pose embedding. Experimental results demonstrate the effectiveness of the proposed approach. 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Illustration of a graph convolutional network (GCN) with a global nodeufor 3D human pose estimation

Learning Global Pose Features in Graph Convolutional Networks for 3D Human Pose Estimation

Dependency relations among visual entities are ubiquitous because both objects and scenes are highly structured. They provide prior knowledge about the real world that can help improve the generalization ability of deep learning approaches. Different from contextual reasoning which focuses on feature aggregation in the spatial domain, visual dependency reasoning explicitly models the dependency relations among visual entities. In this paper, we introduce a novel network architecture, termed the dependency network or DependencyNet, for semantic segmentation. It unifies dependency reasoning at three semantic levels. Intra-class reasoning decouples the representations of different object categories and updates them separately based on the internal object structures. Inter-class reasoning then performs spatial and semantic reasoning based on the dependency relations among different object categories. We will have an in-depth investigation on how to discover the dependency graph from the training annotations. Global dependency reasoning further refines the representations of each object category based on the global scene information. Extensive ablative studies with a controlled model size and the same network depth show that each individual dependency reasoning component benefits semantic segmentation and they together significantly improve the base network. Experimental results on two benchmark datasets show the DependencyNet achieves comparable performance to the recent states of the art. 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Visual dependence relations are ubiquity since both ob-jects and scenes are highly structured. They provide prior knowl-edge about the real world that can be used to improve the gener-alization ability of a learning model.

Exploit Visual Dependency Relations for Semantic Segmentation

Candidate Name: Krishna Bharadwaj 
Date: Tuesday, July 6th, 2021 
Time: 2:00PM 
Location: <a href="https://uic.zoom.us/j/82883119453?pwd=SVMvMUhQZGFCZWtSUnFEZmFiSWl3Zz09">Zoom Link</a> 
Meeting ID: 828 8311 9453 
Passcode: kbEVL8phd 
Committee Members: Andrew Johnson (Chair and Advisor), G. Elisabeta Marai, Debaleena Chattopadhyay, Steve Jones (Department of Communication), Jason Leigh (University of Hawaii at Manoa), Thomas J. Marrinan (University of St. Thomas) 
Abstract: 
Collaborative Virtual Environments have been increasingly used in supporting group work. Several collaborative systems have come up in recent years as a way of meeting the ever-changing needs of group work, with asymmetric setups being the latest addition to a diverse collection of systems. In recent years, we see attempts at multi-domain collaboration. Research on interdisciplinary collaboration reveals a need for supporting a hybrid of tools and representations that are familiar to users of different domains. 
In this dissertation, I present a prototype implementation of a Hybrid Collaborative Virtual Environment that combines multiple representations to facilitate multi-domain collaboration. Preliminary evaluation of the prototype showed that hybridization of different representations can lead to role formation due to dependency on representations, when users are limited to interacting with a single representation. Integrating shared views and shared interaction into this Hybrid Collaborative Virtual Environment with the help of a large shared display environment can help solve the representational dependency issue. Shared views and shared interactions enable users to freely interact with any of the different representations and also eases communication amongst them. I evaluate this integration by means of a formal study having a within group design: A group of three participants perform collaborative tasks using the Hybrid Collaborative Virtual Environment with and without the integration of shared views and shared interactions. I discuss the results of the study and show how shared views and shared interaction improves the collaborative experience.

Collaborative System

PhD Dissertation Announcement: Hybrid Collaborative Virtual Environments

Candidate: Jillian Aurisano 
DATE: Friday, July 9, 2021 
TIME: 1:00PM 
LOCATION: <a href="https://uic.zoom.us/j/84367747564?pwd=T3NiZUR3cjQwanQxa0JiaC92RXQzUT09">Zoom Link</a> 
Meeting ID: 843 6774 7564 
Passcode: 6DwiV1hA 
Committee Members: Andrew Johnson (chair), Barbara Di Eugenio, G. Elisabeta Marai, Debaleena Chattopadhyay, Moira Zellner 
Abstract:
Visual data exploration stands to benefit from environments that permit users to view and juxtapose many views of data, particularly views that present diverse selections of data values and attributes. Large, high-resolution environments are capable of showing many related views of data, but efficiently creating and displaying visualizations in these environments presents significant challenges. In this talk, I will present my research on &ldquo;multi-view data exploration interactions&rdquo; that enable users to create and pivot many views at once, creating sets of views with coherent data value and attribute variations, through multi-modal speech and mid-air pointing gestures in large display environments. This work enables users to rapidly and efficiently generate sets of views in support of multi-view data exploration tasks, organize these views in coherent collections, and operate on sets of views collectively, rather than individually, to efficiently reach large portions of the &rsquo;data and attribute space&rsquo;. I will present three contributions: 1) an observational study of data exploration in a large display environment with speech and mid-air gestures, 2) &rsquo;Traverse&rsquo;, an interaction technique for data exploration, based on this study, which uses natural language to create and pivot sets of views, and 3) &rsquo;Ditto&rsquo;, a multi-modal speech and mid-air pointing gesture interactive environment, which utilizes the multi-view data exploration technique, in large display environments.

PhD Dissertation Announcement: Multi-view, multi-modal speech and mid-air gesture interaction for data exploration in large display environments

Achievable networking performance of applications in a supercomputer depends on the exact combination of the communication patterns of the applications and the routing algorithms used by the supercomputer. In order to achieve the highest networking performance for the applications the routing algorithms need to be designed optimally for those communication patterns. However, while communication patterns usually have a wide variation from application to application and even from phase to phase in an application, routing algorithms have a limited variation and usually are optimized for typical communication patterns. This results in high networking performance for favored communication patterns but low networking performance for others. In this paper we present approaches for improving networking performance by rebalancing load on physical links on the Blue Gene Q supercomputer. We realize our approaches in a framework called OPTIQ and demonstrate the efficacy of our framework via a set of benchmarks. Our results show that we can achieve 30% higher throughput on experiment with data and patterns from a real application. The improvement can be up to several times higher throughput than default MPI_Alltoallv used in the Blue Gene Q supercomputer for certain communication patterns. 
Keywords: multi-path data movement, BG/Q, optimization, heuristic, interconnection network, communication patterns, network load balancing

Multipath Load Balancing for M x N Communication Patterns on the Blue Gene⁄Q Supercomputer Interconnection Network

Candidate: Abeer Alsaiari 
DATE: Thursday, July 15, 2021 
TIME: 1:30PM 
LOCATION: <a href="https://uic.zoom.us/j/83926742060?pwd=RnpVMWJscGE3R3VBajJZaGo0SSs2QT09">Zoom Link</a> 
Meeting ID: 839 2674 2060 
Passcode: Mt5zBdqA 
Committee Members: Andrew Johnson (Chair and advisor), Joseph Michaelis, Wei Tang, Steve Jones (Department of Communication), Khairi Reda (Indiana University-Purdue University Indianapolis) 
Abstract: 
There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for collaborative visual data analysis and exploration. This new thrust of multi-device interfaces supports new models for complex collaboration scenarios and has great potential to support analysts in their data analysis by utilizing each device&rsquo;s capabilities. However, there are some challenges inherently associated with visual data analysis in multi-device environments (MDE). This dissertation investigates how the analytical process occurs in multi-user multi-device environments to provide a theoretical understanding of collaborative exploratory visual data analysis and better inform the design of visualization tools. First, I touched on the challenges of designing cross-device visualization tools by introducing the design and implementation of a multi-device system for collaborative visual data analysis that enables cross-device visualization sharing and simultaneous interaction. Then, through an exploratory user study, I evaluated strategies of exploratory visual data analysis in a collaborative multi-user multi-device environment. I synthesized a two-level characterization of the analysis structure from observed analysis behaviors. I observed that subjects navigate the data space in three identified exploration patterns and the analysis was primarily depth-oriented. In addition, the cost of deciding what to explore next &ldquo;Gulf of Goal Formation&rdquo; is higher in collaborative settings due to short-term memory and the recency effect. I hypothesized that visualizing the dimensions search space would increase the breadth of the analysis and reduce the decision cost. Using a between-groups study, I evaluated the effect of revealing information about what dimension&rsquo;s data space coverage(s) were investigated and what were left. The results indicate that visualizing dimensions search space increases the breadth of the analysis and reduces the decision cost by positively affecting the rate of views generation.

PhD Dissertation Announcement: Supporting Collaborative Exploratory Visual Data Analysis in Multi-device Environments

Collaborative Virtual Environments have been increasingly used in supporting group work. Several collaborative systems have come up in recent years as a way of meeting the ever changing needs of group work, with asymmetric setups being the latest addition to a diverse collection of systems. In recent years, we see attempts at multi-domain collaboration. Research on interdisciplinary collaboration reveals a need for supporting a hybrid of tools and representations that are familiar to users of different domains. 
In this dissertation, I present a prototype implementation of a Hybrid Collaborative Virtual Environment that combines multiple representations to facilitate multi-domain collaboration. Preliminary evaluation of the prototype showed that hybridization of different representations can lead to role formation due to dependency on representations, when users are limited to interacting with a single representation. Integrating shared views and shared interaction into this Hybrid Collaborative Virtual Environment with the help of a large shared display environment can help solve the representational dependency issue. Shared views and shared interactions
enable users to freely interact with any of the different representations and also eases communication amongst them. I evaluate this integration by means of a formal study having a within group design: A group of three participants perform collaborative tasks using the Hybrid Collaborative Virtual Environment with and without the integration of shared views and shared interactions. I discuss the results of the study and show how shared views and shared interaction improves the collaborative experience.

Hybrid Collaborative Virtual Environments

G. Elisabeta Marai and collaborators are recognized with the 10-years Test of Time award at ISMB 2021, the flagship conference for computational biology, for the most impactful paper on biovisualization, awarded by the BioVis ISMB COSI (a COSI is a main track at the ISMB conference). The award was made to: Smith AM, Xu W, Sun Y, Faeder JR, Marai GE. &ldquo;RuleBender: integrated modeling, simulation and visualization for rule-based intracellular biochemistry.&rdquo; BMC Bioinformatics. 2012 Dec;13(8):1-6. This research was originally presented at the 2011 Conference, and is listed a later year due to the journal publication model. RuleBender is a system for the rule-based visualization, modeling and simulation of intracellular biochemistry. 
Beyond the impressive citation record, which shows that this work informed both bioinformatics research and data visualization research, RuleBender is alive and well. It has been remarkably stable over the past ten years, it is still downloaded and used, averaging about 1000 downloads per year in the past five years alone. It has been adopted at over 40 institutions as a research and educational tool. It has been used by at least hundreds of computational biologists to do real and high impact research all over the world, from Ghana to Canada. 
Co-authors Adam, Wen and Yao were Marai&rsquo;s Masters students in 2011 and are now senior software engineers at Argo AI, Microsoft, and Amazon. Jim was a collaborator in Systems Biology at Pitt/CMU.

&ldquo;RuleBender&rdquo; Recognized with Test of Time Award

Background: Rule-based modeling (RBM) is a powerful and increasingly popular approach to modeling cell signaling networks. However, novel visual tools are needed in order to make RBM accessible to a broad range of users, to make specification of models less error prone, and to improve workflows. 
Results: We introduce RuleBender, a novel visualization system for the integrated visualization, modeling and simulation of rule-based intracellular biochemistry. We present the user requirements, visual paradigms, algorithms and design decisions behind RuleBender, with emphasis on visual global/local model exploration and integrated execution of simulations. The support of RBM creation, debugging, and interactive visualization expedites the RBM learning process and reduces model construction time; while built-in model simulation and results with multiple linked views streamline the execution and analysis of newly created models and generated networks. 
Conclusion: RuleBender has been adopted as both an educational and a research tool and is available as a free open source tool at http://www.rulebender.org. A development cycle that includes close interaction with expert users allows RuleBender to better serve the needs of the systems biology community

Contact Map with molecule compartment hierarchy (extracellular, cytoplasmic, nucleus etc).

RuleBender: integrated modeling, simulation and visualization for rule-based intracellular biochemistry

PhD Student

Kazi Shahrukh Omar

PRAGMA is hosting a Zoom-based SAGE3 presentation and live demonstration by SAGE3 Principal Investigator (PI) Jason Leigh and co-PI Luc Renambot as well as other SAGE3 Team members. In addition, SAGE2 users Prapaporn (Nan) Rattanatamrong of Thammasat University in Thailand and Jason Haga of the National Institute of Advanced Industrial Science and Technology (AIST) in Japan start the presentation with several ongoing SAGE2 use cases. 
This presentation is part of the 2021 Re-Connect Webinar series organized by the PRAGMA (Pacific Rim Application and Grid Middleware Assembly) organization. Many PRAGMA members are long-time SAGE users. 
Since receiving an NSF award for SAGE3, the Smart Amplified Group Environment, in May 2020, the SAGE3 Team has been hard at work designing and developing SAGE3. An alpha version of the SAGE3 remote collaboration platform is being presented. 
SAGE3 is being designed to integrate AI technologies with applications, visualization workflows and collaboration services. SAGE3 takes advantage of major technological and sociological changes that have occurred since SAGE2 was introduced in 2013. Notably, web browsers and languages are maturing; scientific communities are relying on notebooks; science workloads incorporate AI programs and programmable infrastructure; and, given the pandemic, the workplace is changing to hybrid office/home environments that require not just video-teleconferencing but data-driven tools for largely distributed groups. 
All are welcome to attend! No pre-registration is required. 
Asia: Aug 27 (Friday), 8am(WIB/ICT) / 9am(MYT/TST/PST) / 10am(JST/KST) 
US: Aug 26 (Thursday), 6pm(PDT) / 8pm(CDT) / 9pm(EDT) / 3pm(HST) 
Zoom: 
<a HREF="https://hawaii.zoom.us/j/96474855666?pwd=UTBNYU4wQkJnc1c2cGwxdkxCTFBCZz09
">https://hawaii.zoom.us/j/96474855666?pwd=UTBNYU4wQkJnc1c2cGwxdkxCTFBCZz09</a> 
Meeting ID: 964 7485 5666 
Passcode: sage3 
The SAGE3 presentation and demonstration were first done for the CineGrid organization in June 2021. However, if you were unable to see it then, we urge you to tune into the PRAGMA webinar and then ask questions of SAGE3 Team members. This presentation will be recorded and available at a later date. 
SAGE3 is being developed by the Laboratory for Advanced Visualization and Applications (LAVA) and the Hawaii Data Science Institute (HI-DSI) at University of Hawaiʻi at Mānoa (UHM), the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC), and the Sanghani Center for AI and Data Analytics (Sanghani) at Virginia Tech (VT). SAGE3 receives major funding from the U.S. National Science Foundation, awards #2004014 (UHM), #2003800 (UIC) and #2003387 (VT). SAGE, SAGE2 and SAGE3 are trademarks of the University of Illinois Board of Trustees.

Introducing SAGE3 - Zoom event hosted by PRAGMA

For those unable to attend the SAGE3 presentations and live demonstrations by SAGE3 Principal Investigator (PI) Jason Leigh and co-PI Luc Renambot as well as other SAGE3 Team members to PRAGMA and CineGrid earlier this summer, the sessions are now available via YouTube. 
<a href="https://youtu.be/UprsquVI6yo">Access the PRAGMA recording.</a> 
The same SAGE3 virtual presentation and demonstration was done for the CineGrid community; however, the Questions & Answers following the demo were very different. <a href="https://www.youtube.com/watch?v=PyE09TxwhPE">Access the CineGrid recording.</a> 
SAGE3 is being developed by the Laboratory for Advanced Visualization and Applications (LAVA) and the Hawaii Data Science Institute (HI-DSI) at University of Hawaiʻi at Mānoa (UHM), the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC), and the Sanghani Center for AI and Data Analytics (Sanghani) at Virginia Tech (VT). SAGE3 receives major funding from the U.S. National Science Foundation (NSF), awards #2004014 (UHM), #2003800 (UIC) and #2003387 (VT). 
SAGE, SAGE2 and SAGE3 are trademarks of the University of Illinois Board of Trustees.

SAGE3 Principal Investigator (PI) Jason Leigh, co-PI Luc Renambot and other SAGE3 Team members introduce and demo SAGE3 to the PRAGMA community as part of its 2021 Re-Connect Webinar series.

SAGE3 Introduction & Demo to PRAGMA and CineGrid - RECORDINGS AVAILABLE

EVL Research Experience Undergraduate (REU) and Computer Science alum (BS 2020) Jonathan Vega discusses the impact of his data visualization research. Under EVL faculty member and Computer Science associate professor G. Elisabeta Marai&rsquo;s supervision Vega undertook the task of studying and visualizing contrails - water vapor and engine exhaust emitted from airplanes. His work helped create a simulation that captured the part that contrails play in climate change. 
Most significantly, Vega&rsquo;s research led him to discover more about the relationship between contrails, climate change and the potential impact upon the Colorado River and his Native American tribe&rsquo;s daily lives, including food, tribal traditions and ceremonies. Unbeknownst to Vega when he embarked on his REU was how the data visualization of contrails could be directly linked to his tribal community and global warming. 
Vega&rsquo;s work earned him the UIC Chancellor&rsquo;s Undergraduate Research Award in 2020. 
See the full article in the <a href="https://cs.uic.edu/news-stories/data-visualizations-unite-water-and-air/">Computer Science News</a>.

UIC Alum Jonathan Vega&rsquo;s Data Visualization Research Featured in CS News

Construction activities for the new $117.8 million, 135,000-square-foot Computer Design Research and Learning Center to be located at 900 W. Taylor St. is underway. The CDRLC will support the Department of Computer Science&rsquo;s growing student enrollment with 16,000 square feet of classroom space for collaborative teaching and learning environments, as well as 35 labs for faculty research activities in the computer science and engineering disciplines. 
The CDRLC will be the new home for the Electronic Visualization Laboratory featuring the latest technological and computing innovations to support research in advanced visualization, networking and collaborative environments. 
Read more about the CDRLC groundbreaking in the <a href="https://today.uic.edu/uic-breaks-ground-on-new-computer-design-research-and-learning-center">July 15th issue of UIC Today.</a> 
See a live webcam of the building&rsquo;s progress via the <a href="https://webcampub.multivista.com/index.cfm?fuseaction=aPublicWebcam.embed&WebcamPublicEmbedUID=5D4DCF05-AB2B-4770-A090-D79BE467F958">Multivista Construction Documentation webcam.</a>

New Computer Design Research and Learning Center Breaks Ground

SEPTEMBER 22, 2021 - The UIC Electronic Visualization Laboratory&rsquo;s (EVL) long-time collaborator, the University of California, San Diego (UCSD), received a National Science Foundation (NSF) grant, to start October 2021, to Enhance and Sustain their CHASE-CI project. As part of EVL&rsquo;s ongoing commitment to this research effort, EVL will host GPU resources and data servers to enhance and extend UCSD&rsquo;s Pacific Research Platform (PRP) Nautilus Hypercluster through 2021-2024. 
In 2017, UCSD received an NSF award for CHASE-CI, the Cognitive Hardware And Software Ecosystem - Community Infrastructure, which initially provided a community of Computer & Information Science and Engineering researchers doing Machine Learning and Computational Media applications with a distributed cluster containing 256 Graphics Processing Units (GPUs). This cluster - named Nautilus - was originally distributed across 10 California campuses and built on PRP. The NSF-funded PRP project focused on integrating California&rsquo;s high-speed regional research & education network (for fast data transfers among campuses) with DTNs, or Data Transfer Nodes, which are specialized storage systems located at each campus edge, optimized to quickly send and receive large data flows. 
Early on, EVL was among 50 UCSD partners to connect to PRP, thanks to UIC&rsquo;s high-bandwidth connectivity (100 Gigabits per second) to national research and education networks via the StarLight International/National Communications Exchange Facility in downtown Chicago. At that time, UCSD provided EVL with the campus&rsquo;s first DTN, which has since been integrated into the Nautilus cluster, providing 192 Terabytes (TB) of data storage for Big Data applications. More recently, as part of its ongoing PRP award, UCSD is providing EVL with an additional 216 TB of data storage. 
Over the past 5 years, Nautilus has grown. PRP interoperability with national and international research and education networks creates a high-capacity data-centric &ldquo;freeway system&rdquo; now stretching across the nation and across oceans that moves Nautilus data among DTNs located at other research institutions. UCSD&rsquo;s Nautilus cluster contains hundreds of GPUs and many Terabytes of storage. While 25% of the computational hardware and data resources are provided by the UCSD CHASE-CI award, 75% is provided by other researchers and institutions who voluntarily contribute resources, building a shared platform that they jointly own and control. EVL has contributed some of its own GPU nodes to Nautilus, with plans to federate more of its GPU resources over time. However, any Computer & Information Science and Engineering researcher can get an account on Nautilus, whether or not they contribute resources. 
This new CHASE-CI Enhance and Sustain award aims to enhance and sustain Nautilus&rsquo;s user community. EVL, as a UCSD partner institution, will receive an 8 GPU Nautilus Machine Learning node, to be integrated with Nautilus, to provide the community with additional GPUs and data services. While EVL currently has substantial computing facilities to support the needs of UIC College of Engineering faculty who require GPUs and data storage, being part of the CHASE-CI Enhance and Sustain grant and integrating the Lab&rsquo;s resources with Nautilus expands faculty and student access to national (and international) scale community computational and data resources. 
NSF award #2120019 to UCSD, ENS: CHASE-CI, 2021-2024 
NSF award #1730158 to UCSD, CHASE-CI, 2017-2020 
NSF award #1541349 to UCSD, PRP, 2015-2022

EVL Continues UCSD Collaboration to Enhance and Sustain CHASE-CI

The COVID-19 pandemic has upended the world over the past year, in a health crisis not seen since the 1918 flu pandemic. As of July 2021, the disease has caused the death of more than 4 million people worldwide, 600,000 in the US alone. While waiting for an
effective vaccine amidst the surging cases worldwide, governments adopted measures to inhibit the spread of the disease. Despite these efforts, waves of COVID-19 infection with varying characteristics, continued to ravage communities [24], highlighting the necessity to understand the spatio-temporal complexity of the problem. Forecast models are among the tools available to public health experts and policymakers to predict likely pandemic outcomes (i.e., number of cases, deaths) and prepare for different scenarios. In the past year, several models have been proposed by a myriad of experts, and using vastly different assumptions, methodologies, parameters and data sources. As a result, models produce a range of sometimes radically different forecasts, especially at higher spatial resolutions, limiting their use by decision makers and the public in general. In order to address this problem, more recently, ensemble models have been proposed for COVID-19 forecasts, with promising results. Our research in this area has resulted in COVID-19 EnsembleVis, a web-based interactive visual interface that allows the assessment of the errors of ensembles and individual models by enabling users to effortlessly navigate through and compare the outputs of models considering their space and time dimensions. COVID-19 EnsembleVis enables a more detailed understanding of uncertainty and the range of forecasts generated by individual models.

Enabling the visual analysis of COVID-19 ensemble forecast models with COVID-19 EnsembleVis.

COVID-19 EnsembleVis: Visual Analysis of County-level Ensemble Forecast Models

EVL in collaboration with Argonne National Laboratory (ANL) hosted a satellite event for VIS2021 featuring a presentation from invited speaker Dr. Alan Keahey, VP for Data Science Visualization at Conversant Media, student lightning talks, a conference virtual watch party, followed by a reception near UIC on Tuesday, October 26, 2021. 
Due to COVID, the IEEE VIS conference was held virtually, complemented by local satellite in-person events to promote face-to-face networking between academic and professional VIS communities. 
Watch a <a href="https://youtu.be/mYEysIVb9Vc">video synopsis on EVLTube</a>. 
<a href="https://www.evl.uic.edu/vis-chicago/">More information on the IEEE VIS2021 Chicago Satellite Event.</a> 
This event is being organized by Fabio Miranda (EVL) and Victor Mateevisti (ANL).

EVL Hosts Satellite Event for IEEE VIS2021

For the last several years EVL faculty member and CS associate professor Liz Marai has been collaborating with the University of Texas M.D. Anderson Cancer Center and the University of Iowa to develop tools to help oncologists make better assessments for treating cancer patients. More recently, Liz Marai and co-investigator Xinhua Zhang also from UIC have been developing a scalable artificial intelligence system based on real patient data to serve as a decision support tool for physicians treating head and neck cancer patients. This research is funded by the National Institutes of Health, National Cancer Institute Award #3R01CA258827-01S1, entitled &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo; with principal investigators from UIC, the University of Texas, and the University of Iowa. 
See the full article in the <a href="https://engineering.uic.edu/about/coe-news/rise/fall-2021/better-ai-for-better-decisions/">College of Engineering&rsquo;s CEO News.</a>

EVL Artificial Intelligence Research Featured in COE News

Exploring large virtual environments, such as cities, is a central task in several domains, such as gaming and urban planning. VR systems can greatly help this task by providing an immersive experience; however, a common issue with viewing and navigating a city in the traditional sense is that users can either obtain a local or a global view, but not both at the same time, requiring them to continuously switch between perspectives, losing context and distracting them from their analysis. In this paper, our goal is to allow users to navigate to points of interest without changing perspectives. To accomplish this, we design an intuitive navigation interface that takes advantage of the strong sense of spatial presence provided by VR. We supplement this interface with a perspective that warps the environment, called UrbanRama, based on a cylindrical projection, providing a mix of local and global views. The design of this interface was performed as an interactive process in collaboration with architects and urban planners. We conducted a qualitative and a quantitative pilot user study to evaluate UrbanRama and the results indicate the effectiveness of our system in reducing perspective changes, while ensuring that the warping doesn&rsquo;t affect distance and orientation perception. 
Index Terms: Virtual Reality, VR Navigation, Cylindrical deformation. 
<a href="https://youtu.be/M8BFZnxq-Qg">Watch the video on YouTube.</a>

The UrbanRama mode in our urban VR system deforms the city allowing the user to obtain both a local as well as a global perspective of the urban environment.

UrbanRama: Navigating Cities in Virtual Reality

The graph convolutional network (GCN) has recently achieved promising performance of 3D human pose estimation (HPE) by modeling the relationship among body parts. However, most prior GCN approaches suffer from two main drawbacks. First, they share a feature transformation for each node within a graph convolution layer. This prevents them from learning different relations between different body joints. Second, the graph is usually defined according to the human skeleton and is suboptimal because human activities often exhibit motion patterns beyond the natural connections of body joints. To address these limitations, we introduce a novel Modulated GCN for 3D HPE. It consists of two main components: weight modulation and affinity modulation. Weight modulation learns different modulation vectors for different nodes so that the feature transformations of different nodes are disentangled while retaining a small model size. Affinity modulation adjusts the graph structure in a GCN so that it can model additional edges beyond the human skeleton. We investigate several affinity modulation methods as well as the impact of regularizations. Rigorous ablation study indicates both types of modulation improve performance with negligible overhead. Compared with state-of-the-art GCNs for 3D HPE, our approach either significantly reduces the estimation errors, e.g., by around 10%, while retaining a small model size or drastically reduces the model size, e.g., from 4.22M to 0.29M (a 14.5 X reduction), while achieving comparable performance. Results on two benchmarks show our Modulated GCN outperforms some recent states of the art. 
<a href="https://github.com/ZhimingZo/Modulated-GCN">Download Code</a> 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Modulated Graph Convolutional Network for 3D Human Pose Estimation

The National Institutes of Health: National Cancer Institute has awarded Liz Marai and collaborators at the University of Texas M.D. Anderson Cancer Center a Research Supplement to Promote Diversity in Health-Related Research Programs, for federal award R01CA258827, &ldquo;Longitudinal Spatial-Nonspatial Decision Support for Competing Outcomes in Head and Neck Cancer Therapy&rdquo; to run 2021 -2024. The award will support the interdisciplinary training of Dr. Rachel Glenn, an individual from a disadvantaged background (as defined by PA-21-071), under the direction of PIs Marai (UIC) and Fuller (U of Texas).

G. Elisabeta Marai Awarded NIH Diversity Supplement

Thanks to new NSF support, beginning May 2020, the SAGE team has been hard at work designing and developing SAGE3, the Smart Amplified Group Environment. This new funding enables us to take SAGE2, introduced in 2013, to a new level, integrating Artificial Intelligence technologies with applications, visualization workflows and collaboration services. The team demonstrated an early version of SAGE3 at two webinars this summer, hosted by the CineGrid and PRAGMA organizations (information below). 
In the past, we have looked forward to seeing old users and meeting new users at the annual Supercomputing (SC) conference, though team members will not be attending this year. However, we continue to be hard at work on SAGE3 and are planning a number of exciting announcements for early December. 
1. A new SAGE3 website. 
2. An &ldquo;alpha release&rdquo; of SAGE3, which is an early, first release of the software, for user testing and feedback. 
3. A &ldquo;how-to&rdquo; video explaining current SAGE3 features. 
4. A new user forum hosted on Slack (tinyurl.com/sagecommunity). 
Given SAGE3 is in an early prototype phase, we expect our users to continue using SAGE2 for work, meetings, classes, etc. We will continue to answer support questions for SAGE2 for the foreseeable future, but will not introduce new releases. 
Stay tuned for SAGE3 news updates as they become available !! 
Maxine Brown 
SAGE team member 
<a href="mailto:maxine@uic.edu">maxine@uic.edu</a> 
<a href="https://youtu.be/PyE09TxwhPE">SAGE3 presentation at CineGrid, June 2021</a> 
<a href="https://youtu.be/UprsquVI6yo">SAGE3 presentation at PRAGMA, August 2021</a> 
SAGE3 is being developed by the Laboratory for Advanced Visualization and Applications (LAVA) and the Hawaii Data Science Institute (HI-DSI) at University of Hawaiʻi at Mānoa (UHM), the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC), and the InfoVis Lab at Virginia Tech (VT). SAGE3 receives major funding from the U.S. National Science Foundation (NSF), awards #2004014 (UHM), #2003800 (UIC) and #2003387 (VT). 
SAGE2 received major NSF support, award #ACI-1441963, to UHM and UIC. 
SAGE, SAGE2 and SAGE3 are trademarks of the University of Illinois Board of Trustees.

Exciting SAGE3 announcements planned for early December

EVL faculty member and Computer Science associate professor Liz Marai has been recognized as a 2020/2021 College of Engineering Teaching Award Recipient. The recipients of the teaching award are traditionally recognized for their excellence in teaching and their contributions to the curriculum. In addition, some of the awardees have rapidly adopted new technologies, modalities, and innovative thinking to achieve learning objectives as UIC transitioned to remote learning during the 2020 pandemic. 
Marai is among the following award recipients: 
BME: Dave Eddington, Dieter Klatt, Zhinan Wang 
CHE: Gang Cheng, Alan Zdunek 
CME: Ahmed Abokifa, Didem Ozevin, Bo Zou 
CS: Gonzalo Bello, Joe Hummel, Chris Kanich, Liz Marai, Natalie Parde, Shanon Reckinger 
ECE: Vahe Caliskan, Alan Feinerman, James Kosmach, Wenjing Rao, Renata Revelo 
MIE: Jonathan Komperda, Scott Reckinger, William Ryan, Yeow Siow, Jamison Szwalek 
Congratulations on this well-deserved honor!

Associate Professor Liz Marai Recognized as 2020/2021 COE Teaching Award Recipient

In recent years deep meta-reinforcement learning has extended the applicability of reinforcement learning (RL) algorithms: by integrating recurrent networks, trained models have the ability to quickly adapt to new unseen environments without the need for further backpropagation. These models, however, cannot adapt without having information on past rewards, and are therefore not directly applicable to a sequential decision-making setting in which multiple steps are required before observing the final reward. 
One of the main applications affected by this limitation are dynamic treatment regimes, i.e. the problem of selecting the optimal medical treatment sequence for a patient at each step, keeping into account the complete past treatment history. By expanding deep meta-reinforcement learning to handle sequential decisions, a model would be able to prescribe the optimal treatment for each patient even if the patient&rsquo;s (or physician&rsquo;s) preferences on the outcome were never encountered by the model in training. 
We propose a recursive deep meta-reinforcement learning approach which enables the model of each decision of the sequential process to learn from and adapt to unseen circumstances by recursively integrating the feedback of the models of other decisions in the process. We evaluate our approach on synthetic two-step processes with fixed transition probabilities but varying reward functions, to test the models' ability to propagate environment information from the final reward to intermediate steps. Finally, we train our model on a dataset of three-step chemo-radiotherapeutic and surgical treatment of oropharyngeal squamous cell carcinoma patients, proving our approach&rsquo;s ability to optimally handle previously unseen patient's preferences on survival and toxicity outcomes.

Main features, outcomes, and decisions of the OPC treatment sequence

Recursive meta-Reinforcement Learning for Personalized Sequential Dynamic Treatment Policies

Former EVL student and graduate research assistant Elisa Tardini is recognized as a recipient of the Outstanding Thesis/Dissertation Award from the Graduate College at the University of Illinois at Chicago for her thesis entitled &ldquo;Recursive deep meta-reinforcement learning for personalized sequential dynamic treatment policies.&rdquo; This research focuses on a recursive deep meta-reinforcement learning approach to develop a model to prescribe an optimal treatment for patients.

2021 Outstanding Thesis/Dissertation Award to Elisa Tardini

Pose guided person image generation means to generate a photo-realistic person image conditioned on an input person image and a desired pose. This task requires spatial manipulation of the source image according to the target pose. However, convolutional neural networks (CNNs) are inherently limited to geometric transformations due to the fixed geometric structures in their building modules, i.e., convolution, pooling and unpooling, which cannot handle large motion and occlusions caused by large pose transform. This paper introduces a novel two-stream context-aware appearance transfer network to address these challenges. It is a three-stage architecture consisting of a source stream and a target stream. Each stage features an appearance transfer module, a multi-scale context module and two-stream feature fusion modules. The appearance transfer module handles large motion by finding the dense correspondence between the two-stream feature maps and then transferring the appearance information from the source stream to the target stream. The multi-scale context module handles occlusion via contextual modeling, which is achieved by atrous convolutions of different sampling rates. Both quantitative and qualitative results indicate the proposed network can effectively handle challenging cases of large pose transform while retaining the appearance details. Compared with state-of-the-art approaches, it achieves comparable or superior performance using much fewer parameters while being significantly faster. 
Keywords: Person image generation, Appearance transfer, Multi-scale context, Efficient image generation 
Relevant Funding: The COMPaaS DLV project (NSF award CNS-1828265) 
<a href="https://doi.org/10.1016/j.patcog.2021.108451">https://doi.org/10.1016/j.patcog.2021.108451</a>

Exploiting appearance transfer and multi-scale context for efficient person image generation

EVL faculty, assistant professor of Computer Science Fabio Miranda is joined by colleagues Jon Froehlich (University of Washington), Maryam Hosseini (Rutgers University) in organizing a 90-minute workshop on urban accessibility at the <a href="http://wp1066709.server-he.de/sdss2021/">2021 Spatial Data Science Symposium</a>. 
The interactive session entitled &ldquo;The Future of Global-Scale Spatial Data Collection and Analyses on Urban (in)Accessibility for People with Disabilities&rdquo; will bring together geographers, computer scientists, public health researchers, disability scholars, and beyond to talk about opportunities and challenges related to data collection, analysis, and tools for urban (in)accessibility. This virtual event is scheduled for Monday, December 13 12:15pm CT. 
<a href="https://www.airmeet.com/e/52c9e110-4848-11ec-a698-619a9d382b96">Register for SDSS</a>. 
Read more about this <a href="http://sdss2021.spatial-data-science.net/session_pdf/1-The%20Future%20of%20Urban%20Accessibility_%20Data%20Collection,%20Analytics,%20and%20Visualization.pdf">specific conference session</a>.

The Future of Global-Scale Spatial Data Collection and Analyses on Urban (in)Accessibility for People with Disabilities

2021 Spatial Data Science Symposium

SAGE2 (Scalable Amplified Group Environment) is the de-facto platform to support group work on wide-band display environments. Unlike most web applications, the SAGE environment, due to the nature of its collaborative model, needs a nuanced handling of security aspects. This paper details the security requirements of SAGE2, the Identity and Access Control model that was developed to address those requirements, and the details of the Identity and Access Management system that the SAGE team implemented based on this new model. Further, we present a comparison of this new system with some of the popular collaboration platforms to highlight the uniqueness of SAGE2 integrated with this new Identity and Access Management system. 
Keywords: security, access control, collaborative environment, wide-band displays

Examples of SAGE2 wide-band displays

Securing Collaborative Work in Wide-band Display Environments

The SAGE team has been hard at work designing and developing SAGE3, the NSF-funded Smart Amplified Group Environment, which is a total rewrite of SAGE2. 
SAGE3 ALPHA RELEASE AND WEBSITE 
<a href="https://sage3.sagecommons.org/">SAGE3 alpha version for Linux, Windows and MacOS</a> is now available for download. For those who wish to test SAGE3 and provide our team with constructive feedback, go to the new SAGE3 website: (1) download the software and display client and (2) watch the &ldquo;How To&rdquo; video. Note: Users who tested earlier SAGE3 versions should delete old versions and use the latest one on the website. 
SAGE2 
For users who rely on SAGE2 for production (collaborative research, teaching, meetings), continue to use. We will answer support questions for the foreseeable future, but will not introduce new releases. The final release and documentation can be found on the <a href="https://sage2.sagecommons.org/">SAGE2 website</a>. 
SAGE3 / SAGE2 COMMUNITY USER FORUMS 
New SAGE3/SAGE2 Community user forums are now hosted on Slack; to join: 
<a href="https://tinyurl.com/sagecommunity">tinyurl.com/sagecommunity</a>. 
<ul>
<li>For SAGE2, subscribe to the sage2-support channel.</li>
<li>For SAGE3, subscribe to the sage3-support channel.</li>
</ul>

Screen shot of SAGE3 session.

SAGE3 Alpha Version Available - Users: Please Help Test and Provide Feedback

MS Student

Veronica Grosso

Figures within biomedical articles present essential evidence of the relevance of a publication in a curation workflow. In particular, visual cues of the image modality or experimental methods can help expert curators identify relevant papers from an increasing number of publications. Automating the identification of these content-bearing images can thus be helpful in computer-assisted curation. However, the paucity of labeled datasets and the specialized training required to label such images hinder the development of such tools. To address this problem, we present the design of ANIMO, a labeling system that integrates extraction and segmentation tools to ease the annotation burden. We first introduce two taxonomies of image modalities and experimental methods, derived in collaboration with curators. On the back-end of the system, we process batches of documents and create a labeling task per document. At the front-end, expert curators can access these tasks through a web interface and access the article of interest. We describe the evaluation of this system by a group of biocurators, and the human factor lessons learned from this interdisciplinary experience. 
doi: 10.1109/BIBM52615.2021.9669898

Front-end interface for labeling a document showing a taxonomy adapted for the CORD-19 dataset.

ANIMO: Annotation of Biomed Image Modalities

EVL Affiliated Faculty Assistant Professor, CS

Nikita Nandish Soni

Assistant professor Nikita Nandish Soni has joined the faculty of computer science at UIC, specializing in human-computer interaction. As affiliated EVL faculty, Dr. Soni will continue her research in designing natural user interactions for multi-touch spherical displays that support collaboration and learning. We look forward to her contributing to the design, development and evaluation of intelligent user-interface technologies, especially for children and families. 
<a href="https://cs.uic.edu/news-stories/cs-department-adds-a-new-human-computer-interaction-researcher/">Read the full article in the COE/CS News.</a>

New Faculty Member in Human-Computer Interaction Joins Computer Science

In the last 20 years, extreme weather-related events like floods, landslides, droughts, and wildfires have caused the death of 1.23 million people and a loss of 2.97 trillion dollars. Studies show that low and lower-middle-income countries are the most impacted ones given the lack of investment in disaster risk management. To reduce the impact of these events, weather researchers have been developing numerical weather models that inform public agencies about the impending extreme events in advance. Despite being powerful tools, these models can suffer from several sources of uncertainty, ranging from the approximation of micro-scale physical processes to the location-dependent calibration of parameters, which is especially critical in developing countries. To minimize uncertainty effects, researchers generate several different weather scenarios to compose an ensemble of simulations that typically are inspected using manual, laborious, and error-prone approaches. In this paper, we propose an interactive visual analytics system, called X-Weather, developed in close collaboration with weather researchers from Brazil. Our system contributes a set of statistics and probability-based visualizations that allows the assessment of extreme weather events by effortlessly navigating through and comparing ensemble members. We demonstrate the effectiveness of the system through two case studies analyzing tragic events that happened in the mountain region of Rio de Janeiro in Brazil. 
Keywords: Visual analytics, Weather visualization, Ensemble visualization 
ISSN 0097-8493 
<a href="https://www.sciencedirect.com/science/article/pii/S0097849322000073">https://www.sciencedirect.com/science/article/pii/S0097849322000073</a>

The X-Weather interface.

Visualizing simulation ensembles of extreme weather events

While designing sustainable and resilient urban built environment is increasingly promoted around the world, significant data gaps have made research on pressing sustainability issues challenging to carry out. Pavements are known to have strong economic and environmental impacts; however, most cities lack a spatial catalog of their surfaces due to the cost-prohibitive and time-consuming nature of data collection. Recent advancements in computer vision, together with the availability of street-level images, provide new opportunities for cities to extract large-scale built environment data with lower implementation costs and higher accuracy. In this paper, we propose CitySurfaces, an active learning-based framework that leverages computer vision techniques for classifying sidewalk materials using widely available street-level images. We trained the framework on images from New York City and Boston and the evaluation results show a 90.5% mIoU score. Furthermore, we evaluated the framework using images from six different cities, demonstrating that it can be applied to regions with distinct urban fabrics, even outside the domain of the training data. CitySurfaces can provide researchers and city agencies with a low-cost, accurate, and extensible method to collect sidewalk material data which plays a critical role in addressing major sustainability issues, including climate change and surface water management. 
Keywords: Sustainable built environment; Surface materials; Urban heat island; Semantic segmentation; Sidewalk assessment; Urban analytics; Computer vision 
ISSN 2210-6707 
<a href="https://www.sciencedirect.com/science/article/pii/S2210670721008933">https://www.sciencedirect.com/science/article/pii/S2210670721008933</a>

Predictions of the model on the held-out test set. Fine details and boundaries of objects like poles, plants, wooden sticks, and fire hydrants are very precisely predicted.

CitySurfaces: City-scale semantic segmentation of sidewalk materials

On January 25, 2022, EVL faculty member assistant professor Fabio Miranda was interviewed by Univision TV to discuss environmental contamination and the implication to Chicagoland Latino communities.  According to an investigation and recent study by UIC and Miranda &ldquo;the southwest of the city has a large presence of pollutants disproportionately affecting Latino communities.&rdquo; The study explores issues of local government perpetuating environmental racism. Miranda&rsquo;s research focuses on how visual data analytics can help address different problems cities face by integrating data on different resolutions and from different sources.

Data Science

Bibliography

pubs (64)

Related Topics

people (22)

events (62)

research (11)

Contact