This collaborative research project led by the colleagues at the University of Delaware aims to develop and advance tools for using image-data appearing in scientific publications, in addition to text, in order to support beneficial, targeted access to the biomedical literature. 
UIC's principal investigator, Computer Science associate professor, G. Elisabeta Marai will be responsible for the image analysis algorithms to distinguish between gel and plate images, the image analysis algorithms to distinguish between Electronic Microscopy vs Fluorescent Microscopy vs Light Microscopy and X-Rays, and the design of the visual interface to present results to end users. She will lead the design, development and implementation of the feature extraction and selection tools and techniques related to this research project.

Incorporating Image-based Features into Biomedical Document Classification

COMPaaS DLV, COMposable Platform as a Service Instrument for Deep Learning & Visualization, was purchased from Liqid, Inc., with NSF funds. Its 4th-generation computer architecture, composable infrastructure, treats computer components - computer processing units (CPUs), graphical processing units (GPUs), networking, memory and storage - as pools of resources. It is: 
<ul><li>Highly flexible. Each application defines the resources it needs, and the operating system composes, or combines, resources on the fly.</li> 
<li>Scalable. As more infrastructure is added, it’s auto-integrated with the existing infrastructure and becomes part of the pool of capacity.</li> 
<li>High-throughput. System components are interconnected with a high-speed internal fabric, enabling Big Data to quickly move among CPU, GPU, memory and storage at optimum speeds with little to no bottlenecks.</ul></li> 
Funded in part by NSF award #CNS-1828265.

Visual Data Science: Advanced Cyberinfrastructure Systems (poster session)

Ph.D. Student: Abeer Alsaiari 
Committee: 
Prof. Andrew Johnson, Chair and Advisor 
Prof. Joseph Michaelis 
Prof. Wei Tang 
Prof. Steve Jones (Department of Communication) 
Prof. Khairi Reda (Indiana University-Purdue University Indianapolis) 
Date: Monday, March 9th 
Time: 1:30 pm 
Location: ERF 3036 
Abstract: 
There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. This new thrust of multi-device interfaces supports new models for complex collaboration scenarios, and have great potential to support analysts in their data analysis by utilizing each device&rsquo;s capabilities. However, there are some challenges inherently associated with visual data analysis in multi-device environments (MDE). As the analytical process is underway, analytical components become scattered among different devices and displays. Building a mental model of the analysis flow can render the analytical process more challenging as it would be difficult to track many visualizations and therefore difficult for analysts to keep track of all the prior analyses. The primary goal of my thesis research is to understand the characterization of the analysis process, with regard to the use and formation around devices, and to support the design with visualization of the search space, which will allow users to efficiently explore their data.

Preliminary Examination Announcement: &ldquo;Supporting Collaborative Visual Data Analysis in Multi-device Environment&rdquo;

Maxine Brown (UIC PI), Andrew Johnson (UIC co-PI), Luc Renambot (UIC co-PI) 
Jason Leigh (University of Hawaii Manoa PI) Christopher North (Virginia Polytechnic PI) 
NSF Award Number: 2003800 
Award Period of Performance: 05/15/2020 - 04/30/2025 
Total Amount Obligated: $2,250,000 
UIC Computer Science Electronic Visualization Lab (EVL) Director Maxine Brown, PI, co-PIs UIC CS Professor Andy Johnson and UIC CS Research Associate Professor Luc Renambot, and Senior Personnel EVL Senior Research Programmer Lance Long are awarded a $2.25 million dollar National Science Foundation grant entitled &ldquo;Collaborative Research: CSSI Frameworks: SAGE3: Smart Amplified Group Environment for Harnessing the Data Revolution.&rdquo; The grant runs from May 2020 to April 2025. It&rsquo;s part of an overall $5 million project, that also includes a team from University of Hawaii Manoa led by Jason Leigh and a team from Virginia Polytechnic led by Christopher North. 
ABSTRACT 
The Big Data revolution necessitates the use of sophisticated tools such as Artificial Intelligence (AI) and Data Visualization to harness the sheer volume, velocity and variety of datasets that are becoming the norm. However, it is the research community that must make sense of the data being amassed, so cyberinfrastructure must extend to people. SAGE3 (Smart Amplified Group Environment) puts the &ldquo;human in the loop&rdquo; by providing scientists with an intuitive framework that integrates state-of-the-art AI technologies with applications, workflows, smart visualizations and collaboration services to help them access, share, explore and analyze their data, come to conclusions, and make decisions with greater speed, accuracy, comprehension and confidence. 
SAGE3 builds upon the NSF-funded first-generation SAGE (Scalable Adaptive Graphics Environment) and second-generation SAGE2 (Scalable Amplified Group Environment), which are today&rsquo;s de facto operating systems to manage Big Data on tiled display walls. Its development focuses on two fundamental components: AI-enhanced &ldquo;smart&rdquo; services and advanced computing resource orchestration to support reproducible work models for secure collaborative work. SAGE3 amplifies user productivity, providing them with commercially available and open-source AI solutions, which autonomously and transparently analyze data while continually learning and improving through user interactions. SAGE3 makes AI technologies broadly accessible, not just a privilege for the technically savvy. SAGE3 further democratizes AI by using Data Visualization to help interpret and explain AI models so users better understand how AI came to its decisions, which engenders user trust and can help identify potentially prejudiced or biased models. 
SAGE3 augments every step of the scientific discovery enterprise - from quickly summarizing large data, to finding trends and similarities or anomalies among one or more linked datasets, to communicating findings to scientists, public policy and government officials, and the general public, to educating the next-generation workforce. Ultimately, it is the scientists and future scientists who must Harness the Big Data revolution to solve the nation&rsquo;s grand challenge problems that will benefit society as a whole - from studying the diversity of life on Earth, to understanding the Earth and its systems from satellite imagery of its poles, to developing response scenarios for natural disasters such as landslides and pandemics that impact the citizens and economies of the world.

UIC Computer Science/EVL awarded $2.25 Million from NSF for SAGE3

UIC PI: Maxine Brown, co-PIs: Andrew Johnson, Luc Renambot 
University of Hawaii Manoa PI: Jason Leigh 
Virginia Polytechnic PI: Chirstopher North 
NSF Award Number: 2003800 
Award Period of Performance: 05/15/2020 - 04/30/2025 
Total Amount Obligated: $2,250,000 
The Big Data revolution necessitates the use of sophisticated tools such as Artificial Intelligence (AI) and Data Visualization to harness the sheer volume, velocity and variety of datasets that are becoming the norm. However, it is the research community that must make sense of the data being amassed, so cyberinfrastructure must extend to people. SAGE3 (Smart Amplified Group Environment) puts the &ldquo;human in the loop&rdquo; by providing scientists with an intuitive framework that integrates state-of-the-art AI technologies with applications, workflows, smart visualizations and collaboration services to help them access, share, explore and analyze their data, come to conclusions, and make decisions with greater speed, accuracy, comprehension and confidence. 
SAGE3 builds upon the NSF-funded first-generation SAGE (Scalable Adaptive Graphics Environment) and second-generation SAGE2 (Scalable Amplified Group Environment), which are today&rsquo;s de facto operating systems to manage Big Data on tiled display walls. Its development focuses on two fundamental components: AI-enhanced &ldquo;smart&rdquo; services and advanced computing resource orchestration to support reproducible work models for secure collaborative work. SAGE3 amplifies user productivity, providing them with commercially available and open-source AI solutions, which autonomously and transparently analyze data while continually learning and improving through user interactions. SAGE3 makes AI technologies broadly accessible, not just a privilege for the technically savvy. SAGE3 further democratizes AI by using Data Visualization to help interpret and explain AI models so users better understand how AI came to its decisions, which engenders user trust and can help identify potentially prejudiced or biased models. 
SAGE3 augments every step of the scientific discovery enterprise - from quickly summarizing large data, to finding trends and similarities or anomalies among one or more linked datasets, to communicating findings to scientists, public policy and government officials, and the general public, to educating the next-generation workforce. Ultimately, it is the scientists and future scientists who must Harness the Big Data revolution to solve the nation&rsquo;s grand challenge problems that will benefit society as a whole - from studying the diversity of life on Earth, to understanding the Earth and its systems from satellite imagery of its poles, to developing response scenarios for natural disasters such as landslides and pandemics that impact the citizens and economies of the world.

SAGE3: Smart Amplified Group Environment for Harnessing the Data Revolution

Led by EVL Director Maxine Brown, a large team of research faculty and staff from UIC&rsquo;s College of
Engineering applied for and received a Major Research Instrumentation (MRI) grant from the National Science Foundation (NSF) for approximately $1-million to acquire a deep learning and visualization infrastructure. 
The design utilizes composable infrastructure architecture from <a href="https://www.liqid.com/">Liqid</a>. The computer&rsquo;s components (traditional processor, GPU, storage, and networking) are pooled so that different applications with different workflows can be run simultaneously, with each configuring the resources it requires almost instantaneously, at any time. 
&ldquo;EVL resources are traditionally very tailored to meet the requirements of specific applications. For example, on any given day, EVL may address everything from computational fluid dynamics, ecology and neuroimaging to natural language processing, urban engineering, robotics, visualization and visual analytics,&rdquo; said Lance Long, EVL Senior Research Programmer. &ldquo;Liqid&rsquo;s composable infrastructure architecture is interesting because it allows us the flexibility to redefine cyberinfrastructure programmatically based on the applications&rsquo; requirements. This enables the EVL team to create a data science platform that presents a full-stack hardware and software solution to the challenges faced by our researchers trying to deploy and scale reproducible experiments.&rdquo;

EVL&rsquo;s COMPaaS DLV system

Case Study: UIC Utilizes Liqid Composable Infrastructure for Uneven Applications in Scientific Research

In October 2018, the UIC Electronic Visualization Laboratory (EVL) received an equipment grant from the National Science Foundation (NSF #1828265) for a GPU-based system for College of Engineering research and education. Since that time, EVL staff has surveyed vendors, analyzed UIC faculty&rsquo;s application requirements, and then designed, purchased, received and installed its GPU-based system from Liqid. EVL refers to its new computing system as Composable Platform as a Service Instrument for Deep Learning & Visualization (COMPaaS DLV). 
<a href="https://www.liqid.com/">Liqid</a> is a start-up company specializing in systems integration (and has major industrial partners such as Dell, Intel, Nvidia and Mellanox), who currently sells its next-generation &ldquo;composable infrastructure&rdquo; computer systems primarily to data centers. Working with UIC faculty, it became apparent that this advanced architecture could address their diverse application requirements as their data moves through the computational pipeline (e.g., data gathering, filtering, computing, storing). 
EVL worked with Liqid to educate them about how its product could be tailored to meet the needs of the academic research market, helping them realize that science and engineering research is a potential market for their product. Working together, Liqid has configured a system to suit the needs of EVL&rsquo;s research focus. 
Liqid has published a <a href="https://www.evl.uic.edu/entry.php?id=2495">case study, &ldquo;UIC Utilizes Liqid Composable Infrastructure for Uneven Applications in Scientific Research&rdquo;</a> that explores EVL&rsquo;s use of COMPaaS DLV.

Liqid publishes a case study on how EVL is using their GPU-based system COMPaaS DLV

This paper describes results from an observational, exploratory study of visual data exploration in a large, multi-view, flexible canvas environment. Participants were provided with a set of data exploration sub-tasks associated with a local crime dataset and were instructed to pose questions to a remote mediator who would respond by generating and organizing visualizations on the large display. We observed that participants frequently posed requests to cast a net around one or several subsets of the data or a set of data attributes. They accomplished this directly and by utilizing existing views in unique ways, including by requesting to copy and pivot a group of views collectively and posing a set of parallel requests on target views expressed in one command. These observed actions depart from multi-view flexible canvas environments that typically provide interfaces in support of generating one view at a time or actions that operate on one view at a time. We describe how participants used these &ldquo;cast-a-net&rdquo; requests for tasks that spanned more than one view and describe design considerations for multi-view environments that would support the observed multi-view generation actions. 
Index Terms: Human-centered computing - Empirical studies in visualization 
This work was supported initially by NSF award IIS 1445751 and currently by NSF award CNS 1625941. 
<a href="https://doi.org/10.1111/cgf.13976">https://doi.org/10.1111/cgf.13976</a>

A participant generates 15 views of her data. In the final request, the participant references a set of three views and poses a request to copy and pivot these views collectively to four new subsets of the data. The final result is a grid of views.

Many At Once: Capturing Intentions to Create And Use Many Views At Once In Large Display Environments

Precision medicine seeks to tailor therapy to the individual patient, based on statistical correlates from patients who are similar to the one under consideration. These correlates can and should go beyond genetics, and in general, beyond tabular or array data that can be easily represented computationally and compared. For example, in many types of cancer, cancer treatment and toxicity depend in large measure on the spatial disease spread - e.g., metastasizes to regional lymph nodes in head and neck cancer. However, there is currently a lack of methodology for integrating spatial information when considering patient similarity. We present a novel modeling methodology for the comparison of cancer patients within a cohort, based on the spatial spread of the lymph nodes affected in each patient. The method uses a topological map, bigrams, and hierarchical clustering to group patients based on their similarity. We compare this approach against a nonspatial (categorical) similarity approach where patients are binned solely by their affected nodes. We present similarity results on a 582 head and neck cancer patient cohort, along with two visual abstractions for analysis of the results, and we present clinician feedback. Our novel methodology partitions a patient cohort into clinically meaningful groups more susceptible to treatment side-effects. Such spatially-aware similarity approaches can help maximize the effectiveness of each patient&rsquo;s treatment. 
Funding: NIH NCI-R01CA214825, NIH NCI-R01CA2251, and NIH NLM-R01LM012527

Topological map and graph representation.

A Spatial Neighborhood Methodology for Computing and Analyzing Lymph Node Carcinoma Similarity in Precision Medicine

PhD Graduate Student

Mir Masood Ali

EVL PhD candidate Juan Trelles Trabucco has been recognized among the UIC Computer Science Department&rsquo;s 2020 Outstanding Graduate TAs. This award typically recognizes graduate TAs for their work over an entire year, but given Juan&rsquo;s stellar performance during the spring semester, and especially during the pandemic, Juan is being duly recognized for his contributions to courses CS426 and CS529. 
&ldquo;Juan&rsquo;s TA performance in CS426 was beyond spectacular. The course had 60 students enrolled and made use of a variety of technologies. Yet with Juan&rsquo;s help, we had a flawless transition to online teaching, when the pandemic lockout was instituted.&rdquo; 
CS associate professor Elisabeta Marai was the instructor for both courses. CS426: Video Game Design and Implementation covers tools and techniques for the design and implementation of 3D interactive video games. CS529: Visual Data Science is an introduction to key design principles and techniques for interactively visualizing and analyzing data in data science. 
In addition, to the usual TA assignments of tutoring students, grading papers, etc. assisting in the transition of these courses to an online virtual platform posed additional challenges, and makes Juan truly deserving of this recognition. 
Please join us in congratulating Juan!

EVL PhD Candidate Juan Trelles Trabucco Receives CS 2020 Outstanding TA Award

Researchers at University of Hawai&rsquo;i at Mānoa, University of Illinois Chicago, and Virginia Tech were awarded a $5 million National Science Foundation grant to synergize two complementary technologies - large-scale data visualization and artificial intelligence - to create the Smart Amplified Group Environment (SAGE3) open-source software. 
SAGE, soon to be on its third iteration as SAGE3, is the most widely used big-data visualization and collaboration software in the world. 
SAGE and SAGE2 are software to enable data-rich collaboration on high-resolution display walls. SAGE2 moved SAGE into cloud computing and SAGE3 ushers in the inclusion of artificial intelligence. 
Principal investigator Jason Leigh is a computer and information science professor at University of Hawai&rsquo;i at Mānoa and the inventor of SAGE. SAGE is software to enable teams of collaborators to work together with data in the form of data visualizations. 
SAGE3&rsquo;s novel interactive artificial intelligence methods will learn from human analytic activity and use the knowledge gained to assist people in synthesizing relevant data connections that lead to new hypotheses and findings, explained Chris North, a professor of computer science in the College of Engineering at Virginia Tech and a co-principal investigator. 
Scientists analyzing their data in SAGE3 will collaborate with each other and with artificial intelligence through large interactive visualization spaces, such as multi-monitor workstations, tiled display walls, and virtual reality headsets. 
Leigh said the applications for the SAGE3 technology are endless. &ldquo;It will be tremendously useful to enable evidence-based response during natural disasters and events like the current COVID-19 pandemic,&rdquo; Leigh said. 
He created SAGE in collaboration with researchers from the University of Illinois Chicago 16 years ago. They&rsquo;ve since further developed SAGE and SAGE2 with 15 years of research funding by the National Science Foundation. 
&ldquo;This new funding provides us with an opportunity to take SAGE to the next level - providing smart end-user services to amplify users&rsquo; productivity, making commercial and open-source artificial intelligence solutions available that can autonomously and transparently analyze data while continually learning and improving through user interactions,&rdquo; said Maxine Brown, director of the Electronic Visualization Laboratory at the University of Illinois Chicago. Brown has been involved with SAGE since its inception. 
&ldquo;SAGE3 also supports resource orchestration services that integrates these smart services with applications, workflows, and visualizations and collaboration services, and makes it easy for users to connect to local and remote display walls and computational systems and to securely run reproducible work models,&rdquo; explained Luc Renambot, Electronic Visualization Laboratory associate research professor and co-architect with Leigh of the SAGE software since its inception. 
&ldquo;As a &rsquo;co-pilot&rsquo; to individuals or teams who are trying to examine large datasets, SAGE3 will provide a discovery pathway from data sources to human insights and knowledge, which is currently a gap in the e-Science pipeline that the National Science Foundation has called the &rsquo;&ldquo;missing middle,&rsquo;&rdquo; said Mahdi Belcaid, assistant professor at University of Hawai&rsquo;i at Mānoa and the artificial intelligence co-principal investigator on the project. 
According to the researchers, SAGE3 will make artificial intelligence technologies broadly accessible, not just a privilege for the technically savvy. North said the project will conduct user-centered design research studies to ensure usability of this novel combination of data visualization and artificial intelligence in SAGE3. 
Through seed funds provided by Virginia Tech&rsquo;s Institute for Creativity, Arts, and Technology and the Hawai&rsquo;i Data Science Institute, North had the opportunity to work side-by-side with Leigh in the Laboratory for Advanced Visualization & Applications at the University of Hawai&rsquo;i during the spring 2020 semester. This in-person exchange, and numerous video-teleconferences with the Electronic Visualization Laboratory, was essential to helping plan and initiate the five-year SAGE3 collaboration, noted North. SAGE3 will serve the Institute&rsquo;s missions of bringing together computer scientists, domain scientists, engineers, artists, and designers to tackle some of the world&rsquo;s most complex challenges. 
The user base goes well beyond scientists. Leigh said he sees SAGE3 helping teachers use and share artificial intelligence concepts to future generations of students. And for those who work in creative media, such as filmmakers and video game designers, Leigh envisions it being a highly visual tool to brainstorm new ideas and analyze all forms of digital media to see how they relate to each other, which ones were most popular and why. 
Written by Jenise L. Jacques, Virginia Tech 
For further information or queries, contact Maxine Brown <a href="mailto:maxine@uic.edu">maxine@uic.edu</a>. 
Also, see <a href="https://cs.uic.edu/news-stories/data-visualization-gets-artificial-intelligence-boost-with-5-million-nsf-grant/">UIC&rsquo;s Computer Science News Story</a>.

An example of SAGE2 at work, using cloud-based and web-browser technologies in order to enhance data intensive co-located and remote collaboration.

Announcing SAGE3: Data visualization gets artificial intelligence boost with $5 million NSF grant

Multi-device environments present new opportunities for collaborative visual data analysis and sense-making by utilizing each device&rsquo;s strengths and capabilities. However, one of the associated challenges with visual data analysis in multi-device environments is the sharing of visual components across devices. We present a framework developed on top of SAGE2 platform for cross-device collaborative visual data exploration. As part of our framework, we contribute to the concept of rapid development and assembling of visualizations that can span multiple devices of different modalities. It provides the users with an environment for visualization compositions that delegate the rendering to the target device, allowing them to augment their large display workspace with portable devices for further exploration territories. Facilitated by its intuitive visualization composition pipeline, users with no programming skills such as data analysts can enhance their analytical scope with no coding barriers. We describe the framework, its implementation with a use case, and the rationale behind its design. 
Keywords: Collaboration, Multi-device environment, Visual Data Analysis

PolyVis: Cross-Device Framework for Collaborative Visual Data Analysis

There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. However, multi-device ecologies pose several challenges as information and tasks are scattered among separate devices and displays. To understand the challenges associated with multi-device environments for information visualization, we performed an exploratory study designed to examine how users employ different tools to perform different kinds of activities in approaching a visual analysis task. Previous work examined three factors (users, tools, and tasks) independently. We study the synthesis of these factors. To do this, we adopted a hybrid analysis approach that focuses on three different aspects: users, tools, and tasks. We believe this analysis will help us identify associated challenges and better inform design goals in developing multi-device tools for visual data analysis. 
Keywords: Collaborative visual analytics, exploratory analysis, multi-device ecology

Illustration of the study setup.

Towards Understanding Collaborative Visual Data Analysis in Multi-Device Environments

Supporting exploratory visual data analysis is essential when multiple analysts collaborate using multiple devices. Yet, we still have no full understanding of how the iterative process of analysis unfolds in complex settings. In this paper, we present the results from an exploratory study where six groups of three participants performed a collaborative visual data analysis task in a complex multi-user multi-device environment. We found that the course of the analysis happens at two levels. Within each level, we observed a set of exploration patterns. We present a categorization of the analysis structure in such a complex environment and discuss the implications of device affordances on this categorization. We also discuss this categorization in relation to the current structural assumptions of exploratory visual analysis. 
Index Terms: Human-centered Computing, Empirical studies in visualization

Evaluating Strategies of Exploratory Visual Data Analysis in Multi Device Environments

EDITOR’S NOTE: The Professional University of Information and Management for Innovation (iU) announces a partnership with 7 international universities, including the University of Illinois at Chicago&rsquo;s (UIC) Electronic Visualization Laboratory (EVL). More specifically, Osamu Kamatani, Dean of iU&rsquo;s Faculty of Information and Management for Innovation, previously worked for Japan&rsquo;s NTT Network Innovation Laboratories, which had a 20-year research collaboration with UIC/EVL focused on advanced networking and collaboration. NTT was an early adopter of EVL&rsquo;s SAGE visualization and collaboration software and was involved in EVL-related advanced networking projects. iU will adopt SAGE2 and SAGE3, and will participate in the Global Research Platform (GRP), which is being organized by UIC/EVL, University of California San Diego (UCSD) and Northwestern University. The MOU formalizes and strengthens iU&rsquo;s collaborations with UIC/EVL. 
* * * * * * * * * * * * * * * * 
iU of Japan announces partnership with 7 international universities 
Friday, August 21, 2020 - The Professional University of Information and Management for Innovation (&ldquo;iU&rdquo;) commenced its first year of operation on April 1st 2020, and has now reached comprehensive partnership agreements with the following seven global prominent schools: The University of California, San Diego (UCSD); The University of Illinois at Chicago (UIC); The University of Sheffield; Raffles University Malaysia; The National University of Singapore; Nikola Tesla Graduate School in the U.K.; and the Federal University of Technology Akure (FUTA), Ondo State, Nigeria, with ongoing efforts for additional international collaborations. 
iU will continue to actively collaborate with major universities in Japan and abroad, with the aim to realize a &ldquo;Global University Network,"&rdquo; representing an intellectual hub for the IT and business industries centered around iU. 
Specifically, in addition to the exchange of students and faculty members and the implementation of joint research, the objectives are as follows: 
　1. A &ldquo;passport&rdquo; system that allows classes at partner universities and recognition of course credits 
　2. Joint development for research of new ventures and new products within a joint strategic special zone 
　3. Establishment of a special visa zone for international students 
　4. International collaborative research on the fields of eSports, super education, super sports, anime, otaku, etc. 
　5. Entrepreneurship support and Human Capital development 
The &ldquo;Global University Network&rdquo; advocated by iU, includes alliances with overseas universities. Through collaboration with each institution, iU will strive to develop and promote international joint lectures, joint experiments, implementation of various IT technologies, and joint research through exchanges of students and faculty from partner universities. iU will also work closely with prominent companies in Japan and overseas that support these activities. 
<a href="https://www.i-u.ac.jp/news/detail.php?id=199">Read the entire press release.</a> 
For Further Information, please contact: 
PR division 
Professional University of Information and Management for Innovation 
TEL: +81356551555 E-mail: <a href="mailto:info@i-u.ac.jp">info@i-u.ac.jp</a> 
For UIC/EVL collaboration information, contact Maxine Brown: <a href="mailto:maxine@uic.edu">maxine@uic.edu</a>

iU of Japan announces partnership with 7 international universities

Graph convolutional networks (GCNs) have been applied to 3D human pose estimation (HPE) from 2D body joint detections and
have shown encouraging performance. One limitation of the vanilla graph convolution is that it models the relationships between neighboring nodes via a shared weight matrix. This is suboptimal for articulated body modeling as the relations between different body joints are different. The objective of this paper is to have a comprehensive and systematic study of weight sharing in GCNs for 3D HPE. We first show there are two different ways to interpret a GCN depending on whether feature transformation occurs before or after feature aggregation. These two interpretations lead to five different weight sharing methods and three more variants can be derived by decoupling the self-connections with other edges. We conduct extensive ablation study on these weight sharing methods under controlled settings and obtain new conclusions that will benefit the community.

Qualitative results of our approach on Human 3.6M

A Comprehensive Study of Weight Sharing in Graph Networks for 3D Human Pose Estimation

Graph convolutional networks (GCNs) have been applied to 3D human pose estimation (HPE) from 2D body joint detections and have demonstrated promising performance. However, since the vanilla graph convolution is performed on the one-hop neighbors of each node, it is unable to capture the long-range dependencies between body joints. They can help reduce the uncertainty caused by occlusion or depth ambiguity. To resolve this issue, we propose a high-order GCN for 3D HPE. Its core building block, termed a high-order graph convolution, aggregates features of nodes at various distances. As a result, the network can model a wide range of interactions among body joints. Furthermore, we investigate different methods to fuse those multi-order features and compare how they affect the performance. Experimental results demonstrate the effectiveness of the proposed approach.

Comparison between the vanilla graph convolution and the high-order graph convolution performed on a skeleton graph.

High-order Graph Convolutional Networks for 3D Human Pose Estimation

Research Experience Undergraduate

Eseroghene Omene

PI: Andrew Johnson (UIC/EVL); co-PIs: Barbara DiEugenio (UIC/CS), Moira Zellner (Norheastern University) 
Collaborator: Jason Leigh (University of Hawaii Manoa) 
The Internet gives the public, including scientists, access to vast amounts of data. Visualization is a powerful tool to take that data and turn it into a form that is easier to understand, helping people make better decisions. However, creating good visualizations from data is not easy. The Articulate+ Project will work to generate new knowledge about how to empower people with information visualization, by using speech and gesture as inputs, in new ways. A goal is to enable users to benefit from imprecise statements about what they want, rather than needing to give specific commands telling the computer what to do. This project will make it easier for citizens to engage in conversations with the computer about economic, medical data, demographic, transportation data, climate, and sports data; to answers that are more meaningful, and so to democratize data literacy. 
This research will use speech, gesture, and log data as new input interaction modalities for visual analytics. It will develop new understandings of how the imprecise and vague nature of natural language queries and gestures, as contextualized by work on a specific visual analysis problem, can be modeled as an opportunity for an intelligent software system to learn more about the underlying intent of the users. This model of intent, in turn, will be used to provide contextualized visualizations, which are expected to help those users gain valuable insights from their data. The audio speech data will be used to computationally model overhearing, to help infer users&rsquo; current contextualized goals. The gesture data will be used to disambiguate expressions that refer to visual elements of a visualization (e.g., &ldquo;that pie chart about electricity consumption&rdquo;). The project will develop new understandings of how to combine the log of visualization state, the computational model of overhearing, and the gesture data in order to generate new visualizations in support of users&rsquo; work on tasks. Evaluations will be performed in both controlled laboratory and naturalistic study settings to determine whether effective semantic parsers can be derived for these specific visual analysis domains, and how this contextual interface affects users&rsquo; experience of visualization and discovery. The integrated annotated datasets from the studies will be made available to the research community satisfying a need for ecologically valid situated language interaction corpora.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Articulate+ - A Conversational Interface for Democratizing Visual Analysis

UIC Computer Science Professors Andy Johnson (PI) and Barbara Di Eugenio (co-PI) have been awarded a $333,000 NSF grant entitled &dquo;CHS: Small: Collaborative Research: Articulate+ - A Conversational Interface for Democratizing Visual Analysis&rdquo; project period October 2020 to September 2022. Dr. Moira Zellner, now of Northeastern University will also participate as co-PI on this grant. This project is part of a larger overall award of $500,000 that also includes Jason Leigh at University of Hawaii at Manoa. 
&ldquo;Articulate+&rdquo; continues EVL&rsquo; research in Natural Language Processing and Visualization also sponsored by NSF for the last 5+ years. See <a href="https://www.evl.uic.edu/entry.php?id=2522">Articulate+ - A Conversational Interface for Democratizing Visual Analysis</a> for more information on this project. 
NSF Award Number (FAIN): 2007257

NSF Grant &ldquo;CHS: Small: Collaborative Research: Articulate+&rdquo; Awarded

October 19, 2020 - - Fabio Miranda joins the EVL faculty as an assistant professor with the Department of Computer Science. Miranda interests include developing techniques that allow for the interactive visual analysis of large-scale data, combining methods from visualization, data management, machine learning, and computer graphics. Of particular concern is how visual data analytics can help address different problems cities face by integrating data on different resolutions and from different sources. 
Miranda&rsquo;s research in data science, human-computer interaction and visualization include: visual data analysis, data structures and indexing techniques for big data visualization, urban analytics, harnessing data sets of different types (images, land use information, 3D building geometry, etc.) to address concrete city problems, and machine learning. He has worked closely with domain experts from different fields, from urban planning to occupational therapy, and the outcome of these collaborations include not only research published in leading visualization, database, HCI, and AI venues, but also systems that were made available to experts in academia, industry, and government agencies. 
Miranda received his Ph.D. from New York University (2018).

Fabio Miranda Joins EVL Faculty

October 1 - - EVL Director of Research Andrew Johnson has been named Interim Director of EVL following Maxine Brown&rsquo;s retirement from UIC. Johnson now assumes the responsibility of heading EVL - overseeing EVL&rsquo;s overall mission in education, research, development, and technology transfer. 
For over 23 years, Johnson has been a valued EVL faculty member (Professor of Computer Science) and researcher who has focused upon interaction and collaboration using advanced visualization displays and the application of those displays to enhance discovery and learning. 
His areas of expertise include visualization, virtual and augmented reality, natural language processing, visual analytics, and high-performance networking. 
Over the years, Johnson has been instrumental in the development of EVL instrumentation and systems, including the Continuum smart environment, CAVE2&trade;, LambdaVision high-resolution tiled displays, LambdaTable tabletop touch system, GeoWall portable geoscience display environment, PARIS augmented reality and ImmersaDesk virtual reality systems. He has overseen the development of numerous visualization and virtual reality applications across diverse domain disciplines from science and engineering to the humanities and art. Johnson is a highly regarded mentor and educator to EVL&rsquo;s graduate students and undergraduates alike. 
Johnson retains his position as Director of Research while leading EVL&rsquo;s future initiatives and innovations as Interim Director.

Andrew Johnson Named Interim Director, EVL

For the Supercomputing (SC20) Conference this year, a research community partnership was formed to design an international networking testbed using existing facilities, with several international extensions. Demonstrations and experiments were conducted prior to the conference with results discussed during a SC20 virtual forum. &ldquo;The Forum For Experiments and Demonstrations Showcasing Innovations In Large-Scale, Data-Intensive-Science Networking&rdquo; was organized by Research Consortium for Data-Intensive Science Networking to communicate: a) the motivation behind these experiments and demonstrations, b) their design, c) technologies used, d) results and e) future research in these areas. 
EVL senior research programmer, Lance Long participated to discuss EVL&rsquo;s research in Composable Infrastructure, specifically MRI: Acquisition of a Composable Platform as a Service Instrument for Deep Learning & Visualization (COMPaaS DLV) funded under NSF Award #CNS-1828265. Items discussed included the benefits of composable infrastructure, implementation for deep learning in an academic research environment, operating systems, dynamic workflows, orchestration middleware, current use cases and future work. 
Other participants included: Ciena Research Lab, NASA Goddard Space Flight Center (GSFC), Naval Research Laboratory, iCAIR/Northwestern, StarLight, UCSD, FNAL, CalTech, SURFnet.

Composable Infrastructure for Data Science

Lance Long Presents EVL&rsquo;s Composable Infrastructure Research at SC20

Advances in data collection in radiation therapy have led to an abundance of opportunities for applying data mining and machine learning techniques to promote new data-driven insights. In light of these advances, supporting collaboration between machine learning experts and clinicians is important for facilitating better development and adoption of these models. Although many medical use-cases rely on spatial data, where understanding and visualizing the underlying structure of the data is important, little is known about the interpretability of spatial clustering results by clinical audiences. In this work, we reflect on the design of visualizations for explaining novel approaches to clustering complex anatomical data from head and neck cancer patients. These visualizations were developed, through participatory design, for clinical audiences during a multi-year collaboration with radiation oncologists and statisticians. We distill this collaboration into a set of lessons learned for creating visual and explainable spatial clustering for clinical users. 
Keywords: Data Clustering and Aggregation, Life Sciences, Collaboration, Mixed Initiative Human-Machine Analysis, Guidelines 
YouTube Video: <a href="https://youtu.be/UM8-_MN3a2Y">https://youtu.be/UM8-_MN3a2Y</a>

(A) Lymph nodes overlaid over a diagram of the neck. (B) Example graphs of diseased nodes for 2 individual patients (datapoint representation). (C) Example consensus graph for 1 cluster (cluster representation).

Explainable Spatial Clustering: Leveraging Spatial Data in Radiation Oncology

Microscopy images are pervasive in biomedical research publications, where images obtained through various microscopy modalities (light, fluorescence, scanning, transmission) are often used to describe and summarize experiments and contributions. Hence, there is growing interest in automatically identifying these microscopy images’ modality and utilizing this knowledge in automated search tools. However, identifying microscopy images poses challenges due to a lack of extensive collections of labeled images. We describe and evaluate two alternative approaches to microscopy image classification. In the first approach, we progressively fine-tuned layers of ResNet models. The second approach uses shallow variants of ResNet networks, where we leverage the outputs from previous convolutional blocks. We compare these results against a Support Vector Machine (SVM)-based baseline. Our results show that fine-tuning specific layers yields better results than fine-tuning the whole model. Furthermore, shallower variants produce competitive results when compared to the entire fine-tuned model. 
Index Terms: microscopy, image classification, deep learning

Four microscopy images from the ImageCLEF dataset showcasing different modalities: (a) Electron microscopy (DMEL), (b) Transmission microscopy (DMTR), (c) Light microscopy (DMLI), and (d) Fluorescence microscopy (DMFL).

Modality-Classification of Microscopy Images Using Shallow Variants of Deep Networks

EVL faculty and Computer Science associate professor G. Elisabeta Marai (MPI) along with H. Shatkay (MPI) at the University of Delaware have been awarded supplemental funding for COVID-19 related research to their NIH research project entitled &ldquo;Uncovering Clinical Evidence in COVID-19 Publications: An Integrated Search via Text &amp; Images&rdquo;. This supplement expands the work done under current R01 funding &ldquo;Incorporating Image-based Features into Biomedical Document Classification&rdquo;, to support discovery within the CORD-19 collection -- a large, continually updated collection of more than 60,000 scientific articles pertaining to COVID-19. This supplement will create a comprehensive platform for retrieving and displaying well-focused, need-targeted medical information within COVID-19 biomedical publications. Our approach is unique compared to other text-mining methods because it leverages the image content in the dataset. 
Funding: NIH R01 NLM Covid-19 supplement to Parent Grant: 1R01LM012527 (&ldquo;Incorporating Image-based Features into Biomedical Document Classification&rdquo;)

Data Science

Bibliography

pubs (11)

Related Topics

people (2)

events (11)

research (3)

Contact