It has often been observed, informally and in academic literature, that speech directed toward individuals who are hard of hearing is naturally modified by talkers in an attempt to accommodate the listeners. These modifications take numerous forms, from the easily expressed &ldquo;louder&rdquo; and/or &ldquo;slower&rdquo; to the highly technical &ldquo;vowel space expansion&rdquo; and many others. There are also speech modifications made in numerous other circumstances, not only for hard of hearing listeners, but for listeners in noisy backgrounds, second language listeners, and many others, known under the collective (is sometimes too-broad) umbrella as clear speech. There is, however, an asymmetry inherent to all these talker-listener scenarios, which is that talkers have only the broadest guidance at best (&ldquo;Louder!&rdquo; &ldquo;Slower!&rdquo;) from their listener partners, which they may simply ignore. 
With an interest in developing new and practical interventions for the elderly and hard of hearing, this dissertation takes as its research questions: First, the feasibility of the production of a signal processing system which allows real-time modification of audio signals. Second, the tolerance and effectiveness of partially emulating clear speech, by using signal processing techniques to modify (specifically, to slow) the rate of received speech signals. And finally, the effectiveness of placing the control of speech rate directly into the hands of listeners, a capability that has never been possible prior to electronic mediation. 
This dissertation is organized as follows: 
Chapter One is a survey of both the circumstances which prompt these spontaneous speech modifications, as well as of the details of the modifications which are made. 
Chapter Two presents an overview of the various methods of changing the rate of speech including the simple but ineffective technique of re-sampling, as well as several frequency domain techniques. These systems were intended for use on pre-existing, rather than live, signals. 
Chapter Three presents the details of several incarnations of this real-time system, including a single-talker system implemented on a laptop computer, presented in [1] and used directly (with two laptops for two talkers) in the study presented in [3]; a two-talker wireless system using Android smartphones and local network connectivity presented at [2]; and an unpublished system using a central laptop base station, to which Android smart phones could
connect even when hundreds of miles apart. This chapter contains details required for the implementation of both live signals, and live control of the playback rate of those signals. Finally, this system includes the design details of systems used in the three user studies of Chapters Five, Six, and Seven. 
Chapter Four presents a user study designed to examine the effects of using real-time temporal modification software in a conversational setting, previously published in [3]. The key element of the study design was the use of a scored Diapix test [7] which was designed to elicit spontaneous conversational speech between two study participants without the involvement of or prompting by researchers. The main results of this study are (i) that at modest amounts of stretching (40% additional playback time, with lengthy silences not stretched) the speech patterns of subjects are not changed to a statistically significant degree, and (ii) under those same circumstances, there is no statistical evidence of changes in performance on the Diapix test. 
Chapter Five presents the first of three user studies focusing on the little-studied subject of the interaction of user choice and user control over received speech rates in adverse conditions. This study was based off the preexisting QuickSIN test [8], designed to test and characterize a listener&rsquo;s ability to understand short sentences of speech played against a background of noise. In the first part of this study, users were asked to listen to several short sentences each played against multiple levels of background noise. For each level of background noise, the subjects were asked to use a computer interface to set the rate of the speech to whatever rate they believed was most helpful in understanding the foreground speech. (The background babble did not change rate.) These preferred expansion rates were recorded for later use and analysis. In the second part of the study, subjects were asked to listen to and immediately repeat back sentences at those same noise conditions, both with speech modified to their preferred expansion rates, and unmodified. The subjects’ repetition of those sentences was also recorded for analysis. 
Our results included statistically significant differences in received speech rate at opposite ends of the noise condition, with subjects requesting slower speech in the presence of increase noise; as well as an overall belief that slowing speech was beneficial to subjects’ performance. However, analysis of the subjects&rsquo; ability to repeat was in fact mildly degraded at moderate to high noise conditions and was not improved at any noise condition. This work was published at Interspeech 2018 [4]. 
Chapter Six presents the second of the three user control studies. In this study, non-native students with recent TOEFL scores were recruited as test subjects. After familiarization with a graphical interface, test subjects were asked to participate in six comprehension tasks, each of which was a lengthy (several minutes long) audio track of a TOEFL test to which they had not previously been exposed. During three of these tasks, the subjects were given
the ability to change the rate of playback as they saw fit, and asked to do so in whatever fashion they believed would best help them understand the audio passage. The other three comprehension tasks were presented unmodified, without a control interface. After each task, the subjects were immediately given a short multiple choice quiz about the contents of the previous audio passage. Subjects&rsquo; behavior (i.e., the rate changes specified) and quiz responses were collected and analyzed. 
Our analysis of the data shows that a considerable majority of the subjects used speech slowing in all three of their trials (roughly 2/3) and an overwhelming majority used the technique in at least one of their trials (80%), but did not reveal a statistically significant improvement or degradation in listening comprehension. There is also a weak correlation between lower TOEFL scores and more slowing. This work was published at Interspeech 2019 [5]. In the work described above, the algorithms employed stretch speech uniformly - if set to expand an audio track by 40%, all sounds, including all parts of speech all background noises, and even silences, are expanded by the same rate. For large expansion factors, the speech so produced sounds somewhat unnatural, because this is not how talkers spontaneously produce slow speech. With this in mind, Chapter Seven returns to the topic of speech in noise, but puts even more precise control in the hands of test subjects: A relatively simple neural network was developed to classify individual phonemes into six broad phonemic categories within an audio track. A study was also designed around this tool which asked subjects to listen to short sentences in noise, and then expand or contract the audio only for particular phonemes (e.g., first modify vowels, then modify fricatives, etc.) Following this, all modifications were applied simultaneously, and subjects were asked to listen to modified and unmodified/control sentences in three levels
of background noise and asked to transcribe the sentences into a computer interface. 
Our analysis found either no statistically significant improvement or degradation (at one noise level) or statistically significant degradation (at two noise levels) of intelligibility while using this technique. 
Finally, Chapter Eight presents overall conclusions, with additional discussion of the entire work and of directions for future studies.

Unwrapped Schematic of Cochlea

Audio Expansion Techniques For Real-Time Communication And Intelligibility Enhancement

Cluster schedulers are crucial in high-performance computing (HPC). They determine when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with speciﬁc HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backﬁlling. An efﬁcient training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a speciﬁc scheduling objective given by the system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. We implement DRAS into an HPC scheduling platform called CQGym. CQGym provides a common platform allowing users to ﬂexibly evaluate DRAS and other scheduling methods such as heuristic and optimization methods. The experiments using CQGym with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 50%. 
Index Terms - High-performance computing, cluster scheduling, deep reinforcement learning, job starvation, backﬁlling, resource reservation, OpenAI Gym 
<a href="https://ieeexplore.ieee.org/abstract/document/9894371">https://ieeexplore.ieee.org/abstract/document/9894371</a>

DRAS overview. DRAS agent (at the bottom) represents the scheduling agent; the environment (at the top) comprises the rest of the system, including job wait queue and HPC cluster.

DRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing

BACKGROUND: Acute lung injury and the acute respiratory distress syndrome (ARDS) are characterized by pulmonary inflammation, reduced endothelial barrier integrity and filling of the alveolar space with protein-rich edema fluid and infiltrating leukocytes. Animal models are critical to uncovering the pathologic mechanisms of this devastating syndrome. Intravital imaging of the intact lung via two photon intravital microscopy has proven a valuable method to investigate lung injury in small rodent models through characterization of inflammatory cells and vascular changes in real-time. However, respiratory motion complicates the analysis of these time series images and requires selective data extraction to stabilize the image. Consequently, analysis of individual alveoli may not provide a complete picture of the integrated mechanical, vascular and inflammatory processes occurring simultaneously in the intact lung. To address these challenges, we developed a web browser-based visualization application named Alveolus Analysis to process, analyze and graphically display intravital lung microscopy data. RESULTS: The designed tool takes raw temporal image data as input, performs image preprocessing and feature extraction offline, and visualizes the extracted information in a web browser-based interface. The interface allows users to explore multiple experiments in three panels corresponding to different levels of detail: summary statistics of alveolar/neutrophil behavior, characterization of alveolar dynamics including lung edema and inflammatory cells at specific time points, and cross-experiment analysis. We performed a case study on the utility of the visualization with two members or our research team and they found the tool useful because of its ability to preprocess data consistently and visualize information in a digestible and informative format. CONCLUSIONS: Application of our software tool, Alveolus Analysis, to intravital lung microscopy data has the potential to enhance the information gained from these experiments and provide new insights into the pathologic mechanisms of inflammatory lung injury. 
<a href="https://github.com/uic-evl/AlveolusAnalysis">Visits the code repository.</a>

Alveolus Analysis: A Web Browser-based Tool to Analyze Lung Intravital Microscopy

Inaccessible urban infrastructure creates and reinforces systemic exclusion of people with disabilities and impacts public health, physical activity, and quality of life for all. To improve the design of our cities and to enable more equitable policies and location-centric
technology designs, we need new data collection techniques, data standards, and accessibility-infused analytic tools and interactive maps focused on the quality, safety, and accessibility of pathways, transit ecosystems, and buildings. In this workshop, we bring together leading experts in human mobility, urban design, disability, and accessible computing to discuss pressing urban access challenges across the world and brainstorm solutions. We invite contributions from practitioners, transit officials, disability advocates,
and researchers.

The Future of Urban Accessibility for People with Disabilities: Data Collection, Analytics, Policy, and Tools

Background. Personalized radiotherapy can improve treatment outcomes of head and neck cancer (HNC) patients, where currently a &rsquo;one-dose-fits-all&rsquo; approach is the standard. The aim was to establish individualized outcome prediction based on multi-institutional international &ldquo;big data&rdquo; to facilitate risk-based stratification of HNC patients. 
Methods. The data of 4611 HNC radiotherapy patients from three academic cancer centers was split into 4 cohorts: a training (n=2241), independent test (n=786), and external validation cohorts 1 (n=1087) and 2 (n=497). Tumor- and patient-related clinical variables were considered in a machine learning pipeline to predict overall survival (primary endpoint) and local and regional tumor control (secondary endpoints); serially, imaging features were considered for optional model improvement. Finally, patients were stratified into high, intermediate, and low risk groups. 
Results. Performance score, AJCC8th stage, pack-years, and Age were identified as predictors for overall survival, demonstrating good performance in both the training cohort (c-index=0.72 [95% CI, 0.66-0.77]) and in all three validation cohorts (c-indices: 0.76 [0.69-0.83], 0.73 [0.68- 0.77], and 0.75 [0.68-0.80]). Excellent stratification of HNC patients into high, intermediate, and low mortality risk was achieved; with 5-year overall survival rates of 17-46% for the high-risk group compared to 92-98% for the low-risk group. The addition of morphological image feature further improved the performance (c-index=0.73 [0.64-0.81]). These models are integrated in a clinic-ready interactive web-interface: <a href="https://uic-evl.github.io/hnc-predictor/">https://uic-evl.github.io/hnc-predictor/</a>. 
Conclusions. Robust model-based prediction was able to stratify HNC patients in distinct high, intermediate and low mortality risk groups. This can effectively be capitalized for personalized radiotherapy, e.g., for tumor radiation dose escalation/de-escalation. 
See <a href+"https://www.sciencedirect.com/science/article/pii/S095980492200805X">Science Direct</a> for more information.

Head and Neck Cancer Predictive Risk Estimator to Determine Control and Therapeutic Outcomes of Radiotherapy (HNCPREDICTOR)

Sidewalks play a pivotal role in urban mobility of everyday life. Ideally, sidewalks provide a safe walkway for pedestrians, link public transportation facilities, and equip people with routing and navigation services. However, there is a scarcity of open sidewalk data, which not only impacts the accessibility and walkability of cities but also limits policymakers in generating insightful measures to improve the current state of pedestrian facilities. As one of the most famous crowdsourced data repositories, OpenStreetMap (OSM) could aid the lack of open sidewalk data to a large extent. However, completeness and quality of OSM data have long been a major issue. In this paper, we offer a preliminary study on the availability and trustworthiness of OSM sidewalk data. First, we compare OSM
sidewalk data coverage in over 50 major cities in the United States. Then, we select three major cities (Seattle, Chicago, and New York City) to further analyze the completeness of sidewalk data and its features, and to compute a trustworthiness index leveraging historical OSM sidewalk data.

Coverage of sidewalks and roads with sidewalk info in Chicago, Seattle, and New York City (from left to right). Seattle OSM has comparably higher percentage of sidewalk geometries and roads with sidewalk information mapped than the other two cities.

Crowdsourcing and Sidewalk Data: A Preliminary Study on the Trustworthiness of OpenStreetMap Data in the US

Contrails are condensation trails generated from emitted particles by aircraft engines, which perturb Earth&rsquo;s radiation budget. Simulation modeling is used to interpret the formation and development of contrails. These simulations are computationally intensive and rely on high-performance computing solutions, and the contrail structures are not well defined. We propose a visual computing system to assist in defining contrails and their characteristics, as well as in the analysis of parameters for computer-generated aircraft engine simulations. The back-end of our system leverages a contrail-formation criterion and clustering methods to detect contrails&rsquo; shape and evolution and identify similar simulation runs. The front-end system helps analyze contrails and their parameters across multiple simulation runs. The evaluation with domain experts shows this approach successfully aids in contrail data investigation. 
Index Terms: Scalar Field Data, Physical & Environmental Sciences, Mathematics, Feature Detection, Tracking & Transformation 
 <iframe width="420" height="315"src="https://www.youtube.com/embed/mFllS2wQW8I">
</iframe>

Run analysis.

Visual Analysis and Detection of Contrails in Aircraft Engine Simulations

Recent technological innovations have led to an increase in the availability of 3D urban data, such as shadow, noise, solar
potential, and earthquake simulations. These spatiotemporal datasets create opportunities for new visualizations to engage experts
from different domains to study the dynamic behavior of urban spaces in this under-explored dimension. However, designing 3D
spatiotemporal urban visualizations is challenging, as it requires visual strategies to support analysis of time-varying data referent
to the city geometry. Although different visual strategies have been used in 3D urban visual analytics, the question of how effective
these visual designs are at supporting spatiotemporal analysis on building surfaces remains open. To investigate this, in this paper we first contribute a series of analytical tasks elicited after interviews with practitioners from three urban domains. We also contribute a quantitative user study comparing the effectiveness of four representative visual designs used to visualize 3D spatiotemporal urban data: spatial juxtaposition, temporal juxtaposition, linked view, and embedded view. Participants performed a series of tasks that required them to identify extreme values on building surfaces over time. Tasks varied in granularity for both space and time dimensions. Our results demonstrate that participants were more accurate using plot-based visualizations (linked view, embedded view) but faster using color-coded visualizations (spatial juxtaposition, temporal juxtaposition). Our results also show that, with increasing task complexity, plot-based visualizations perform better in preserving efficiency (time, accuracy) compared to color-coded visualizations. Based on our findings, we present a set of takeaways with design recommendations for 3D spatiotemporal urban visualizations for researchers and practitioners. Lastly, we report on a series of interviews with four practitioners, and their feedback and suggestions for further work on the visualizations to support 3D spatiotemporal urban data analysis. 
Index Terms - Visualization, urban analytics, urban data, spatiotemporal data, empirical evaluation.

Urban surface data from different domains: (a)flooding simulation analysis-environment, (b)view impact analysis-urban planning, (c)building design performance analysis-urban design, (d)solar potential & sunlight analysis-urban design/architecture

A Comparison of Spatiotemporal Visualizations for 3D Urban Analytics

Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly discarding some data elements or by directing instruments to relevant areas of experimental space. Thus, methods are required for configuring and running distributed computing pipelines-what we call flows-that link instruments, computers (e.g., for analysis, simulation, artificial intelligence [AI] model
training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. We review common patterns associated with such flows and describe methods for instantiating these patterns. 
We present experiences with the application of these methods to the processing of data from five different scientific instruments, each of which engages powerful computers for data inversion, model training, or other purposes. We also discuss implications of such methods for operators and users of scientific facilities. 
<a href="https://www.cell.com/patterns/fulltext/S2666-3899(22)00231-8">https://www.cell.com/patterns/fulltext/S2666-3899(22)00231-8</a>

Linking scientific instruments and computation: Patterns, technologies, and experiences

Understanding functional correlations between the activities of neuron populations are vital for the analysis of neuronal networks. Analyzing large-scale neuroimaging data obtained from hundreds of neurons simultaneously poses significant visualization challenges. We developed V-NeuroStack, a novel network visualization tool to visualize data obtained using calcium imaging of spontaneous activity of neurons in a mouse brain slice as well as in vivo using two-photon imaging. V-NeuroStack creates 3D time stacks by stacking 2D time frames for a time-series dataset. It provides a web interface to explore and analyze data using both 3D and 2D visualization techniques. Previous attempts to analyze such data have been limited by the tools available to visualize large numbers of correlated activity traces. V-NeuroStack&rsquo;s 3D view is used to explore patterns in dynamic large-scale correlations between neurons over time. The 2D view is used to examine any timestep of interest in greater detail. Furthermore, a dual-line graph provides the ability to explore the raw and first-derivative values of activity from an individual or a functional cluster of neurons. V-NeuroStack can scale to datasets with at least a few thousand temporal snapshots. It can potentially support future advancements in in vitro and in vivo data-capturing techniques to bring forth novel hypotheses by allowing unambiguous visualization of massive patterns in neuronal activity data. See <a href="https://onlinelibrary.wiley.com/doi/10.1002/jnr.25139">https://onlinelibrary.wiley.com/doi/10.1002/jnr.25139</a> 
Keywords: 3D visualization, calcium imaging, dynamic network, neural network, spatio-temporal data

V-NeuroStack application with a loaded sample dataset from brain slice data.

V-NeuroStack: Open-source 3D time stack software for identifying patterns in neuronal data

Hummingbird: is a modern, innovative performance merging live theater and interactive virtual reality by bringing a group of active participants into a shared space for a live performance. The performance premiered as part of Chicago&rsquo;s Tony Award-winning Goodman Theatre New Stages Festival showcasing innovative and ground-breaking theater works in December 2021. This project bridges art, science and live theater through a collaborative research effort between computer science and design faculty and students at the University of Illinois Chicago (UIC) Electronic Visualization Laboratory (EVL) and Chicago theater directors, actors, videographers and producers. Hummingbird&rsquo;s story celebrates courage and coming of age through the eyes of a gutsy teen who must outsmart her mother&rsquo's narcissistic boss and survive dangerous new technology in a live, immersive adventure. Hummingbird extends traditional live theater and makes virtual reality art accessible to a broader audience, demonstrating how virtual reality can transform theatrical storytelling. 
CCS CONCEPTS 
Applied computing - Arts and humanities; Performing arts; Human-centered computing - Collaborative and social computing; Collaborative and social computing theory, concepts and paradigms; Computer-supported cooperative work. 
KEYWORDS 
Virtual reality, Storytelling, Theater, Multi-user, Performance

&ldquo;Hummingbird&rdquo; being performed at EVL during pilot performances summer of 2021

Hummingbird: A Collaborative Live Theater and Virtual Reality Adventure

Probability distributions are omnipresent in data analysis. They are often used to model the natural uncertainty present in real phenomena or to describe the properties of a data set. Designing efficient visual metaphors to convey probability distributions is, however, a difficult problem. This fact is especially true for geographical data, where conveying the spatial context constrains the design space. While many different alternatives have been proposed to solve this problem, they focus on representing data variability. However, they are not designed to support spatial analytical tasks involving probability quantification. The present work aims to adapt recent non-spatial approaches to the geographical context, in order to support probability quantification tasks. We also present a user study that compares the efficiency of these approaches in terms of both accuracy and usability. 
Keywords: spatial visualization; uncertainty visualization

A choropleth map showing the number of taxi trips in each neighborhood of Midtown Manhattan (NYC) with each neighborhood having an associated monthly distribution of number of taxi trips, as highlighted for the regions of Midtown Center and Kips Bay.

A Comparative Study of Methods for the Visualization of Probability Distributions of Geographical Data

In today&rsquo;s Big Data era, data scientists require modern workflows to quickly analyze large-scale datasets using complex codes to maintain the rate of scientific progress. These scientists often rely on available campus resources or off-the-shelf
computational systems for their applications. Unified infrastructure or over-provisioned servers can quickly become bottlenecks for specific tasks, wasting time and resources. Composable infrastructure helps solve these problems by providing users with new ways to increase resource utilization. Composable infrastructure disaggregates a computer&rsquo;s components - CPU, GPU (accelerators), storage and networking - into fluid pools of resources, but typically relies upon infrastructure engineers to architect individual machines. Infrastructure is either managed with specialized command-line utilities, user interfaces or specification files. These management models are cumbersome and difficult to incorporate into data-science workflows. We developed a high-level software API, Composastructure, which, when integrated into modern workflows, can be used by infrastructure engineers as well as data scientists to reorganize composable resources on demand. Composastructure enables infrastructures to be programmable, secure, persistent and reproducible. Our API composes machines, frees resources, supports multi-rack operations, and includes a Python module for Jupyter Notebooks. 
Keywords: distributed systems, testbed implementation and deployment, composable infrastructure, deep learning, visualization, infrastructure as code

Moving from Composable to Programmable

Composable infrastructure holds the promise of accelerating the pace of academic research and discovery by enabling researchers to tailor the resources of a machine (e.g., GPUs, storage, NICs), on-demand, to address application needs. We were first introduced to composable infrastructure in 2018, and at the same time, there was growing demand among our College of Engineering faculty for GPU systems for data science, artificial intelligence / machine learning / deep learning, and visualization. Many purchased their own individual desktop or deskside systems, a few pursued more costly cloud and HPC solutions, and others looked to the College or campus computer center for GPU resources which, at the time, were scarce. After surveying the diverse needs of our faculty and studying product offerings by a few nascent startups in the composable infrastructure sector, we applied for and received a grant from the National Science Foundation in November 2019 to purchase a mid-scale system, configured to our specifications, for use by faculty and students for research and research training. 
This paper describes our composable infrastructure solution and implementation for our academic community. Given how modern workflows are progressively moving to containers and cloud frameworks (using Kubernetes) and to programming notebooks (primarily Jupyter), both for ease of use and for ensuring reproducible experiments, we initially adapted these tools for our system. We have since made it simpler to use our system, and now provide our users with a public facing JupyterHub server. We also added an expansion chassis to our system to enable composable co-location, which is a shared central architecture in which our researchers can insert and integrate specialized resources (GPUs, accelerators, networking cards, etc.) needed for their research. 
In February 2020, installation of our system was finalized and made operational and we began providing access to faculty in the College of Engineering. Now, two years later, it is used by over 40 faculty and students plus some external collaborators for research and research training. Their use cases and experiences are briefly described in this paper. Composable infrastructure has proven to be a useful computational system for workload variability, uneven applications, and modern workflows in academic environments. 
Keywords: composable infrastructure, deep learning, visualization, resource management, workload management, user workflow, composable co-location, infrastructure as code

Composable Infrastructures for an Academic Research Environment: Lessons Learned

The Electronic Visualization Laboratory at the University of Illinois Chicago acquired a 24 compute node, 64 GPU composable infrastructure compute cluster, COMPaaS DLV - Composable Platform as a Service: Instrument for Deep Learning & Visualization in 2019.  Since then 40+ users have designed and run computations on COMPaaS.  To track cluster utilization, evaluate hardware
performance, and simply analyze application resource utilization of this experimental composable infrastructure system we set out to
collect metrics on the hardware and software infrastructure.  Users deploy their applications on the cluster using Kubernetes, a container orchestration platform that enables users to deploy applications on the cluster without the need to have complete knowledge of the underlying hardware and with minimal assistance from system administrators.  Following this, we utilized tooling already adapted for Kubernetes-based clusters to collect hardware and software metrics.  The metrics provided by this tooling is useful for system administrators in evaluating cluster performance and utilization, as well as useful for users in determining their application&rsquo;s performance and discovering bottlenecks.

GPU Power Consumption

Monitoring COMPaaS

In today&rsquo;s Big Data era, data scientists require modern workflows to quickly analyze large-scale datasets using complex codes to maintain the rate of scientific progress. These scientists often rely on available campus resources or off-the-shelf computational systems for their applications. Unified infrastructure or over-provisioned servers can quickly become
bottlenecks for specific tasks, wasting time and resources. Composable infrastructure helps solve these problems by providing users with new ways to increase resource utilization. Composable infrastructure disaggregates a computer&rsquo;s components – CPU, GPU (accelerators), storage and networking - into fluid pools of resources, but typically relies upon infrastructure engineers to
architect individual machines. Infrastructure is either managed with specialized command-line utilities, user interfaces or specification files. These management models are cumbersome and difficult to incorporate into data-science workflows. We
developed a high-level software API, Composastructure, which, when integrated into modern workflows, can be used by infrastructure engineers as well as data scientists to reorganize composable resources on demand. Composastructure enables infrastructures to be programmable, secure, persistent and reproducible. Our API composes machines, frees resources, supports multi-rack operations, and includes a Python module for Jupyter Notebooks. 
Keywords - distributed systems, testbed implementation and deployment, composable infrastructure, deep learning, visualization, infrastructure as code

Composastructure System Architecture

Composable infrastructure holds the promise of accelerating the pace of academic research and discovery by enabling researchers to tailor the resources of a machine (e.g., GPUs, storage, NICs), on-demand, to address application needs. We were first introduced to composable infrastructure in 2018, and at the same time, there was growing demand among our College of Engineering faculty for GPU systems for data science, artificial intelligence / machine learning / deep learning, and visualization. Many purchased their own individual desktop or deskside systems, a few pursued more costly cloud and HPC solutions, and others looked to the College or campus computer center for GPU resources which, at the time, were scarce. After surveying the diverse needs of our faculty and studying product offerings by a few nascent startups in the composable infrastructure sector, we applied for and received a grant from the National Science Foundation in November 2019 to purchase a mid-scale system, configured to our specifications, for use by faculty and students for research and research training. 
This paper describes our composable infrastructure solution and implementation for our academic community. Given how modern workflows are progressively moving to containers and cloud frameworks (using Kubernetes) and to programming notebooks (primarily Jupyter), both for ease of use and for ensuring reproducible experiments, we initially adapted these tools for our system. We have since made it simpler to use our system, and now provide our users with a public facing JupyterHub server. We also added an expansion chassis to our system to enable composable co-location, which is a shared central architecture in which our researchers can insert and integrate specialized resources (GPUs, accelerators, networking cards, etc.) needed for their research. 
In February 2020, installation of our system was finalized and made operational and we began providing access to faculty in the College of Engineering. Now, two years later, it is used by over 40 faculty and students plus some external collaborators for research and research training. Their use cases and experiences are briefly described in this paper. Composable infrastructure has proven to be a useful computational system for workload variability, uneven applications, and modern workflows in academic environments. 
Keywords - composable infrastructure, deep learning, visualization, resource management, workload management, user workflow, composable co-location, infrastructure as code

COMPaaS Hardware Racks

The annual incidence of head and neck cancers (HNC) worldwide is more than 550,000 cases, with around 300,000 deaths each year. However, the incidence rates and disease-characteristics of HNC differ between treatment centers and different populations, due to undetermined reasons, which may or not include socioeconomic factors. The multi-faceted and multi-variate nature of the data in the context of the emerging field of health disparities research makes automated analysis impractical. Hence, we present a visual analysis approach to explore the health disparities in the data of HNC patients from two different cohorts at two cancer care centers. Our approach integrates data from multiple sources, including census data and city data, with custom visual encodings and with a nearest
neighbor approach. Our design, created in collaboration with oncology experts, makes it possible to analyze the patients’ demographic, disease characteristics, treatments and outcomes, and to make significant comparisons of these two cohorts and of individual patients. We evaluate this approach through two case studies performed with domain experts. The results demonstrate that this visual analysis approach successfully accomplishes the goal of comparing two cohorts in terms of different significant factors, and can provide insights into the main source of health disparities between the two centers. 
Index Terms: Biomedical and medical visualization - Design studies - Hypothesis forming

A tale of two head and neck cancer centers, University of Illinois and MD Anderson Cancer Center.

A Tale of Two Centers: Visual Exploration of Health Disparities in Cancer Care

Pose guided person image generation means to generate a photo-realistic person image conditioned on an input person image and a desired pose. This task requires spatial manipulation of the source image according to the target pose. However, convolutional neural networks (CNNs) are inherently limited to geometric transformations due to the fixed geometric structures in their building modules, i.e., convolution, pooling and unpooling, which cannot handle large motion and occlusions caused by large pose transform. This paper introduces a novel two-stream context-aware appearance transfer network to address these challenges. It is a three-stage architecture consisting of a source stream and a target stream. Each stage features an appearance transfer module, a multi-scale context module and two-stream feature fusion modules. The appearance transfer module handles large motion by finding the dense correspondence between the two-stream feature maps and then transferring the appearance information from the source stream to the target stream. The multi-scale context module handles occlusion via contextual modeling, which is achieved by atrous convolutions of different sampling rates. Both quantitative and qualitative results indicate the proposed network can effectively handle challenging cases of large pose transform while retaining the appearance details. Compared with state-of-the-art approaches, it achieves comparable or superior performance using much fewer parameters while being significantly faster. 
Keywords: Person image generation, Appearance transfer, Multi-scale context, Efficient image generation 
Relevant Funding: The COMPaaS DLV project (NSF award CNS-1828265) 
<a href="https://doi.org/10.1016/j.patcog.2021.108451">https://doi.org/10.1016/j.patcog.2021.108451</a>

Exploiting appearance transfer and multi-scale context for efficient person image generation

While designing sustainable and resilient urban built environment is increasingly promoted around the world, significant data gaps have made research on pressing sustainability issues challenging to carry out. Pavements are known to have strong economic and environmental impacts; however, most cities lack a spatial catalog of their surfaces due to the cost-prohibitive and time-consuming nature of data collection. Recent advancements in computer vision, together with the availability of street-level images, provide new opportunities for cities to extract large-scale built environment data with lower implementation costs and higher accuracy. In this paper, we propose CitySurfaces, an active learning-based framework that leverages computer vision techniques for classifying sidewalk materials using widely available street-level images. We trained the framework on images from New York City and Boston and the evaluation results show a 90.5% mIoU score. Furthermore, we evaluated the framework using images from six different cities, demonstrating that it can be applied to regions with distinct urban fabrics, even outside the domain of the training data. CitySurfaces can provide researchers and city agencies with a low-cost, accurate, and extensible method to collect sidewalk material data which plays a critical role in addressing major sustainability issues, including climate change and surface water management. 
Keywords: Sustainable built environment; Surface materials; Urban heat island; Semantic segmentation; Sidewalk assessment; Urban analytics; Computer vision 
ISSN 2210-6707 
<a href="https://www.sciencedirect.com/science/article/pii/S2210670721008933">https://www.sciencedirect.com/science/article/pii/S2210670721008933</a>

Predictions of the model on the held-out test set. Fine details and boundaries of objects like poles, plants, wooden sticks, and fire hydrants are very precisely predicted.

CitySurfaces: City-scale semantic segmentation of sidewalk materials

In the last 20 years, extreme weather-related events like floods, landslides, droughts, and wildfires have caused the death of 1.23 million people and a loss of 2.97 trillion dollars. Studies show that low and lower-middle-income countries are the most impacted ones given the lack of investment in disaster risk management. To reduce the impact of these events, weather researchers have been developing numerical weather models that inform public agencies about the impending extreme events in advance. Despite being powerful tools, these models can suffer from several sources of uncertainty, ranging from the approximation of micro-scale physical processes to the location-dependent calibration of parameters, which is especially critical in developing countries. To minimize uncertainty effects, researchers generate several different weather scenarios to compose an ensemble of simulations that typically are inspected using manual, laborious, and error-prone approaches. In this paper, we propose an interactive visual analytics system, called X-Weather, developed in close collaboration with weather researchers from Brazil. Our system contributes a set of statistics and probability-based visualizations that allows the assessment of extreme weather events by effortlessly navigating through and comparing ensemble members. We demonstrate the effectiveness of the system through two case studies analyzing tragic events that happened in the mountain region of Rio de Janeiro in Brazil. 
Keywords: Visual analytics, Weather visualization, Ensemble visualization 
ISSN 0097-8493 
<a href="https://www.sciencedirect.com/science/article/pii/S0097849322000073">https://www.sciencedirect.com/science/article/pii/S0097849322000073</a>

The X-Weather interface.

Visualizing simulation ensembles of extreme weather events

Although cancer patients survive years after oncologic therapy, they are plagued with long-lasting or permanent residual symptoms, whose severity, rate of development, and resolution after treatment vary largely between survivors. The analysis and interpretation of symptoms is complicated by their partial co-occurrence, variability across populations and across time, and, in the case of cancers that use radiotherapy, by further symptom dependency on the tumor location and prescribed treatment. We describe THALIS, an environment for visual analysis and knowledge discovery from cancer therapy symptom data, developed in close collaboration with oncology experts. Our approach leverages unsupervised machine learning methodology over cohorts of patients, and, in conjunction with custom visual encodings and interactions, provides context for new patients based on patients with similar diagnostic features and symptom evolution. We evaluate this approach on data collected from a cohort of head and neck cancer patients. Feedback from our clinician collaborators indicates that THALIS supports knowledge discovery beyond the limits of machines or humans alone, and that it serves as a valuable tool in both the clinic and symptom research. 
keywords: Cancer, Visualization, Medical Treatment, Data Visualization, Encoding, Principal Component Analysis, Neck, Temporal Data, Application Motivated Visualization, Life Sciences, Mixed Initiative Human Machine Analysis