MPI_Alltoallv generalizes the uniform all-to-all communication (MPI_Alltoall) by enabling the exchange of data-blocks of varied sizes among processes. This function plays a crucial role in facilitating many computational tasks, such as FFT calculations and graph mining operations. Popular MPI libraries, such as MPICH and OpenMPI, implement MPI_Alltoall using a combination of linear and logarithmic algorithms. However, MPI_Alltoallv typically relies only on variations of linear algorithms, missing the benefits of logarithmic approaches. Furthermore, current algorithms also overlook the intricacies of modern HPC system architectures, such as the significant performance gap between intra-node (local) and inter-node (global) communication. To address these problems, this paper presents two novel algorithms: Parameterized Logarithmic non-uniform All-to-all (ParLogNa) and Parameterized Linear nonuniform All-to-all (ParLinNa). ParLogNa is a tunable logarithmic time algorithm for non-uniform all-to-all, and ParLinNa is a hierarchical and tunable near-linear-time algorithm for non-uniform all-to-all. These algorithms efficiently address the trade-off between bandwidth maximization and latency minimization that existing implementations struggle to optimize. We show a performance improvement over the state-of-the-art implementations by factors of 42x and 138x on Polaris and Fugaku, respectively. 
<a href="https://doi.org/10.1145/3731545.3731590">https://doi.org/10.1145/3731545.3731590</a>

An example of ParLinNa when P = 15, N = 3, r = 2 and Q = 5.

Parameterized Algorithms for Non-uniform All-to-all

We present the development, architecture, and features of a new multi-device mHealth software platform to support near real-time remote monitoring of metabolic health and timely intervention in the treatment and survivorship of cancer patients. Our platform, mEnergy, leverages a human-centered design process, and integrates in a unified, web-based framework consumer-grade hardware - Fitbit wearable sensor devices, smartphones, and Withings smart scales. mEnergy can
aid oncologists in identifying early indicators of muscle-wasting (sarcopenia) due to sleep disturbance, insufficient weight recovery, or reduced/limited activity. The platform aims for a smooth transition into clinical practice and increased adherence to evidence-based recommendations, in particular in underserved geographical areas. This toxicity-surveillance approach based on mHealth technologies can improve treatment outcomes, quality of life, and survivorship.

Visual analysis of multi-device health data.

mEnergy: Development of a Novel mHealth Multi-device Platform for Near Real-time Remote Assessment of Energy Disorders

Efficient job scheduling is crucial in high-performance computing (HPC), balancing user demands for quick job turnaround with facility goals for high resource utilization. Traditional scheduling requires users to specify a system at job submission, which can lead to inefficiencies. A unified scheduling approach, viewing the resources within a computing facility as an integrated pool, promises improved resource use and reduced job wait times. This paper presents CQSim+, an open-source, discrete event-driven simulator tailored for symbiotic multi-resource scheduling. CQSim+ supports dynamic simulation by continuously integrating real-time data from job schedulers, enabling adaptive scheduling based on the system&rsquo;s current state. Through extensive experimentation, we demonstrate CQSim+&rsquo;s ability to enhance resource utilization and decrease job wait times in both homogeneous and heterogeneous HPC environments. 
Additionally, we present a case study that coordinates job scheduling between two production systems, illustrating how CQSim+ can effectively optimize job scheduling across distinct systems. 
CCS Concepts: Computing methodologies - Modeling methodologies; Discrete-event simulation; Real-time simulation; Simulation tools; Social and professional topics - System management 
Keywords: Multi-resource scheduling, Simulation tool, Resource management systems, High-performance computing 
<a href="https://doi.org/10.1145/3726301.3728404">https://doi.org/10.1145/3726301.3728404</a>

A time diagram of CQSim+ performing meta-scheduling for multiple systems.

CQSim+: Symbiotic Simulation for Multi-Resource Scheduling in High-Performance Computing

Motivation: 
- The Large Hadron Collider (LHC) has created hundreds of petabytes of data. The High-Luminosity LHC (HL-LHC) will produce orders of magnitude more.
The HL-LHC is a major upgrade to the LHC, aiming to be operational by the end of 2030. It will produce more data in one year than the LHC produced during its first 10 years of operation. 
- Significant R&D efforts are needed to handle the incoming tide of data. Without using new technologies, the data volume of the HL-LHC will rapidly outgrow CERN&rsquo;s compute and storage capacities. This means everything is on the table, even historically disfavored methods like lossy compression.

﻿﻿Evaluating SZ3 Compressor Performance on High Energy Physics Data (poster)

In recent years, numerous hardware accelerators have been developed to meet the rising demand for machine learning (ML) workloads, and Large Language Models inference in particular. GPUs are currently the standard for ML training and inference. However, they require substantial data movements that hurt performance and increase power consumption, making systems extremely energy-intensive. In response, many companies, including Intel, AMD, and Google, as well as numerous startups such as Groq, SambaNova, Cerebras, and Graphcore, have introduced specialized accelerators for ML workloads that leverage a dataflow design, which aims to reduce data movement and thus improve both performance and power consumption. This article presents a comparative analysis of the performance and energy efficiency of various AI accelerators and GPUs for large language model (LLM) inference, using popular open-source models evaluated on both synthetic and real-world datasets.

Energy Efficiency of LLM Inference Across Various AI Accelerators (poster)

While large language models (LLMs) are increasingly used for generating parallel scientific code, most current efforts emphasize functional correctness, often overlooking performance and energy considerations. In this work, we propose LASSI-EE, an automated LLM-based refactoring framework that generates energy-efficient parallel code on a target parallel system for a given parallel code as input. 
Through a multi-stage, iterative pipeline process, LASSI-EE achieved an average energy reduction of 47% across 85% of the 20 HeCBench benchmarks tested on NVIDIA A100 GPUs. Our findings demonstrate the broader potential of LLMs, not only for generating correct code but also for enabling energy-aware programming. We also address key insights and limitations within the framework, offering valuable guidance for future improvements.

LASSI-EE: Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes (poster)

HEP workflows often consist of complex DAGs with streaming job arrivals that challenge traditional static scheduling approaches. We introduce a dynamic pilot job scheduling model that allocates HPC resources on demand when queued DAG jobs reach a configurable threshold. The simulation framework models interactions between the trace parser, DAG manager, pilot job manager, and HPC resources. The system improves scheduling responsiveness and resource utilization by reserving nodes as needed and scheduling jobs in batches based on DAG dependencies.

On-Demand Pilot Job Scheduling for Adaptive Execution of HEP Workflows on HPC Systems (poster)

Motivation: 
•	Parallel Discrete Event Simulations (PDES) offers accurate HPC simulations but is computationally intensive and slow to scale. 
•	Surrogate models can accelerate simulations, and we explore if causal insights can improve their long-term forecasting stability. 
Research Questions: 
•	Can Causal Signals hidden in HPC simulations unlock better forecasting? 
•	Can different Causal Discovery methods identify key drivers for accurate surrogate forecasting?

Temporal Causal Graph Discovery  in Complex HPC Network Traffic Simulations (poster)

Motivation: 
- Dragonfly networks suffer from inter-job interference due to shared global links. 
- Existing routing algorithms fail to account for system-wide job patterns and link congestion. 
- Modern HPC systems offer flexible job placement, allowing users to specify workload locations for optimized performance.

Preventing Workload Interference with Intelligent Routing and Flexible Job Placement Strategy on Dragonfly system (poster)

High-performance computing (HPC) job scheduling has seen promising advances with Deep Reinforcement Learning (DRL). However, challenges such as low interpretability, instability, and high computational cost hinder DRL&rsquo;s practical adoption. We explore a model-based alternative using Monte Carlo Tree Search (MCTS) to overcome these limitations. By leveraging existing HPC simulators as models for MCTS and focusing on transparent decision-making, we aim to develop a scalable and interpretable scheduling solution fit for real-world deployment.

Overcoming RL Limitations in HPC Scheduling: A Model-Based MCTS Approach for Practical Deployment (poster)

Large language model (LLM) requests vary widely: a short factual query may produce only a few dozen tokens, while multi-step reasoning or proof generation can run to tens of thousands. Because generation is autoregressive, inference time grows proportionally with output length. Current serving stacks such as vLLM and Orca schedule requests using a simple first-come-first-serve (FCFS) policy. While fair, FCFS suffers from a well-known issue in LLM serving called head-of-line blocking, where a single long request delays all shorter ones behind it - leading to inflated tail latency and wasted throughput. A classical remedy is the Shortest-Job-First (SJF) policy, which improves efficiency by serving shorter jobs first. However, SJF requires knowledge of job length in advance - information that is unavailable in LLMs until generation completes, due to their autoregressive nature. 
We introduce a prompt-sensitive LLM task scheduler based on pairwise learning method from learning-to-rank utilizing Margin Ranking Loss and show that our method significantly improves scheduling performance with minimal overhead.

From Prompts to Priorities: Pairwise Learning-to-Rank Scheduling for Low-Latency  LLM Serving (poster)

Prior uncore frequency tuning studies have primarily focused on conventional HPC workloads running on CPU-only systems. As HPC advances toward heterogeneous computing, integrating diverse GPU workloads on heterogeneous CPU-GPU systems, it becomes imperative to revisit and enhance uncore scaling. Our investigation reveals that uncore frequency scales down only when CPU power approaches its TDP (Thermal Design Power) -- a rare scenario in GPU-dominant applications -- resulting in unnecessary power waste in modern computing systems.

MAGUS: Memory-Throughput-Based Uncore Frequency Scaling on Heterogeneous Systems (poster)

AllReduce is a critical collective in both HPC and large-scale AI workloads. However, scaling it to Exascale systems presents key challenges due to inter-node communication bottlenecks and underutilization of intra-node resources like shared memory and NVLink. This work analyzes state-of-the-art AllReduce algorithms to identify inefficiencies and opportunities for hybrid strategies that explicitly separate intra- and inter-node communication. 
We introduce a preliminary algorithmic design that leverages tunable intra-node communication patterns and discuss key performance criteria, including message count and data volume. Our early results provide insight into communication trade-offs and guide the development of adaptive AllReduce implementations optimized for Exascale systems.

EMMA: Efficient Multi-node Memory-aware AllReduce Algorithms (poster)

While behavioral studies have explored the spectrum between ASD and ADHD, there is a lack of research examining this continuum through fMRI-based brain connectivity. Our motivation is to fill this gap by leveraging topological data analysis to reveal transitional patterns in functional networks underlying these conditions.

Neurological Diseases Spectrum Mapping using Topological Data Analysis (poster)

Non-uniform MPI_Alltoallv communication is critical in many high-performance computing (HPC) applications where data exchange patterns vary significantly between processes. However, existing MPI implementations rely on fixed heuristics that are not well-suited for dynamic, irregular workloads, often leading to suboptimal performance. Furthermore, there are customized non-uniform All-to-all functions having performance advantage compared to official MPI Alltoallv, under certain scenarios. Creating a framework which can accurately select the optimal algorithm will be extremely beneficial, but challenging due to its potential complexity. 
Currently, there are research works regarding the data-driven approach for tuning MPI functions, but most of them they focus on MPI functions such as MPI_Scatter, MPI_Reduce, MPI_gather, uniform MPI_Alltoall Non-uniform Alltoall is still an under-developed area. One challenge of developing a tuning framework for non-uniform Alltoall is that the blocksizes between processes are not fixed and it introduces significant additional complexity to the tuning workflow.

Performance Characterization and Tuning of Non-uniform All-to-all Data Exchanges (poster)

Bottom‐up logic programming (e.g., Datalog) runs via repeated relational‐algebra kernels - selection, projection, join, and reorder - until no new facts emerge. While GPUs promise massive parallelism for these primitives, existing engines remain CPU-bound (8–16 threads) and struggle with deduplication and growing result sets. 
We introduce a WebGPU-based hash-join pipeline that: 
- Maps each RA primitive (hash‐join, GPU radix sort, deduplication, set difference, insertion into the full relation) to its own WGSL compute shader 
- Uses dynamic, growable GPU buffers and load-factor-tuned hash tables for memory management 
- Drives a delta-only fixed-point loop, processing only newly discovered tuples each iteration

Accelerating Iterative Relational Algebra Operations with WebGPU (poster)

Research Contributions: 
- First ever Datalog engine designed for multinode multi-GPU HPC systems, outperforming state-of-the-art shared-memory, distributed-memory, and GPU-based engines 
- Introduces novel GPU-Aware communication and buffer preparation strategies for scalable recursive query evaluation 
- Supports recursive aggregation for Datalog rules using high-throughput GPU kernels

MNMGDatalog: A Scalable Multi-Node Multi-GPU Datalog Engine (poster)

Gait disorders are commonly observed in older adults, who frequently experience various issues related to walking. Additionally, researchers and clinicians extensively investigate mobility related to gait in typically and atypically developing children, athletes, and individuals with orthopedic and neurological disorders. Effective gait analysis enables the understanding of the causal mechanisms of mobility and balance control of patients, the development of tailored treatment plans
to improve mobility, the reduction of fall risk, and the tracking of rehabilitation progress. However, analyzing gait data is a complex task due to the multivariate nature of the data, the large volume of information to be interpreted, and the technical skills required. Existing tools for gait analysis are often limited to specific patient groups (e.g., cerebral palsy), only handle a specific subset of tasks in the entire workflow, and are not openly accessible. To address these shortcomings, we conducted a requirements assessment with gait practitioners (e.g., researchers, clinicians) via surveys and identified key components of the workflow, including (1) data processing and (2) data analysis and visualization. Based on the findings, we designed VIGMA, an open-access visual analytics framework integrated with computational notebooks and a Python library, to meet the identified requirements. Notably, the framework supports analytical capabilities for assessing disease progression and for comparing multiple patient groups. We validated the framework through usage scenarios with experts specializing in gait and mobility rehabilitation. VIGMA is available at <Za href="https://github.com/komar41/vigma">github.com/komar41/VIGMA</a>.
Index Terms: Gait, stroke, rehabilitation, visual analytics, data processing, multivariate data, ensemble, survey, computational notebook. 
<a href="https://doi.org/10.48550/arXiv.2504.17960">https://doi.org/10.48550/arXiv.2504.17960</a> 
<a href="https://youtu.be/rh_u81BPU94">See a VIGMA demonstration.</a>

Access to raw trial videos - is provided through a scrollable video exploration view. Users can toggle between spatiotemporal views and the video exploration view using a checkbox.

VIGMA: An Open-Access Framework for Visual Gait and Motion Analytics

AI development is shaped by academics and industry leaders - let us call them &ldquo;infuencers&rdquo; - but it is unclear how their views align with those of the public. To address this gap, we developed an interactive platform that served as a data collection tool for exploring public views on AI, including their fears, hopes, and overall sense of hopefulness. We made the platform available to 330 participants representative of the U.S. population in terms of age, sex, ethnicity, and political leaning, and compared their views with those of 100 AI infuencers identifed by Time magazine. The public fears AI getting out of control, while infuencers emphasize regulation, seemingly to defect attention from their alleged focus on monetizing A&rsquo;s potential. Interestingly, the views of AI infuencers from
underrepresented groups such as women and people of color often differ from the views of underrepresented groups in the public. 
:CCS Concepts: Human-centered computing - Empirical studies in HCI; Empirical studies in collaborative and social computing; HCI design and evaluation methods; Visualization systems and tools; Computing methodologies - Artifcial intelligence; Social and professional topics - Codes of ethics; Information systems - Crowdsourcing. 
Keywords: responsible AI, ethical AI, AI governance, empirical ethics, value alignment, AI fears, AI hopes, AI infuencers, participatory AI ethics, crowdsourcing

The frst version of our platform for collecting public views on AI encourages participants to share their honest opinions to help break down opinion bubbles (R2).

The Hall of AI Fears and Hopes: Comparing the Views of AI Influencers and those of Members of the U.S. Public Through an Interactive Platform

Mapping the extent of flood events is a necessary and important aspect of disaster management. In recent years, deep learning methods have evolved as an effective tool to quickly label high-resolution imagery and provide necessary flood extent mappings. These methods, though, require large amounts of annotated training data to create models that are accurate and robust to new flooded imagery. In this work, we present FloodTrace, a web-based application that enables effective crowdsourcing of flooded region annotation for machine learning applications. To create this application, we conducted extensive interviews with domain experts to produce a set of formal requirements. Our work brings topological segmentation tools to the web and greatly improves annotation efficiency compared to the state-of-the-art. The user-friendliness of our solution allows researchers to outsource annotations to non-experts and utilize them to produce training data with equal quality to fully expert-labeled data. We conducted a user study to confirm our application’s effectiveness in which 266 graduate students annotated high-resolution aerial imagery from Hurricane Matthew in North Carolina. Experimental results show the efficiency benefits of our application for untrained users, with median annotation time less than half the state-of-the-art annotation method. In addition, using our application’s aggregation and correction framework, flood detection models trained on crowdsourced annotations were able to achieve performance equal to models trained on fully expert-labeled annotations, while requiring a fraction of the expert&rsquo;s time. 
Keywords: Terrain visualization, machine learning, collaborative and distributed visualization, topology-based techniques 
Index Terms: Human-centered computing [Visualization]: Visualization application domains - Geographic visualization

Visualization of aggregated crowdsourced data from 45 participants.

Enabling Fast and Accurate Crowdsourced Annotation for Elevation-Aware Flood Extent Mapping

Graph drawing, or graph layout creation, is a computationally difficult challenge in visualization that involves placing the vertices of a graph into a layout that provides insight into its structure. In order to visualize large-scale graphs, effective layouts are necessary for understanding. Previous work has shown the potential for graph drawing directly in the web browser by using WebGPU, a new API that brings the full capabilities of modern GPUs to the web. Compared to the existing state-of-the-art for web-based graph visualization, which rely on CPU-based graph drawing algorithms, WebGPU-accelerated work improves performance and scalability. However, we find that existing WebGPU solutions utilize suboptimal quadtree data structures for graph drawing. In this work, we implement a modified quadtree data structure that uses a Hilbert spatial ordering for a fully parallelizable bottom-up construction algorithm in WebGPU. We utilize this data structure, along with optimizations to the quadtree traversal, to propose a massively more performant graph drawing algorithm. We evaluate the performance of our work against the existing state-of-the-art and demonstrate up to 69.5× speed-ups for layout creation of relevant graphs while enabling graph drawing for datasets of much larger size. 
Keywords: Graph layout creation, web-based, quadtree, GPU. 
Index Terms: Human-centered computing [Visualization]: Visualization techniques - Graph drawings

Example graph drawn for the Pkustk13 dataset from the SuiteSparse matrix collection, consisting of 94,893 vertices and 6,616,827 edges.

Accelerating Web-Based Graph Drawing with Bottom-Up GPU Quadtree Construction

Data visualization methodologies were intensively leveraged during the COVID-19 pandemic. We review our design experience working on a set of interdisciplinary COVID-19 pandemic projects. We describe the challenges we met in these projects, characterize the respective user communities, the goals and tasks we supported, and the data types and visual media we worked with. Furthermore, we instantiate these characterizations in a series of case studies. Finally, we describe the visual analysis lessons we learned, considering future pandemics. 
<a href="https://ieeexplore.ieee.org/document/10964487">https://ieeexplore.ieee.org/document/10964487</a>

Empowering Communities: Tailored Pandemic Data Visualization for Varied Tasks and Users

Authors: Humbert-Vidan. L., Kamel, S., Wentzel, A., Kaffey, Z., Abdelaal ,M., Spier, K. B., West, N. A., Marai, G. E., Canahuate, G., Zhang, X, Chen, M.M., Wahid, K. A., Rigert, J., Hosseinian, S., Schaefer, A. J., Brock, K. K., Chambers, M., Otun, A. O., Aponte-Wesson, R., Patel, V., Hope, A., Phan, J., Garden, A. S., Frank, S. J., Morrison, W. H., Spiott, M. T., Rosenthal, D., Lee, A., He, R., Naser, M. A., Watson, E., Hutcheson, K. A., Mohamed, A. S. R., Sandulache, V. C., van Dijk, L. V., Moreno, A. C., Urbano, T. G., Fuller, C. D., Lai, S. Y. 
Background 
Existing studies on osteoradionecrosis of the jaw (ORNJ) have primarily used cross-sectional data, assessing risk factors at a single time point. Determining the time-to-event profile of ORNJ has important implications to monitor oral health in head and neck cancer (HNC) long-term survivors. 
Methods 
Data were retrospectively obtained for a clinical observational cohort of 1129 patients (198 ORNJ cases) with HNC treated with radiotherapy (RT) at The University of Texas MD Anderson Cancer Center. A Weibull Accelerated Failure Time model was trained on previously identified dosimetric, clinical and demographic predictors. External validation was performed using an independent cohort of 265 patients (92 ORNJ cases) treated at Guy’s and St. Thomas’ Hospitals. To facilitate clinical implementation of the model, an online graphical user interface (GUI) was developed, including formal stakeholder usability testing. 
Results 
Our model identified that gender (males), pre-RT dental extractions and D25% were associated with a 38 %, 27 % and 12 % faster onset of ORNJ, respectively, with adjusted time ratios of 0.62 (p = 0.11), 0.73 (p = 0.13) and 0.88 (p < 0.005). The model demonstrated strong internal calibration (integrated Brier score of 0.133, D-calibration p-value 0.998) and optimal discrimination at 72 months (Harrell’s C-index of 0.72). 
Conclusion 
This study is the first to demonstrate a direct relationship between radiation dose and the time to ORNJ onset, providing a novel characterization of the impact of delivered dose and patient-related factors not only on the probability of a late effect (ORNJ), but the conditional risk during survivorship. 
Keywords: Osteoradionecrosis of the jaw, Time-to-event prediction models, Normal tissue complication prediction, Radiation-induced toxicity, Decision support tool, Radiotherapy, Head and neck cancers 
<a href="https://doi.org/10.1016/j.radonc.2025.110890">https://doi.org/10.1016/j.radonc.2025.110890</a>

Screenshot of WAFT-based time-to-ORNJ online calculator GUI. The user can either obtain a predicted risk of developing ORNJ at a specific time point or visually assess the time-dependency of ORNJ risk with the different covariates of the ORNJ WAFT model.

Externally validated digital decision support tool for time-to-osteoradionecrosis risk-stratification using right-censored multi-institutional observational cohorts

September 2023 marked the 50th anniversary of the Electronic Visualization Laboratory (EVL). This paper summarizes EVL&rsquo;s efforts in Visual Data Science, with a focus on the many networked, immersive, collaborative visualization and virtual reality (VR) systems and applications the Lab has developed and deployed, as
well as lessons learned and future plans. 
Index terms: Visualization, Virtual Reality, Visual Data Science, Collaboration

An EVL CAD Rendering of the DOCC Space

Immersive Analytics at the Electronic Visualization Laboratory

In consumer research, understanding consumer behavior and experiences is vital for making informed decisions about product design, innovation and marketing. Zaltman&rsqu;s Metaphor Elicitation Technique (ZMET) leverages metaphors and non-verbal communication to uncover and gain deeper insights into consumers&rsquo; thoughts and emotions. This paper introduces a novel system that enables consumer researchers (interviewers) to perform a modified version of metaphor elicitation interviews in virtual reality (VR). Consumers (participants) use 3D objects in the virtual environment to express their thoughts and emotions, instead of pictures conventionally used in a ZMET interview. The system features an asymmetric setup where the participant is immersed in VR using a head-mounted display (HMD), while the interviewer views the participant&rsquo;s perspective on a monitor. We discuss the technical and design aspects of the VR system and present the results of a user study (N = 17) that we conducted to validate the effectiveness of performing the metaphor elicitation interviews in VR. This work also explores the experiences of both participants and interviewers during the interview sessions, aiming to identify potential improvements. The qualitative and quantitative analysis of the data demonstrated how immersion, presence and embodied interaction positively affect and aid in sense-making and deeper expression of the participants&rsquo; thoughts, perspectives and emotions. 
Index Terms - Human computer interaction (HCI), virtual reality (VR), collaborative virtual environments, asymmetric VR, VR applications, embodied interaction, presence, immersion, perception and cognition, deep metaphors, consumer research, Zaltman&rsquo;s metaphor elicitation technique (ZMET).

Areas in the virtual home environment (A) Exercise area, (B) Outdoor space - summer and (C) Outdoor space - winter.

Enhancing Consumer Insights through VR Metaphor Elicitation

Patient-Reported Outcomes (PRO) consist of information provided directly by the patients about their health status including symptom ratings. PROs are commonly used in clinical practice to support clinical decision-making and have recently been incorporated into machine learning models to improve risk prediction. In this work, we aim to evaluate whether the inclusion of a patient stratification based on 12-month post-treatment predicted Patient Reported Outcomes improves risk prediction of radiation-induced toxicity and overall survival for head and neck cancer patients. A bidirectional long-short term memory (Bi-LSTM) recurrent neural network was used to model the longitudinal PRO data and to predict symptom ratings 12 months post-treatment. Patients were stratified using hierarchical clustering over the LSTM-predicted data. A logistic regression model was trained to predict Xerostomia at 12 months and a Cox regression model to predict overall survival. Results show that the inclusion of symptom burden clusters derived from the predicted Patient Reported Outcomes improves radiation-induced toxicity and overall survival prediction for head and neck cancer patients. 
Index Terms: Patient reported outcomes (PRO), deep learning, patient clustering, regression, survival analysis, xerostomia. 
<a href="https://doi.org/10.1109/JBHI.2024.3515092">https://doi.org/10.1109/JBHI.2024.3515092</a>

RMSEs at M12 for all 22 symptoms between Bi-LSTM prediction and actual testing data.

PRO-Based Stratification Improves Model Prediction for Toxicity and Survival of Head and Neck Cancer Patients

Hummingbird is an innovative, award-winning performance engaging participants in active storytelling that bridges live theater and collaborative interaction through virtual reality (VR). Hummingbird&rsquo;s story celebrates courage and coming of age through the eyes of a gutsy teen who must outsmart her mother&rsquo;s egotistic boss and survive a dangerous new technology in a live, immersive adventure. Developed at the University of Illinois Chicago by faculty and over 30 students from the departments of Computer Science and Design in partnership with professional theater producers, directors, actors, videographers and composers, this project advanced interdisciplinary collaboration, provided a unique learning environment and broadened the research experience for several cohorts of students. Over 500 people attended Hummingbird’s performances at the Tony Award-winning Goodman Theatre&rsquo;s New Stages Festival, Chicago Children&rsquo;s Theater and SIGGRAPH 2022 in Vancouver, Canada, with over 200 active VR participants. In each performance, five VR participants actively collaborate with each other and a lead actor within the VR adventure, contributing problem-solving, collaboration and teamwork, while a greater audience simultaneously follows the virtual performance aspects on a large video wall in real-time. Discussion sessions and audience evaluations followed each performance, informed the Hummingbird team on script, design and interactivity to improve future performances. Through qualitative analysis of audience experiences and insights from our collaboration, we discuss key considerations and design recommendations for integrating VR with live theater. Hummingbird demonstrates how VR can revolutionize theatrical storytelling by enabling traditional theater to narrate epic stories that were once considered too ambitious for traditional stage by extending live theater and making VR accessible to a broader audience. This project profit theater and interdisciplinary research institutions to increase opportunities for cross-disciplinary student education. 
CCS CONCEPTS: Computing methodologies; Computer Graphics; Graphics systems and interfaces; Virtual reality; Applied computing; Arts and humanities; Performing arts 
KEYWORDS Virtual reality, Storytelling, Theater, Multi-user, Performance, Interdisciplinary Collaboration, Design Process

VR participants interact in the Orchard scene while observers watch their interaction on the large screen.

Hummingbird: Live Theater Adventure Empowering Collaboration in Virtual Reality

Research in visual computing (VC) focuses on integrating visual representations of datasets with computing systems, thus leveraging the complementary abilities of humans and machines. The resulting VC systems may include machine learning (ML) techniques, which have remarkable abilities in terms of processing vast amounts of data. However, creating VC+ML systems that can operate on spatio-temporal data is a largely unexplored, difficult topic. Challenges in this area include: capturing domain characterizations for spatial VC+ML problems, in particular when serving both ML model builders and ML model clients; creating explainable VC+ML models that can operate on spatial data, including measures that can capture spatial structure similarity; designing visual encodings for ML models that use spatial information and that enhance the modeler and client understanding of the model; and measuring the effect of deploying such VC+ML systems in practice. br>
Based on several multi-year collaborations, in this dissertation I will document several instances where spatial and explainable VC+ML solutions are required, and what role spatiality plays in these situations. Throughout this process, I extract and describe design requirements for explainable spatial VC+ML systems, and I identify necessary VC advances, including spatially-aware similarity measures and spatial visual explainability. First, I construct spatially-aware similarity measures to support ML problems in precision oncology, patient risk stratification in data mining, explainable interactive regression modeling for hypothesis testing in social science, and digital twin interfaces for cancer treatment planning. Second, I introduce novel encodings for 3-dimensional and geospatial data and explore how
to integrate such encodings with ML model explanations. Finally, I implement and evaluate several VC+ML systems and several resulting models that are explicitly designed to integrate domain-specific structural spatial information and unsupervised ML. Throughout this process, I also articulate design lessons related to model actionability.

Development of the organ diagram used to encode organ-specific values. (Left) Organs that are located in the center of the head (Right) Bilateral organs that occur on both the left and right sides of the head.

Visual Computing Design for Explainable, Spatially-aware Machine Learning

With the decreasing cost of consumer display technologies making it easier for universities to have larger displays in classrooms, and the ubiquitous use of online tools such as collaborative whiteboards for remote learning during the COVID-19 pandemic, combining the two can be useful in higher education. This is especially true in visually intensive classes, such as data visualization courses, that can benefit from additional &ldquo;space to teach,&rdquo; coined after the &ldquo;space to think&rdquo; sense-making idiom. In this paper, we reflect on our approach to using SAGE3, a collaborative whiteboard with advanced features, in higher education to teach visually intensive classes, provide examples of activities from our own visually-intensive courses, and present student feedback. We gather our observations into usage patterns for using content-rich canvases in education.

A SAGE3 session can be shared across multiple locations: some locations may serve multiple users co-located in front of a large wall, while others may serve an individual with varied types of displays.

Space to Teach: Content-Rich Canvases for Visually-Intensive Education

Over the past decade, several urban visual analytics systems and tools have been proposed to tackle a host of challenges faced by cities, in areas as diverse as transportation, weather, and real estate. Many of these tools have been designed through collaborations with urban experts, aiming to distill intricate urban analysis workflows into interactive visualizations and interfaces. However, the design, implementation, and practical use of these tools still rely on siloed approaches, resulting in bespoke systems that are difficult to reproduce and extend. At the design level, these tools undervalue rich data workflows from urban experts, typically treating them only as data providers and evaluators. At the implementation level, they lack interoperability with other technical frameworks. At the practical use level, they tend to be narrowly focused on specific fields, inadvertently creating barriers to cross-domain collaboration. To address these gaps, we present Curio, a framework for collaborative urban visual analytics. Curio uses a dataflow model with multiple abstraction levels (code, grammar, GUI elements) to facilitate collaboration across the design and implementation of visual analytics components. The framework allows experts to intertwine data preprocessing, management, and visualization stages while tracking the provenance of code and visualizations. In collaboration with urban experts, we evaluate Curio through a diverse set of usage scenarios targeting urban accessibility, urban microclimate, and sunlight access. These scenarios use different types of data and domain methodologies to illustrate Curio&rsquo;s flexibility in tackling pressing societal challenges. Curio is available at <a=href"urbantk.org/curio.">urbantk.org/curio</a>. 
Index Terms - Urban analytics, urban data, spatial data, dataflow, provenance, visualization framework, visualization system.

Using Curio to build a dataflow to analyze the shadow impact of a proposed building in Boston.

Curio: A Dataflow-Based Framework for Collaborative Urban Visual Analytics

Unsupervised video anomaly detection (UVAD) aims to detect abnormal events in videos without any annotations. It remains challenging because anomalies are rare, diverse, and usually not well-defined. Existing UVAD methods are purely data-driven and perform unsupervised learning by identifying various abnormal patterns in videos. Since these methods largely rely on the feature representation and data distribution, they can only learn salient anomalies that are substantially different from normal events but ignore the less distinct ones. To address this challenge, this paper pursues a different approach that leverages data-irrelevant prior knowledge about normal and abnormal events for UVAD. We first propose a new normality prior for UVAD, suggesting that the start and end of a video are predominantly normal. We then propose normality propagation, which propagates normal knowledge based on relationships between video snippets to estimate the normal magnitudes of unlabeled snippets. Finally, unsupervised learning of abnormal detection is performed based on the propagated labels and a new loss re-weighting method. These components are complementary to normality propagation and mitigate the negative impact of incorrectly propagated labels. Extensive experiments on the ShanghaiTech and UCF-Crime benchmarks demonstrate the superior performance of our method. 
Funding: NSF award ECCS-2400900

Learning Anomalies with Normality Prior for Unsupervised Video Anomaly Detection

Deep learning approaches have made significant success in single-view 3D reconstruction, but they often rely on expensive 3D annotations for training. Recent efforts tackle this challenge by adopting an analysis-by-synthesis paradigm to learn 3D reconstruction with only 2D annotations. However, existing methods face limitations in both shape reconstruction and texture generation. This paper introduces an innovative Analysis-by-Synthesis Transformer that addresses these limitations in a unified framework by effectively modeling pixel-to-shape and pixel-to-texture relationships. It consists of a Shape Transformer and a Texture Transformer. The Shape Transformer employs learnable shape queries to fetch pixel-level features from the image, thereby achieving high-quality mesh reconstruction and recovering occluded vertices. The Texture Transformer employs texture queries for non-local gathering of texture information and thus eliminates incorrect inductive bias. Experimental results on CUB-200-2011 and ShapeNet datasets demonstrate superior performance in shape reconstruction and texture generation compared to previous methods. 
Funding: NSF award CNS-1828265, NSF award ECCS-2400900

Analysis-by-Synthesis Transformer for Single-View 3D Reconstruction

In the biomedical domain, taxonomies organize the acquisition modalities of scientific images in hierarchical structures. Such taxonomies leverage large sets of correct image labels and provide essential information about the importance of a scientific publication, which could then be used in biocuration tasks. However, the hierarchical nature of the labels, the overhead of processing images, the absence or incompleteness of labeled data, and the expertise required to label this type of data impede the creation of useful datasets for biocuration. From a multi-year collaboration with biocurators and text-mining researchers, we derive an iterative visual analytics and active learning strategy to address these challenges. We implement this strategy in a system called BI-LAVA—Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analytics. BILAVA leverages a small set of image labels, a hierarchical set of image classifiers, and active learning to help model builders deal with incomplete ground-truth labels, target a hierarchical taxonomy of image modalities, and classify a large pool of unlabeled images. BI-LAVA&rsquo;s front end uses custom encodings to represent data distributions, taxonomies, image projections, and neighborhoods of image thumbnails, which help model builders explore an unfamiliar image dataset and taxonomy and correct and generate labels. An evaluation with machine learning practitioners shows that our mixed human-machine approach successfully supports domain experts in understanding the characteristics of classes within the taxonomy, as well as validating and improving data quality in labeled and unlabeled collections. 
CCS Concepts: Human-centered computing -> Visual analytics; Computing methodologies -> Machine learning algorithms

Interaction during the case study: COVID-19 Image Modality Labeling.

BI-LAVA: Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analytics

Over the past decade, there has been a significant increase in the development of visual analytics systems dedicated to addressing urban issues. These systems distill intricate urban analysis workflows into intuitive, interactive visual representations and interfaces, enabling users to explore, understand, and derive insights from large and complex data, including street-level imagery, street networks, and building geometries. Developing urban visual analytics systems, however, is a challenging endeavor that requires considerable programming expertise and interaction between various multidisciplinary stakeholders. This situation often leads to monolithic and isolated prototypes that are hard to reproduce, combine, or extend. Concurrently, there has been an increase in the availability of general and urban-specific toolkits, frameworks, and authoring tools that are open source and abstract away the need to implement low-level visual analytics functionalities. This paper provides a hierarchical taxonomy of urban visual analytics systems to contextualize how they are usually designed, implemented, and evaluated. We develop this taxonomy across three distinct levels (i.e., dimensions, categories, and tags), juxtaposing visualization with analytics, data, and system dimensions. We then assess the extent to which current open-source toolkits, frameworks, and authoring tools can effectively support the development of components tailored to urban visual analytics, identifying their strengths and limitations in addressing the unique challenges posed by urban data. In doing so, we offer a roadmap that can guide the effective employment of existing resources and chart a pathway for developing and refining future systems. 
Keywords: Visual Analytics, Visualization Toolkits, Visualization Grammars, Visualization Authoring, Urban Visual Analytics

Assessing the Landscape of Toolkits, Frameworks, and Authoring Tools for Urban Visual Analytics Systems

This study addresses the prevalent issue of missing data in patient-reported outcome datasets, particularly focusing on head and neck cancer patient symptom ratings sourced from the MD Anderson Symptom Inventory. Given that many data mining and machine learning algorithms necessitate complete datasets, the accurate imputation of missing data as an initial step becomes crucial. In this study we propose, for the first time, the use of collaborative filtering for imputing missing head and neck cancer patient symptom ratings. Two configurations of collaborative filtering, namely patient-based and symptom-based, leverage known ratings to infer the missing ones. Additionally, this study compares the performance of collaborative filtering with alternative imputation methods such as Multiple Imputation by Chained Equations, Nearest Neighbor Imputation, and Linear interpolation. Performance is compared using Root Mean Squared Error and Mean Absolute Error metrics. Findings demonstrate that collaborative filtering is a viable and comparatively superior approach for imputing missing patient symptom data. 
Keywords: Head and Neck Cancer, Imputation, Collaborative Filtering 
<a href="https://doi.org/10.1007/978-3-031-68309-1_20">https://doi.org/10.1007/978-3-031-68309-1_20</a>

Collaborative Filtering for the Imputation of Patient Reported Outcomes

September 2023 marks the 50th anniversary of the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC). EVL&rsquo;s introduction of the CAVE Automatic Virtual Environment in 1992, the first widely replicated, projection-based, walk-in, virtual-reality (VR) system in the world, put EVL at the forefront of collaborative, immersive data exploration and analytics. However, the journey did not begin then. Since its founding in 1973, EVL has been developing tools and techniques for real-time, interactive visualizations - pillars of VR. But EVL&rsquo;s culture is also relevant to its successes, as it has always been an interdisciplinary lab that fosters teamwork, where each person&rsquo;s expertise contributes to the development of the necessary tools, hardware, system software, applications, and human interface models to solve problems. Over the years, as multidisciplinary collaborations evolved and advanced scientific instruments and data resources were distributed globally, the need to access and share data and visualizations while working with colleagues, local and remote, synchronous and asynchronous, also became important fields of study. This paper is a retrospective of EVL&rsquo;s past 50 years that surveys the many networked, immersive, collaborative visualization and VR systems and applications it developed and deployed, as well as lessons learned and future plans. 
Article appears in the journal PRESENCE: Virtual and Augmented Reality, in the Special Issue on Immersive Visualization Laboratories - Past, Present and Future, edited by S. Su and W. Sherman, <a href="https://direct.mit.edu/pvar/issue/volume/33">https://direct.mit.edu/pvar/issue/volume/33</a>. 
Article copyrighted by Massachusetts Institute of Technology (MIT). 
DOI: <a href="https://doi.org/10.1162/pres_a_00421">https://doi.org/10.1162/pres_a_00421</a>

EVL faculty, students, and collaborators 1975.

Electronic Visualization Laboratory&rsquo;s 50th Anniversary Retrospective: Look to the Future, Build on the Past

Meetings and conversations today are multimodal in nature, including the usage of diverse technologies in the form of text, video, images, voice, AI agents, and also immersive technologies. If it were possible to replay important multimodal conversations in their spatial and chronological contexts, it would benefit several audiences, such as - researchers to study and analyze multimodal conversations, novices in any field who could learn from experts, experts who have missed being part of crucial conversations. However, recreating such multimodal interactions requires integrating all of the modalities used in the conversation. Immersive environments provide a convenient setting to present data with its spatial and temporal referents through the use of situated analytics. We present MuSA (Multimodal Situated Analytics), an immersive environment prototype, as a way to recreate multimodal conversations for exploration and analysis in immersive environments. Our development pipeline consisted of various stages such as data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We conducted two comprehensive user studies to evaluate the usability, user adoption, and space usage during the exploration of multimodal conversations with our application. In the first study, we explore conversations between seated participants (n=12), and in the second, we explore conversations between non-seated moving participants (n=13). We share insights gained during the development phase, as well as empirical results and feedback collected through both user studies.

Depiction of an analyst investigating a multimodal conversation that includes participants and other entities of the conversation through MuSA in Mixed Reality (HoloLens2).

Multimodal Situated Analytics (MuSA) for Analyzing Conversations in Extended Reality

SAGE3 (the Smart Amplified Group Environment) is an open-source collaboration software platform designed for real-time interactive information and visualization sharing. Seamlessly complementing video conferencing tools like Zoom, Microsoft Teams, and Google Meets, it enhances group environments for information-rich collaboration. With SAGE3, users can collaborate on their laptops, meeting room projectors, and on large tiled display walls. SAGE3 on large display spaces has proven to enable people to think, brainstorm, discover and reach decisions with greater speed, accuracy, comprehensiveness and confidence. SAGE3 provides a large collaborative canvas for sharing and editing of media, data, code, visualizations, dashboards, and screen-shares. SAGE3 is ideally suiting for collaborative data science, project meetings, and classroom education scenarios. <a href="https://sage3.sagecommons.org/">https://sage3.sagecommons.org/</a>

SAGE3: Smart Amplified Group Environment for harnessing the data revolution (poster)

Digital twin models are of high interest to Head and Neck Cancer (HNC) oncologists, who have to navigate a series of complex treatment decisions that weigh the efficacy of tumor control against toxicity and mortality risks. Evaluating individual risk profiles necessitates a deeper understanding of the interplay between different factors such as patient health, spatial tumor location and spread, and risk of subsequent toxicities that can not be adequately captured through simple heuristics. To support clinicians in better understanding tradeoffs when deciding on treatment courses, we developed DITTO, a digital-twin and visual computing system that allows clinicians to analyze detailed risk profiles for each patient, and decide on a treatment plan. DITTO relies on a sequential Deep Reinforcement Learning digital twin (DT) to deliver personalized risk of both long-term and short-term disease outcomes and toxicity risk for HNC patients. Based on a participatory collaborative design alongside oncologists, we also implement several visual explainability methods to promote clinical trust and encourage healthy skepticism when using our system. We evaluate the efficacy of DITTO through quantitative evaluation of performance and case studies with qualitative feedback. Finally, we discuss design lessons for developing clinical visual XAI applications for clinical end users. 
Index Terms: Medicine; Machine Learning; Application Domains; High Dimensional data; Spatial Data; Activity Centered Design

Overview of DITTO.

DITTO: A Visual Digital Twin for Interventions and Temporal Treatment Outcomes in Head and Neck Cancer

Though the object detection performance on standard benchmarks has been improved drastically in the last decade, current object detectors are often vulnerable to domain shift between the training data and testing images. Domain adaptation techniques have been developed to adapt an object detector trained in a source domain to a target domain. However, they assume that the target domain is known and fixed and that a target dataset is available for training, which cannot be satisfied in many real-world applications. To close this gap, this paper investigates fully test-time adaptation for object detection. It means to update a trained object detector on a single testing image before making a prediction, without access to the training data. Through a diagnostic study of a baseline self-training framework, we show that a great challenge of this task is the unreliability of pseudo labels caused by domain shift. We then propose a simple yet effective method, termed the IoU Filter, to address this challenge. It consists of two new IoU-based indicators, both of which are complementary to the detection confidence. Experimental results on five datasets demonstrate that our approach could effectively adapt a trained detector to various kinds of domain shifts at test time and bring substantial performance gains. Code is available at https://github.com/XiaoqianRuan1/IoU-filter. Code is available at https://github.com/ XiaoqianRuan1/IoU-filter. 
Funding:: The COMPaaS DLV project (NSF award CNS-1828265)

Fully Test-time Adaptation for Object Detection

Unsupervised semantic segmentation (USS) aims at partitioning an image into semantically meaningful segments by learning from a collection of unlabeled images. The effectiveness of current approaches is plagued by difficulties in coordinating representation learning and pixel clustering, modeling the varying feature distributions of different classes, handling outliers and noise, and addressing the pixel class imbalance problem. This paper introduces a novel approach, termed Imbalance-Aware Dense Discriminative Clustering (IDDC), for USS, which addresses all these difficulties in a unified framework. Different from existing approaches, which learn USS in two stages (i.e., generating and updating pseudo masks, or refining and clustering embeddings), IDDC learns pixel-wise feature representation and dense discriminative clustering in an end-to-end and self-supervised manner, through a novel objective function that transfers the manifold structure of pixels in the embedding space of a vision Transformer (ViT) to the label space while tolerating the noise in pixel affinities. During inference, the trained model directly outputs the classification probability of each pixel conditioned on the image. In addition, this paper proposes a new regularizer, based on the Weibull function, to handle pixel class imbalance and cluster degeneration in a single shot. Experimental results demonstrate that IDDC significantly outperforms all previous USS methods on three real-world datasets, COCO-Stuff-27, COCO-Stuff-171, and Cityscapes. Extensive ablation studies validate the effectiveness of each design. Our code is available at https://github.com/MY-LIU100101/IDDC. Our code is available at https://github.com/MY-LIU100101/IDDC. 
Keywords: Unsupervised semantic segmentation · Imbalance-Aware Dense Discriminative Clustering · End-to-end training ·
Deep clustering 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Imbalance-Aware Discriminative Clustering for Unsupervised Semantic Segmentation

Studies have proven the effectiveness of mindfulness and forest-immersion therapy as an option for stress reduction. However, what happens when an individual is seeking stress relief but the outdoors is unavailable due to unpredictable weather or even a lack of accessibility? The purpose of this research project is to observe the effects of guided scenic meditation in a virtual reality environment on physiological stress. Additionally, it performs a platform comparison by deploying the same meditative experience and testing its effectiveness using a room-scale immersive environment and a desktop computer, as desktops ($400 - $1000) are far more affordable and widely available compared to room-scale immersive environments ($1 million+). Using each platform, twenty participants were asked to engage with a meditative application that simulates the sights, sounds, and smells of a forest environment. Based on the objective data collected, the participants experienced a decrease in both heart rate and blood pressure, demonstrating the effectiveness of the simulation in reducing physiological stress. However, the room-scale immersive environment proved to be more successful in reducing the heart rate than the desktop computer. Furthermore, according to the participants, their subjectively perceived stress levels decreased, resulting in a far more relaxed state than before. The results showcase that a virtual simulation of a real environment actively contributes to stress reduction and can be used in cases when the outdoors is unavailable. The platform comparison demonstrates that the product of the research - a guided scenic meditation application that simulates a forest - is accessible and does not require a million-dollar room-scale display in order to be effective in reducing stress.

The guided scenic meditation application deployed to the CAVE2, pictured on the left, and the desktop, pictured on the right.

The Effects of Virtual Reality Environments on Physiological Stress: A Platform Comparison Between Room-Scale Displays and Desktop Computers (MS Thesis Project)

This project was performed during the COVID-19 pandemic. It presents a visual computing framework (MOTIV) for analyzing moral rhetoric on social media around controversial topics such as Stay-at-Home orders. Using Moral Foundation Theory, it proposes a methodology for deconstructing and visualizing the when, where, and who behind each of these moral dimensions as expressed in microblog data. The project was developed in collaboration with experts from language processing, communications, and causal inference. Our approach integrates microblog data with multiple sources of geospatial and temporal data, and leverages unsupervised machine learning (generalized additive models) to support collaborative hypothesis discovery and testing. 
Abstract: We present a visual computing framework for analyzing moral rhetoric on social media around controversial topics. Using Moral Foundation Theory, we propose a methodology for deconstructing and visualizing the when, where, and who behind each of these moral dimensions as expressed in microblog data. We characterize the design of this framework, developed in collaboration with experts from language processing, communications, and causal inference. Our approach integrates microblog data with multiple sources of geospatial and temporal data, and leverages unsupervised machine learning (generalized additive models) to support collaborative hypothesis discovery and testing. We implement this approach in a system named MOTIV. We illustrate this approach on two problems, one related to Stay-at-home policies during the COVID-19 pandemic, and the other related to the Black Lives Matter movement. Through detailed case studies and discussions with collaborators, we identify several insights discovered regarding the different drivers of moral sentiment in social media. Our results indicate that this visual approach supports rapid, collaborative hypothesis testing, and can help give insights into the underlying moral values behind controversial political issues. 
Supplemental Material: <a href="https://osf.io/ygkzn/?view_only=6310c0886938415391d977b8aae8b749">https://osf.io/ygkzn/?view_only=6310c0886938415391d977b8aae8b749</a>

MOTIV visualization of moral framing on social media in 2020.

MOTIV: Visual Exploration of Moral Framing in Social Media

Sunlight and shadow play critical roles in how urban spaces are utilized, thrive, and grow. While access to sunlight is essential to the success of urban environments, shadows can provide shaded places to stay during the hot seasons, mitigate heat island effect, and increase pedestrian comfort levels. Properly quantifying sunlight access and shadows in large urban environments is key in tackling some of the important challenges facing cities today. In this paper, we propose Dee Umbra, a novel computational framework that enables the quantification of sunlight access and shadows at a global scale. Our framework is based on a conditional generative adversarial network that considers the physical form of cities to compute high-resolution spatial information of accumulated sunlight access for the different seasons of the year. We use data from seven different cities to train our model, and show, through an extensive set of experiments, its low overall RMSE (below 0.1) as well as its extensibility to cities that were not part of the training set. Additionally, we contribute a set of case studies and a comprehensive dataset with sunlight access information for more than 100 cities across six continents of the world. Deep Umbra is available at <a href="https://urbantk.org/shadows">https://urbantk.org/shadows</a>. 
Index Terms - Urban computing, Urban analytics, Sunlight access, Shadow, Generative adversarial networks.

Example where shadows are cast across tiles.

Deep Umbra: A Generative Approach for Sunlight Access Computation in Urban Spaces

e-JUST - Environmental Justice using Urban Scalable Toolkit, a state-of-the-art GIS-based web platform, is being developed to inform local and regional decision-making regarding air quality and heat in the Chicago metropolitan area, utilizing Earth science and high-resolution climate model outputs. e-JUST contributes to mitigating risks associated with extreme heat and air pollution by improving decision-making related to heat and air quality stressors in the Chicago metropolitan region. e-JUST technology architecture involves ReactJS, Vega-Lite, and Urban Toolkit's functionalities. e-JUST integrates a variety of meteorological and air quality datasets. Among these are NASA&rsquo;s remote sensing observations, such as GOES 16 and ECOSTRESS, that measure surface temperatures at 100 m and hourly intervals. The environmental data layers include high-resolution (1 km and higher) outputs from urban climate models related to temperature, NO2 and PM2.5, ozone, and other pollutants. To account for demographic data, we use the 2020 US Census. Our data fusion approaches include further downscaling environmental data layers to street scale using machine learning algorithms for spatiotemporal interpolation, enabling more accurate predictions at scales relevant to communities. e-JUST&rsquo;s statistical view includes a heat matrix that illustrates correlations between several environmental and socioeconomic variables at a grid point in time, as well as a scatterplot that allows users to compare observed data and forecasts of two different variables. Overall, this toolkit provides information for the near past and near-real-time forecasting to aid in decision-making and emergency management preparedness for Chicago communities. Additionally, our platform is designed to provide users with easy access to data, enabling them to view and analyze data immediately. The e-JUST toolkit is anticipated to benefit a wide audience, ranging from the general public to academic researchers.

e-JUST - Environmental Justice using Urban Scalable Toolkit

Similarity detection seeks to identify similar, but distinct items over multivariate datasets. Often, similarity cannot be defined computationally, leading to a need for visual analysis, such as in cases with ensemble, computational, patient cohort, or geospatial data. In this work, we empirically evaluate the effectiveness of common visual encodings for multivariate data in the context of visual similarity detection. We conducted a user study with 40 participants to measure similarity detection performance and response time under moderate scale (16 items) and large scale (36 items). Our analysis shows that there are significant differences in performance between encodings, especially as the number of items increases. Surprisingly, we found that juxtaposed star plots outperformed superposed parallel coordinate plots. Furthermore, color-cues significantly improved response time, and attenuated error at larger scales. In contrast to existing guidelines, we found that filled star plots (Kiviats) outperformed other encodings in terms of scalability and error.

Similarity detection over multivariate data using five visual encodings (clockwise order from top left): Kiviat diagrams, Lines-Only glyphs, Color-cue Kiviat diagrams, Parallel Coordinate Plots, and Color-cue Parallel Coordinate Plots.

Kiviat Defense: An Empirical Evaluation of Visual Encoding Effectiveness in Multivariate Data Similarity Detection

Biocuration is the process of analyzing biological or biomedical articles to organize biological data into data repositories using taxonomies and ontologies. Due to the expanding number of articles and the relatively small number of biocurators, automation is desired to improve the workflow of assessing articles worth curating. As figures convey essential information, automatically integrating images may improve curation. In this work, we instantiate and evaluate a first-in-kind, hybrid image+text document search system for biocuration. The system, MouseScholar, leverages an image modality taxonomy derived in collaboration with biocurators, in addition to figure segmentation, and classifiers components as a back-end and a streamlined front-end interface to search and present document results. We formally evaluated the system with ten biocurators on a mouse genome informatics biocuration dataset and collected feedback. The results demonstrate the benefits of blending text and image information when presenting scientific articles for biocuration. 
Index Terms - document search, biocuration

MouseScholar interface.

MouseScholar: Evaluating an Image+Text Search System for Biocuration

Rayleigh-Benard (RB) convection occurs in a fluid layer when there is a temperature difference between the top and bottom boundaries, resulting in the spontaneous formation of convective motion.  In this study, a direct numerical simulation has been performed aimed at elucidating the impact of introducing dispersed droplets into a single-phase flow on the heat transfer dynamics within this flow.  The dimensionless parameters considered in this study encompass a Prandtl number of 4, a Weber number of 6000, and a Rayleigh number of 10^8.  Dispersed droplets were incorporated over a range spanning from 0 to 50%.

Effects of employing liquid-liquid emulsions on heat transfer within a turbulent Rayleigh-Benard convection

In the realm of Computational Fluid Dynamics (CFD), the demand for memory and computation resources is extreme, necessitating the use of leadership-scale computing platforms for practical domain sizes. This intensive requirement renders traditional checkpointing methods ineffective due to the significant slowdown in simulations while saving state data to disk. As we progress towards exascale and GPU-driven High-Performance Computing (HPC) and confront larger problem sizes, the choice becomes increasingly stark: to compromise data fidelity or to reduce resolution. To navigate this challenge, this study advocates for the use of in situ analysis and visualization techniques. These allow more frequent data &ldquo;snapshots&rdquo; to be taken directly from memory, thus avoiding the need for disruptive checkpointing. We detail our approach of instrumenting NekRS, a GPU-focused thermal-fluid simulation code employing the spectral element method (SEM), and describe varied in situ and in transit strategies for data rendering. Additionally, we provide concrete scientific use cases and report on runs performed on Polaris, Argonne Leadership Computing Facility’s (ALCF) 44 Petaflop supercomputer and Jülich Wizard for European Leadership Science (JUWELS) Booster, Jülich Supercomputing Centre’s (JSC) 71 Petaflop High Performance Computing (HPC) system, offering practical insight into the implications of our methodology.

Side view visualization of a RBC case.

Scaling Computational Fluid Dynamics: In Situ Visualization of NekRS using SENSEI

SAGE3, the newest and most advanced generation of the Smart Amplified Group Environment, is an open-source software designed to facilitate collaboration among scientists, researchers, students, and professionals across various fields. This tutorial aims to introduce attendees to the capabilities of SAGE3, demonstrating its ability to enhance collaboration and productivity in diverse settings, from co-located office collaboration to remote collaboration to both at once, with diverse displays, from personal laptops to large-scale display walls. Participants will learn how to effectively use SAGE3 for brainstorming, data analysis, and presentation purposes, as well as installation of private collaboration servers and development of custom applications. 
CCS CONCEPTS: Human-centered computing, Collaborative and social computing systems and tools; Open source software; Visualization systems and tools; Interactive systems and tools. 
KEYWORDS: Large Displays, Space to Think, Collaboration, Visualization, Data Science, Data Analysis, Computational Narratives

SAGE3 for Interactive Collaborative Visualization, Analysis, and Storytelling

Applications in immersive environments have gained popularity for training, learning, and recreational tasks. Due to the increasing availability of sensors and data-capturing devices, research is extending the use of immersive environments to support visual analytics processes, including sensemaking and strategic immersion for interaction and task completion. In this work, we propose Personal Situated Analytics (PSA) framework to embed users into recorded meetings with support for multiple degrees of immersion in the Reality-Virtuality spectrum. Our proposed framework encompasses various stages such as tracking, data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We evaluate this framework on a data analysis scenario between human subjects and a conversational AI agent. Our pilot study (n=12) using this framework compares user experiences when using two different devices: Hololens2 and Quest2.

A user embedded in a conversation using Personal Situated Analytics in Virtual Reality as virtual avatars.

Using Personal Situated Analytics (PSA) to Interpret Recorded Meetings

Personalized head and neck cancer therapeutics have greatly improved survival rates for patients, but are often leading to understudied long-lasting symptoms which affect quality of life. Sequential rule mining (SRM) is a promising unsupervised machine
learning method for predicting longitudinal patterns in temporal data which, however, can output many repetitive patterns that are difficult to interpret without the assistance of visual analytics. We present a data-driven, human-machine analysis visual system
developed in collaboration with SRM model builders in cancer symptom research, which facilitates mechanistic knowledge discovery in large-scale, multivariate cohort symptom data. Our system supports multivariate predictive modeling of post-treatment symptoms
based on during-treatment symptoms. It supports this goal through an SRM, clustering, aggregation back end, and a custom front
end to help develop and tune the predictive models. The system also explains the resulting predictions in the context of therapeutic
decisions typical in personalized care delivery. We evaluate the resulting models and system with an interdisciplinary group of modelers and head and neck oncology researchers. The results demonstrate that our system effectively supports clinical and symptom research. 
Index Terms: Temporal Data; Life Sciences; Mixed Initiative Human-Machine Analysis; Data Clustering and Aggregation

Longitudinal symptom analysis and prediction for head and neck patients (ICC treatment).

Roses Have Thorns: Understanding the Downside of Oncological Care Delivery Through Visual Analytics and Sequential Rule Mining

While cities around the world are looking for smart ways to channel new advances in data collection, management, and analysis to address their day-to-day problems, the complex nature of urban issues and the overwhelming amount of available structured and unstructured data have posed significant challenges in translating these efforts into actionable insights. In the past few years, urban visual analytics tools have significantly helped tackle these challenges. When analyzing a feature of interest, an urban scientist must transform, integrate and visualize different thematic (e.g., sunlight, flows) and physical (e.g., buildings, sidewalks) layers, oftentimes across multiple spatial and temporal scales. However, integrating and analyzing these layers requires expertise in different fields, increasing development time and effort. This makes the entire visual data exploration and system implementation hard for experienced programmers and also sets a high entry barrier for urban experts outside of computer science. With this in mind, in this paper, we present the Urban Toolkit (UTK), a flexible and extensible visualization framework that enables the easy authoring of web-based visualizations through a new high-level grammar specifically built with common urban use cases in mind. In order to facilitate the integration and visualization of different urban data, we also propose the concept of knots to merge thematic and physical urban layers. We evaluate our approach through use cases and a series of interviews with experts and practitioners from different domains, including urban accessibility, urban planning, architecture, and climate science. UTK is available at urbantk.org. 
Index Terms: Urban visual analytics, Urban analytics, Urban data, Visualization toolkit.

UTK abstracts spatial joins through the use of knots.

The Urban Toolkit: A Grammar-based Framework for Urban Visual Analytics

Weather forecasting is essential for decision-making and is usually performed using numerical modeling. Numerical weather models, in turn, are complex tools that require specialized training and laborious setup and are challenging even for weather experts. Moreover, weather simulations are data-intensive computations and may take hours to days to complete. When the simulation is finished, the experts face challenges analyzing its outputs, a large mass of spatiotemporal and multivariate data. From the simulation setup to the analysis of results, working with weather simulations involves several manual and error-prone steps. The complexity of the problem increases exponentially when the experts must deal with ensembles of simulations, a frequent task in their daily duties. To tackle these challenges, we propose ProWis: an interactive and provenance-oriented system to help weather experts build, manage, and analyze simulation ensembles at runtime. Our system follows a human-in-the-loop approach to enable the exploration of multiple atmospheric variables and weather scenarios. ProWis was built in close collaboration with weather experts, and we demonstrate its effectiveness by presenting two case studies of rainfall events in Brazil. 
Index Terms - Weather visualization, Ensemble visualization, Provenance management, WRF visual setup

Building, managing, and analyzing a Weather Research and Forecasting (WRF) model ensemble using PROWIS to study an extreme rainfall event in Maricá (RJ, Brazil) in April 2022.

ProWis: A visual approach for building, managing, and analyzing weather simulation ensembles in runtime

New web technologies have enabled the deployment of powerful GPU-based computational pipelines that run entirely in the web browser, opening a new frontier for accessible scientific visualization applications. However, these new capabilities do not address the memory constraints of lightweight end-user devices encountered when attempting to visualize the massive data sets produced by today&rsquo;s simulations and data acquisition systems. In this paper, we propose a novel implicit isosurface rendering algorithm for interactive visualization of massive volumes within a small memory footprint. We achieve this by progressively traversing a wavefront of rays through the volume and decompressing blocks of the data on-demand to perform implicit ray-isosurface intersections. The progressively rendered surface is displayed after each pass to improve interactivity. Furthermore, to accelerate rendering and increase GPU utilization, we introduce speculative ray-block intersection into our algorithm, where additional blocks are traversed and intersected speculatively along rays as other rays terminate to exploit additional parallelism in the workload. Our entire pipeline is run in parallel on the GPU to leverage the parallel computing power that is available even on lightweight end-user devices. We compare our algorithm to the state of the art in low-overhead isosurface extraction and demonstrate that it achieves 1.7×–5.7× reductions in memory overhead and up to 8.4× reductions in data decompressed.

Interactive full-resolution isosurface visualization of the 2048×2048×1920 Richtmyer-Meshkov (R-M) data set in the browser.

Speculative Progressive Raycasting for Memory Constrained Isosurface Visualization of Massive Volumes

Using immersive environments for training, education, and recreation is becoming increasingly popular among individuals. In recent years these environments have also been used to facilitate data analytics processes involving interaction and task completion. In this work, we discuss Personal Situated Analytics (PSA) a cross-platform framework to embed users into recorded conversations with support for multiple degrees of immersion in the Reality-Virtuality spectrum. Our development pipeline consisted of several stages: tracking, data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We evaluate PSA by analyzing data from a recorded meeting involving human participants and a visual conversational AI agent. Our pilot study (n = 12) using this framework compares user experiences for MR and VR environments. 
Index Terms: Immersive Analytics, Situated Analytics, Visualization techniques, Human-centered computing

PSA is an immersive MR/VR framework for experiencing and analyzing recorded conversations and events. We see two participants represented as virtual avatars engaged in a conversation.

PSA: A Cross-Platform Framework for Situated Analytics in MR and VR

Estimating the human posture from an image or a video is a fundamental task in computer vision. It not only enhances other vision tasks like action recognition, person re-identification, and virtual try-on but also facilitates many real-world applications including robotics, healthcare, sports, and retail. An effective and efficient human pose estimation system can enable robots to acquire skills from demonstrations, support physical therapists in diagnosing and rehabilitating patients, assist sports analysts and coaches in tracking and training athletes, and empower retailers to establish employee-free stores. Thanks to the advancement of deep learning and the availability of large-scale datasets, the performance of state-of-the-art human pose estimation methods has drastically improved in recent years. These approaches can accurately estimate postures in daily activities and sports. However, several challenges persist. For example, (1) it remains challenging to estimate postures that occur rarely or are entirely absent in the training data; (2) handling complex scenarios, such as crowded environments, motion blur, low-light conditions, and occlusions, is still a formidable task; (3) there is a growing need to develop efficient models that can estimate human poses in real-time or on low-power devices; (4) exploring novel applications for human pose estimation that can bring societal benefits or transform industries is an exciting avenue of research. This special issue presents a collection of high-quality papers on human pose estimation and its applications. They cover a wide range of topics, from fundamental human pose models with improved accuracy and efficiency to applications in human behavior understanding and assessment, sign language recognition, and multi-camera calibration. Through this special issue, we would like to open the path toward more discussion between practitioners and researchers in this very important yet challenging field of human pose estimation.

Guest editorial: special issue on human pose estimation and its applications

Semi-Supervised Temporal Action Localization (SS-TAL) aims to improve the generalization ability of action detectors with large-scale unlabeled videos. Albeit the recent advancement, one of the major challenges still remains: noisy pseudo labels hinder efficient learning on abundant unlabeled videos, embodied as location biases and category errors. In this paper, we dive deep into such an important but understudied dilemma. To this end, we propose a unified framework, termed Noisy Pseudo-Label Learning, to handle both location biases and category errors. Specifically, our method is featured with (1) Noisy Label Ranking to rank pseudo labels based on the semantic confidence and boundary reliability, (2) Noisy Label Filtering to address the class-imbalance problem of pseudo labels caused by category errors, (3) Noisy Label Learning to penalize inconsistent boundary predictions to achieve noise-tolerant learning for heavy location biases. As a result, our method could effectively handle the label noise problem and improve the utilization of a large amount of unlabeled videos. Extensive experiments on THUMOS14 and ActivityNet v1.3 demonstrate the effectiveness of our method. The code is available at github.com/kunnxia/NPL.

Learning from noisy pseudo labels for semi-supervised temporal action localization

* Authors: 
Belcaid, M., Leigh, J., Theriot, R., Kirshenbaum, N., Tabalba, R., Rogers, M. University of Hawai’i at Manoa, Honolulu, HI 
Johnson, A., Brown, M., Renambot, L., Long, L., Nishimoto, A., University of Illinois Chicago, Chicago, IL 
North C., Harden, J., Virginia Tech, Blacksburg, VA 
Translational software research bridges the gap between scientific innovations and practical applications, driving impactful societal advancements. However, developing such software is challenging due to interdisciplinary collaboration, technology adoption, and post-funding sustainability. This article presents the experiences and insights of the Scalable Adaptive Graphics Environment (SAGE) team, which has spent two decades developing translational, cross-disciplinary, collaboration tools to benefit computational science research. With a focus on SAGE and its next-generation iterations, we explore the inherent challenges in translational research, such as fostering cross-disciplinary collaboration, motivating technology adoption, and ensuring post-funding product sustainability. We also discuss the roles of funding agencies, policymakers, and academic institutions in promoting translational research. Although the journey is fraught with challenges, the societal impact and satisfaction derived from translational research underscore its significance in the broader scientific landscape. This article aims to encourage further conversation and the development of effective models for
translational software projects.

The process that defines domain science, computer science (CS), and sustainable translational research.

Reflecting on the Scalable Adaptive Graphics Environment Team’s 20-Year Translational Research Endeavor in Digital Collaboration Tools

Recent advances in precision medicine for treating Oropharyngeal cancer (OPC) have led to significant improvements in survival for patients. However, OPC survivors often are left with severe side effects that result from damage to vital tissues in the head and neck caused by secondary radiation received during treatment. These side effects can persist for months after treatment cessation, and diminish the patients’ quality of life. Existing guidelines for radiation therapy plans identify dose limits to individual organs with the goal to avoid or minimize permanent tissue damage and their related toxicity. However, existing tools fail to account for either the three-dimensional nature of dose distributions or the fact that critical organs often work as a system that can have multiple failure points. 
We present an unsupervised approach for stratifying patients based on three-dimensional dose distributions to multiple organs and evaluate their association with long-term patient-reported symptoms. We apply this method to predicting severe patient-reported symptoms 6 months after ceasing radiation therapy for OPC patients. For actionable interpretability, we introduce a rule-mining method that simplifies our strata into a set of dose thresholds, and demonstrate that these thresholds outperform existing dose treatment guidelines. 
Keywords: Radiation therapy; Oropharynx cancer; Head and Neck cancer; Stratification; Symptom Burden; Quality of Life

Results of a chi-squared test between covariates and membership in each set of clusters for each outcome.

Multi-Organ Spatial Stratification of 3-D Dose Distributions Improves Risk Prediction of Long-Term Self-Reported Severe Symptoms in Oropharyngeal Cancer Patients Receiving Radiotherapy: Development of a Pre-Treatment Decision Support Tool.

Motivation: Figures in biomedical papers communicate essential information with the potential to identify relevant documents in biomedical and clinical settings. However, academic search interfaces mainly search over text fields. 
Results: We describe a search system for biomedical documents that leverages image modalities and an existing index server. We integrate a problem-specific taxonomy of image modalities and image-based data into a custom search system. Our solution features a front-end interface to enhance classical document search results with image-related data, including page thumbnails, figures, captions, and image-modality information. We demonstrate the system on a subset of the CORD-19 document collection. A quantitative evaluation demonstrates higher precision and recall for biomedical document retrieval. A qualitative evaluation with domain experts further highlights our solution’s benefits to biomedical search. 
Availability and implementation: A demonstration is available at <a href="https://runachay.evl.uic.edu/scholar">https://runachay.evl.uic.edu/scholar</a>. Our code and image models can be accessed via <a href="https://github.com/uic-evl/bio-search">https://github.com/uic-evl/bio-search</a>. The dataset is continuously expanded. 
Image caption: Search interface displaying the results of a boolean query. (A) The color-coded legend displays the image modalities in the first level of the taxonomy. (B) The search bar, with a keyword box showing a boolean query, a time filter, and an image modalities filter limiting results to documents containing \textit{radiology} or \textit{fluorescence microscopy} figures. (C) The metadata of a result includes the document title, year, venue, authors, and the hits on the abstract and full text. Below is the count of figures and modalities in the document. (D) The enhanced document surrogate shows the image content, including the page thumbnail, extracted figure, subfigures and modalities, and caption. 
<a href="https://doi.org/10.1093/bioadv/vbad095">https://doi.org/10.1093/bioadv/vbad095</a>

Search interface displaying the results of a boolean query.

Enhancing biomedical search interfaces with images

Objective: Evaluate the effectiveness of machine learning tools that incorporate spatial information such as disease location and lymph node metastatic patterns-of-spread, for prediction of survival and toxicity in HPV+ oropharyngeal cancer (OPC). 
Materials & methods: 675 HPV+ OPC patients that were treated at MD Anderson Cancer Center between 2005 and 2013 with curative intent IMRT were retrospectively collected under IRB approval. Risk stratifications incorporating patient radiometric data and lymph node metastasis patterns via an anatomically-adjacent representation with hierarchical clustering were identified. These clusterings were combined into a 3-level patient stratification and included along with other known clinical features in a Cox model for predicting survival outcomes, and logistic regression for toxicity, using independent subsets for training and validation. 
Results: Four groups were identified and combined into a 3-level stratification. The inclusion of patient stratification in predictive models for 5-yr Overall survival (OS), 5-year recurrence free survival, (RFS) and Radiation associated dysphagia (RAD) consistently improved model performance measured using the area under the curve (AUC). Test set AUC improvements over models with clinical covariates, was 9 % for predicting OS, and 18 % for predicting RFS, and 7 % for predicting RAD. For models with both clinical and AJCC covariates, AUC improvement was 7 %, 9 %, and 2 % for OS, RFS, and RAD, respectively. 
Conclusion: Including data-driven patient stratifications considerably improve prognosis for survival and toxicity outcomes over the performance achieved by clinical staging and clinical covariates alone. These stratification generalize well to across cohorts, and sufficient information for reproducing these clusters is included. 
Keywords: Head & Neck cancer, Statistical and research methods, Machine learning, Radiation oncology, Health Informatics, Imaging, Risk stratification, Oropharyngeal cancer, Clustering, Radiomics

Average lymph node involvement and clinical staging categories of the 3 derived lymph node + radiomics clusters across both the training and validation datasets.

Spatially-aware clustering improves AJCC-8 risk stratification performance in oropharyngeal carcinomas

Honorable Mention in the EuroVis Best Paper Category 
Exploratory analysis of the chemical space is an important task in the field of cheminformatics. For example, in drug discovery research, chemists investigate sets of thousands of chemical compounds in order to identify novel yet structurally similar synthetic compounds to replace natural products. Manually exploring the chemical space inhabited by all possible molecules and chemical compounds is impractical, and therefore presents a challenge. To fill this gap, we present ChemoGraph, a novel visual analytics technique for interactively exploring related chemicals. In ChemoGraph, we formalize a chemical space as a hypergraph and apply novel machine-learning models to compute related chemical compounds. It uses a database to find related compounds from a known space and a machine learning model to generate new ones, which helps enlarge the known space. Moreover, ChemoGraph highlights interactive features that support users in viewing, comparing, and organizing computationally identified related chemicals. With a drug discovery usage scenario and initial expert feedback from a case study, we demonstrate the usefulness of ChemoGraph. 
Keywords: chemical space exploration, cheminformatics, multipartite graphs, data visualization 
CCS Concepts: Applied computing → Chemistry 
<iframe width="560" height="315" src="https://www.youtube.com/embed/kVpz6TDjQMw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

An overview of ChemoGraph.

ChemoGraph: Interactive Visual Exploration of the Chemical Space

Most real-world networks are both dynamic and multivariate in nature, meaning that the network is associated with various attributes and both the network structure and attributes evolve over time. Visualizing dynamic multivariate networks is of great significance to the visualization community because of their wide applications across multiple domains. However, it remains challenging because the techniques should focus on representing the network structure, attributes, and their evolution concurrently. Many real-world network analysis tasks require the concurrent usage of the three aspects of the dynamic multivariate networks. In this paper, we analyze current techniques and present a taxonomy to classify the existing visualization techniques based on three aspects: temporal encoding, topology encoding, and attribute encoding. Finally, we survey application areas and evaluation methods; and discuss challenges for future research. 
CCS Concepts Human-centered computing → Graph drawings

Overview of the taxonomy of DMVN visualization techniques along with illustrations of the categories.

The State of the Art in Visualizing Dynamic Multivariate Networks

Developing applicable clinical machine learning models is a difficult task when the data includes spatial information, for example, radiation dose distributions across adjacent organs at risk. We describe the co-design of a modeling system, DASS,to support the hybrid human-machine development and validation of predictive models for estimating long-term toxicities related to radiotherapy doses in head and neck cancer patients. Developed in collaboration with domain experts in oncology and data mining, DASS incorporates human-in-the-loop visual steering, spatial data, and explainable AI to augment domain knowledge with automatic data mining. We demonstrate DASS with the development of two practical clinical stratification models and report feedback from domain experts. Finally, we describe the design lessons learned from this collaborative experience. 
CCS Concepts 
Human-centered computing→Scientific visualization; Computing methodologies→Machine learning; Applied computing→Life and medical sciences

DASS interactive model building for head/neck cancer.

DASS Good: Explainable Data Mining of Oncology Imaging and Toxicity Data

Methane the main component of natural gas is also a potent greenhouse gas that requires robust monitoring, so that scientists, educators, and the general public can get better informed of its effect(s) on local/regional air quality and current short-term mitigations for climate change. However, current data visualizations for methane, and other scientific data, are either static 2-D graphs/plots or animated through using videos/gifs. These data visualization approaches can be limiting in that they lack interactiveness for an end user to gain exploratory insights by comparing and contrasting different areas of a dataset. This report thereby showcases how to build a functional and interactive visualization as the solution to the previously mentioned limitation. 
Keywords: Interactive Scientific Data Visualization, Climate Data Visualization, Methane Visualization

This figure shows the respective volumetric/spheric plots for 07/02/2021 at 1000 hPa. Figure 2:

Visualizing the NOAA CarbonTracker-CH4 3-D Mole Fraction (Pressure Corrected) Dataset

Background: Existing studies have reported that forest therapy utilizing sensory stimuli increases the activity of the parasympathetic nervous system and decreases the activity of the sympathetic nervous system. 
Purpose: To observe the effects of guided scenic meditation in a virtual reality environment on physiological stress. 
To perform a platform comparison by deploying the same meditative experience and testing its effectiveness using a room-scale display and a desktop computer. 
Significance: Rather than using a headset, the study uses a room-scale display - the CAVE2 Hybrid-Reality Environment - to test the effects of virtual reality environments on stress reduction. 
The platform comparison intends to showcase that if proven effective, the product is accessible, as desktop computers are far more affordable ($400 - $1000) and widely available relative to room-scale displays ($1 million+).

Fig 1. On the left is the CAVE2 version of the application. Fig 2. On the right is the desktop version of the application.

The Effects of Virtual Reality Environments on Physiological Stress: A Platform Comparison Between Room-Scale Displays and Desktop Computers (poster)

In the past, existing studies have proven the effectiveness of forest therapy as an option for stress reduction, but what happens when an individual is seeking stress relief but the outdoors is unavailable due to unpredictable weather or even a lack of accessibility? The purpose of this research project is to observe the effects of guided scenic meditation in a virtual reality environment on physiological stress. Additionally, it intends to perform a platform comparison by deploying the same meditative experience and testing its effectiveness using a room-scale display and a desktop computer, as the latter ($400 - $1000) is far more affordable and widely available relative to the former ($1 million+). Using both platforms, ten participants were asked to engage with a Unity-built meditative application that simulates the sights, sounds, and smells of a forest environment. Based on the data collected, the participants experienced a decrease in both heart rate and blood pressure, demonstrating the effectiveness of the simulation in reducing physiological stress. Similarly, according to the participants, their subjectively perceived stress levels decreased, resulting in a far more relaxed state than before. The results showcase that a virtual simulation of an environment actively contributes to stress reduction and can be used in cases when the outdoors is unavailable. The platform comparison demonstrates that the product
of the research - a guided scenic meditation application that simulates a forest - is accessible and does not require a million-dollar room-scale display in order to be effective in reducing stress.

The forest model developed in Unity.

The Effects of Virtual Reality Environments on Physiological Stress: A Platform Comparison Between Room-Scale Displays and Desktop Computers

While cities around the world are increasingly promoting streets and public spaces that prioritize pedestrians over vehicles, significant data gaps have made pedestrian mapping, analysis, and modeling challenging to carry out. Most cities, even in industrialized economies, still lack information about the location and connectivity of their sidewalks, making it difficult to implement research on pedestrian infrastructure and holding the technology industry back from developing accurate, location-based Apps for pedestrians, wheelchair users, street vendors, and other sidewalk users. To address this gap, we have designed and implemented an end-to-end open-source tool - TILE2NET - for extracting sidewalk, crosswalk, and footpath polygons from orthorectified aerial imagery using semantic segmentation. The segmentation model, trained on aerial imagery from Cambridge, MA, Washington DC, and New York City, offers the first open-source scene classification model for pedestrian infrastructure from sub-meter resolution aerial tiles, which can be used to generate planimetric sidewalk data in North American cities. TILE2NET also generates pedestrian networks from the resulting polygons, which can be used to prepare datasets for pedestrian routing applications. The work offers a low-cost and scalable data collection methodology for systematically generating sidewalk network datasets, where orthorectified aerial imagery is available, contributing to over-due efforts to equalize data opportunities for pedestrians, particularly in cities that lack the resources necessary to collect such data using more conventional methods. 
Keywords: Pedestrian network extraction, Semantic segmentation, Automated network generation, Pedestrian infrastructure

Model results showing detected sidewalk, crosswalk and footpath centerlines in a) Boston and Cambridge, b) Manhattan and parts of Brooklyn, c) Wash- ington, DC. The maps are shown at the same scale for comparison.

Mapping the walk: A scalable computer vision approach for generating sidewalk network datasets from aerial imagery

Weakly-supervised Video Anomaly Detection (WVAD) aims to detect abnormal events in videos given only video-level labels for training. Recent methods relying on multiple instance learning (MIL) and self-training achieve good performance, but they tend to focus on learning easy abnormal patterns while ignoring hard ones, e.g., unusual driving trajectory or over-speeding driving. How to detect hard anomalies is a critical but largely ignored problem in W-VAD. To tackle this challenge, we propose a novel framework, termed Abnormal Ratios guided Multi-phase Self-training (ARMS), for W-VAD. It includes a new abnormal ratio-based MIL (AR-MIL) loss and a new multi-phase self-training paradigm. The AR-MIL loss guides the learning of hard anomalies by enforcing a minimum ratio of abnormal snippets in an abnormal video and no abnormal snippets in a normal video. Our multi-phase self-training paradigm sequentially performs bootstrapping, hard anomalies mining, and adaptive self-training so as to address pseudo labeling on easy anomalies, detect hard anomalies, and setting adaptive abnormal ratios for different videos in a unified framework. Experimental results on three benchmark datasets, i.e., ShanghaiTech, UCF-Crime, and XD-Violence, show that ARMS outperforms all previous state-of-the-art methods and has a great advantage in detecting hard anomalies. 
Index Terms: Anomaly detection, weakly-supervised video anomaly detection, multiple instance learning.

Abnormal Ratios Guided Multi-Phase Self-Training for Weakly-Supervised Video Anomaly Detection

Contrails are condensation trails generated from emitted particles by aircraft engines, which perturb Earth’s radiation budget. Simulation modeling is used to interpret the formation and development of contrails. These simulations are computationally intensive and rely on high-performance computing solutions, and the contrail structures are not well defined. We propose a visual computing system to assist in defining contrails and their characteristics, as well as in the analysis of parameters for computer-generated aircraft engine simulations. The back end of our system leverages a contrail-formation criterion and clustering methods to detect contrails&rsquo; shape and evolution and identify similar simulation runs. The front-end system helps analyze contrails and their parameters across multiple simulation runs. The evaluation with domain experts shows this approach successfully aids in contrail data investigation. 
Index Terms: Scalar Field Data, Physical & Environmental Sciences, Mathematics, Feature Detection, Tracking & Transformation 
<a href="http://dx.doi.org/10.1109/TVCG.2022.3209356">http://dx.doi.org/10.1109/TVCG.2022.3209356</a>

Contrail structure groups and evolution over time.

Visual Analysis and Detection of Contrails in Aircraft Engine Simulations

It has often been observed, informally and in academic literature, that speech directed toward individuals who are hard of hearing is naturally modified by talkers in an attempt to accommodate the listeners. These modifications take numerous forms, from the easily expressed &ldquo;louder&rdquo; and/or &ldquo;slower&rdquo; to the highly technical &ldquo;vowel space expansion&rdquo; and many others. There are also speech modifications made in numerous other circumstances, not only for hard of hearing listeners, but for listeners in noisy backgrounds, second language listeners, and many others, known under the collective (is sometimes too-broad) umbrella as clear speech. There is, however, an asymmetry inherent to all these talker-listener scenarios, which is that talkers have only the broadest guidance at best (&ldquo;Louder!&rdquo; &ldquo;Slower!&rdquo;) from their listener partners, which they may simply ignore. 
With an interest in developing new and practical interventions for the elderly and hard of hearing, this dissertation takes as its research questions: First, the feasibility of the production of a signal processing system which allows real-time modification of audio signals. Second, the tolerance and effectiveness of partially emulating clear speech, by using signal processing techniques to modify (specifically, to slow) the rate of received speech signals. And finally, the effectiveness of placing the control of speech rate directly into the hands of listeners, a capability that has never been possible prior to electronic mediation. 
This dissertation is organized as follows: 
Chapter One is a survey of both the circumstances which prompt these spontaneous speech modifications, as well as of the details of the modifications which are made. 
Chapter Two presents an overview of the various methods of changing the rate of speech including the simple but ineffective technique of re-sampling, as well as several frequency domain techniques. These systems were intended for use on pre-existing, rather than live, signals. 
Chapter Three presents the details of several incarnations of this real-time system, including a single-talker system implemented on a laptop computer, presented in [1] and used directly (with two laptops for two talkers) in the study presented in [3]; a two-talker wireless system using Android smartphones and local network connectivity presented at [2]; and an unpublished system using a central laptop base station, to which Android smart phones could
connect even when hundreds of miles apart. This chapter contains details required for the implementation of both live signals, and live control of the playback rate of those signals. Finally, this system includes the design details of systems used in the three user studies of Chapters Five, Six, and Seven. 
Chapter Four presents a user study designed to examine the effects of using real-time temporal modification software in a conversational setting, previously published in [3]. The key element of the study design was the use of a scored Diapix test [7] which was designed to elicit spontaneous conversational speech between two study participants without the involvement of or prompting by researchers. The main results of this study are (i) that at modest amounts of stretching (40% additional playback time, with lengthy silences not stretched) the speech patterns of subjects are not changed to a statistically significant degree, and (ii) under those same circumstances, there is no statistical evidence of changes in performance on the Diapix test. 
Chapter Five presents the first of three user studies focusing on the little-studied subject of the interaction of user choice and user control over received speech rates in adverse conditions. This study was based off the preexisting QuickSIN test [8], designed to test and characterize a listener&rsquo;s ability to understand short sentences of speech played against a background of noise. In the first part of this study, users were asked to listen to several short sentences each played against multiple levels of background noise. For each level of background noise, the subjects were asked to use a computer interface to set the rate of the speech to whatever rate they believed was most helpful in understanding the foreground speech. (The background babble did not change rate.) These preferred expansion rates were recorded for later use and analysis. In the second part of the study, subjects were asked to listen to and immediately repeat back sentences at those same noise conditions, both with speech modified to their preferred expansion rates, and unmodified. The subjects’ repetition of those sentences was also recorded for analysis. 
Our results included statistically significant differences in received speech rate at opposite ends of the noise condition, with subjects requesting slower speech in the presence of increase noise; as well as an overall belief that slowing speech was beneficial to subjects’ performance. However, analysis of the subjects&rsquo; ability to repeat was in fact mildly degraded at moderate to high noise conditions and was not improved at any noise condition. This work was published at Interspeech 2018 [4]. 
Chapter Six presents the second of the three user control studies. In this study, non-native students with recent TOEFL scores were recruited as test subjects. After familiarization with a graphical interface, test subjects were asked to participate in six comprehension tasks, each of which was a lengthy (several minutes long) audio track of a TOEFL test to which they had not previously been exposed. During three of these tasks, the subjects were given
the ability to change the rate of playback as they saw fit, and asked to do so in whatever fashion they believed would best help them understand the audio passage. The other three comprehension tasks were presented unmodified, without a control interface. After each task, the subjects were immediately given a short multiple choice quiz about the contents of the previous audio passage. Subjects&rsquo; behavior (i.e., the rate changes specified) and quiz responses were collected and analyzed. 
Our analysis of the data shows that a considerable majority of the subjects used speech slowing in all three of their trials (roughly 2/3) and an overwhelming majority used the technique in at least one of their trials (80%), but did not reveal a statistically significant improvement or degradation in listening comprehension. There is also a weak correlation between lower TOEFL scores and more slowing. This work was published at Interspeech 2019 [5]. In the work described above, the algorithms employed stretch speech uniformly - if set to expand an audio track by 40%, all sounds, including all parts of speech all background noises, and even silences, are expanded by the same rate. For large expansion factors, the speech so produced sounds somewhat unnatural, because this is not how talkers spontaneously produce slow speech. With this in mind, Chapter Seven returns to the topic of speech in noise, but puts even more precise control in the hands of test subjects: A relatively simple neural network was developed to classify individual phonemes into six broad phonemic categories within an audio track. A study was also designed around this tool which asked subjects to listen to short sentences in noise, and then expand or contract the audio only for particular phonemes (e.g., first modify vowels, then modify fricatives, etc.) Following this, all modifications were applied simultaneously, and subjects were asked to listen to modified and unmodified/control sentences in three levels
of background noise and asked to transcribe the sentences into a computer interface. 
Our analysis found either no statistically significant improvement or degradation (at one noise level) or statistically significant degradation (at two noise levels) of intelligibility while using this technique. 
Finally, Chapter Eight presents overall conclusions, with additional discussion of the entire work and of directions for future studies.

Unwrapped Schematic of Cochlea

Audio Expansion Techniques For Real-Time Communication And Intelligibility Enhancement

Cluster schedulers are crucial in high-performance computing (HPC). They determine when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with speciﬁc HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backﬁlling. An efﬁcient training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a speciﬁc scheduling objective given by the system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. We implement DRAS into an HPC scheduling platform called CQGym. CQGym provides a common platform allowing users to ﬂexibly evaluate DRAS and other scheduling methods such as heuristic and optimization methods. The experiments using CQGym with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 50%. 
Index Terms - High-performance computing, cluster scheduling, deep reinforcement learning, job starvation, backﬁlling, resource reservation, OpenAI Gym 
<a href="https://ieeexplore.ieee.org/abstract/document/9894371">https://ieeexplore.ieee.org/abstract/document/9894371</a>

DRAS overview. DRAS agent (at the bottom) represents the scheduling agent; the environment (at the top) comprises the rest of the system, including job wait queue and HPC cluster.

DRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing

BACKGROUND: Acute lung injury and the acute respiratory distress syndrome (ARDS) are characterized by pulmonary inflammation, reduced endothelial barrier integrity and filling of the alveolar space with protein-rich edema fluid and infiltrating leukocytes. Animal models are critical to uncovering the pathologic mechanisms of this devastating syndrome. Intravital imaging of the intact lung via two photon intravital microscopy has proven a valuable method to investigate lung injury in small rodent models through characterization of inflammatory cells and vascular changes in real-time. However, respiratory motion complicates the analysis of these time series images and requires selective data extraction to stabilize the image. Consequently, analysis of individual alveoli may not provide a complete picture of the integrated mechanical, vascular and inflammatory processes occurring simultaneously in the intact lung. To address these challenges, we developed a web browser-based visualization application named Alveolus Analysis to process, analyze and graphically display intravital lung microscopy data. RESULTS: The designed tool takes raw temporal image data as input, performs image preprocessing and feature extraction offline, and visualizes the extracted information in a web browser-based interface. The interface allows users to explore multiple experiments in three panels corresponding to different levels of detail: summary statistics of alveolar/neutrophil behavior, characterization of alveolar dynamics including lung edema and inflammatory cells at specific time points, and cross-experiment analysis. We performed a case study on the utility of the visualization with two members or our research team and they found the tool useful because of its ability to preprocess data consistently and visualize information in a digestible and informative format. CONCLUSIONS: Application of our software tool, Alveolus Analysis, to intravital lung microscopy data has the potential to enhance the information gained from these experiments and provide new insights into the pathologic mechanisms of inflammatory lung injury. 
<a href="https://github.com/uic-evl/AlveolusAnalysis">Visits the code repository.</a>

Alveolus Analysis: A Web Browser-based Tool to Analyze Lung Intravital Microscopy

Inaccessible urban infrastructure creates and reinforces systemic exclusion of people with disabilities and impacts public health, physical activity, and quality of life for all. To improve the design of our cities and to enable more equitable policies and location-centric
technology designs, we need new data collection techniques, data standards, and accessibility-infused analytic tools and interactive maps focused on the quality, safety, and accessibility of pathways, transit ecosystems, and buildings. In this workshop, we bring together leading experts in human mobility, urban design, disability, and accessible computing to discuss pressing urban access challenges across the world and brainstorm solutions. We invite contributions from practitioners, transit officials, disability advocates,
and researchers.

The Future of Urban Accessibility for People with Disabilities: Data Collection, Analytics, Policy, and Tools

Background. Personalized radiotherapy can improve treatment outcomes of head and neck cancer (HNC) patients, where currently a &rsquo;one-dose-fits-all&rsquo; approach is the standard. The aim was to establish individualized outcome prediction based on multi-institutional international &ldquo;big data&rdquo; to facilitate risk-based stratification of HNC patients. 
Methods. The data of 4611 HNC radiotherapy patients from three academic cancer centers was split into 4 cohorts: a training (n=2241), independent test (n=786), and external validation cohorts 1 (n=1087) and 2 (n=497). Tumor- and patient-related clinical variables were considered in a machine learning pipeline to predict overall survival (primary endpoint) and local and regional tumor control (secondary endpoints); serially, imaging features were considered for optional model improvement. Finally, patients were stratified into high, intermediate, and low risk groups. 
Results. Performance score, AJCC8th stage, pack-years, and Age were identified as predictors for overall survival, demonstrating good performance in both the training cohort (c-index=0.72 [95% CI, 0.66-0.77]) and in all three validation cohorts (c-indices: 0.76 [0.69-0.83], 0.73 [0.68- 0.77], and 0.75 [0.68-0.80]). Excellent stratification of HNC patients into high, intermediate, and low mortality risk was achieved; with 5-year overall survival rates of 17-46% for the high-risk group compared to 92-98% for the low-risk group. The addition of morphological image feature further improved the performance (c-index=0.73 [0.64-0.81]). These models are integrated in a clinic-ready interactive web-interface: <a href="https://uic-evl.github.io/hnc-predictor/">https://uic-evl.github.io/hnc-predictor/</a>. 
Conclusions. Robust model-based prediction was able to stratify HNC patients in distinct high, intermediate and low mortality risk groups. This can effectively be capitalized for personalized radiotherapy, e.g., for tumor radiation dose escalation/de-escalation. 
See <a href+"https://www.sciencedirect.com/science/article/pii/S095980492200805X">Science Direct</a> for more information.

Head and Neck Cancer Predictive Risk Estimator to Determine Control and Therapeutic Outcomes of Radiotherapy (HNCPREDICTOR)

Sidewalks play a pivotal role in urban mobility of everyday life. Ideally, sidewalks provide a safe walkway for pedestrians, link public transportation facilities, and equip people with routing and navigation services. However, there is a scarcity of open sidewalk data, which not only impacts the accessibility and walkability of cities but also limits policymakers in generating insightful measures to improve the current state of pedestrian facilities. As one of the most famous crowdsourced data repositories, OpenStreetMap (OSM) could aid the lack of open sidewalk data to a large extent. However, completeness and quality of OSM data have long been a major issue. In this paper, we offer a preliminary study on the availability and trustworthiness of OSM sidewalk data. First, we compare OSM
sidewalk data coverage in over 50 major cities in the United States. Then, we select three major cities (Seattle, Chicago, and New York City) to further analyze the completeness of sidewalk data and its features, and to compute a trustworthiness index leveraging historical OSM sidewalk data.

Coverage of sidewalks and roads with sidewalk info in Chicago, Seattle, and New York City (from left to right). Seattle OSM has comparably higher percentage of sidewalk geometries and roads with sidewalk information mapped than the other two cities.

Crowdsourcing and Sidewalk Data: A Preliminary Study on the Trustworthiness of OpenStreetMap Data in the US

Contrails are condensation trails generated from emitted particles by aircraft engines, which perturb Earth&rsquo;s radiation budget. Simulation modeling is used to interpret the formation and development of contrails. These simulations are computationally intensive and rely on high-performance computing solutions, and the contrail structures are not well defined. We propose a visual computing system to assist in defining contrails and their characteristics, as well as in the analysis of parameters for computer-generated aircraft engine simulations. The back-end of our system leverages a contrail-formation criterion and clustering methods to detect contrails&rsquo; shape and evolution and identify similar simulation runs. The front-end system helps analyze contrails and their parameters across multiple simulation runs. The evaluation with domain experts shows this approach successfully aids in contrail data investigation. 
Index Terms: Scalar Field Data, Physical & Environmental Sciences, Mathematics, Feature Detection, Tracking & Transformation 
 <iframe width="420" height="315"src="https://www.youtube.com/embed/mFllS2wQW8I">
</iframe>

Publications

Parameterized Algorithms for Non-uniform All-to-all

mEnergy: Development of a Novel mHealth Multi-device Platform for Near Real-time Remote Assessment of Energy Disorders

CQSim+: Symbiotic Simulation for Multi-Resource Scheduling in High-Performance Computing

Evaluating SZ3 Compressor Performance on High Energy Physics Data (poster)

Energy Efficiency of LLM Inference Across Various AI Accelerators (poster)

LASSI-EE: Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes (poster)

Contact

Publications

Parameterized Algorithms for Non-uniform All-to-all

mEnergy: Development of a Novel mHealth Multi-device Platform for Near Real-time Remote Assessment of Energy Disorders

CQSim+: Symbiotic Simulation for Multi-Resource Scheduling in High-Performance Computing

﻿﻿Evaluating SZ3 Compressor Performance on High Energy Physics Data (poster)

Energy Efficiency of LLM Inference Across Various AI Accelerators (poster)

LASSI-EE: Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes (poster)

Contact

Evaluating SZ3 Compressor Performance on High Energy Physics Data (poster)