Post-treatment decision-making is a process that depends on longitudinal studies of patient cohorts, in which identifying the risk of adverse treatment outcomes is critical to improving personalized care. This process benefits from visual analytics because it can overcome many of the challenges that arise with complex patient cohort data. Visual analytics can help to interpret treatment progressions and outcomes, and stratify cohorts by risk categories, which are crucial for improving treatment decision-making. However, post-treatment cohort analysis uses large-scale, heterogeneous, temporal, multivariate datasets with associated
attributes and missing values. Visualization needs to provide scalable, effective methods for cohort analysis at different levels of detail that can uncover patterns and associations among patient attributes that correspond to negative treatment outcomes. Moreover, posttreatment care planning relies on computational cohort data modeling and, as a result, uses both objective and subjective evidence, namely, the clinician’s interpretation of the modeling results. Consequently, cohort modeling and analysis depend on collaborations between clinicians and data modelers. Therefore, visual analytics solutions need to facilitate these collaborations and the interpretation and evaluation of modeling results in a clinical context. 
I am running a few minutes late; my previous meeting is running over. This dissertation explores visual analytics techniques for cohort modeling and analysis
and applies these techniques to post-treatment decision-making. This work addresses the challenges identified above by designing, developing, and evaluating four application-specific visualization systems in collaboration with clinical researchers and data modelers. I first identified the design requirements for a family of cohort modeling problems in cancer symptom and digital biomarker research. Next, I design several systems that integrate unsupervised modeling for the computational back-end and data visualization for the front-end. I propose novel, custom visual encodings for multivariate temporal cohorts that enable iterative risk assessment across cohort stratifications. A first system, OpenDBM, uses visual analytics for behavioral risk assessment in digital biomarker research, using cohorts with hundreds of modeled attributes, and it was designed for the open-source community. This work proposes a novel encoding that aggregates multivariate, spatial, and non-spatial temporal attributes on anatomical locations to explain behavioral biomarkers. A second system, THALIS, shifts
the focus to clinician-modeler collaborations in head and neck cancer cohort modeling and to a multi-stage patient monitoring process, namely, during and post-treatment. This system uses scalable visual encodings to interpret attribute associations and introduces a new encoding for evaluating patient outcomes in multivariate, multi-stage time series. A third system, Roses, builds on the previous work, using custom visualizations for the interpretation and evaluation of outcome risk predictions, this time accommodating configurable analytical workflows for clinicians and modelers. The system introduces a visual encoding
to summarize multi-stage networks, with temporal nodes, which helps to evaluate patterns and associations in modeled outcome risk components. A fourth system, L-VISP, explores visual analytics for understanding and assessing black-box models in cohort risk prediction, with an emphasis on the design requirements for data modeler activities. To support model evaluation, the system visualizes results for machine-derived (cluster) or user-specified cohort stratifications and introduces custom encodings for weighted associations in multivariate attributes. Together, these systems contribute to data visualization and modeling solutions for the challenges that data modelers and clinicians face during collaborations. 
Patient records were used for the cancer research projects. These records contained demographic and diagnostic details, as well as longitudinal symptom ratings. The records were anonymized by our clinical expert collaborators and stored on a private institution cloud. Access was limited and given by our collaborators on a case-by-case basis. For the digital biomarker project, the collaborators provided longitudinal biomarker records. These records were extracted from actor, not patient, videos using a feature-extraction toolkit and stored in a private cloud. There were no personal identifiers in these data, and no research involving human subjects was conducted in this dissertation.

Simplified framework for patient cohort post-treatment decision-making and dissertation contributions (i.e., OpenDBM, THALIS, Roses, and L-VISP) within the framework.

Patient Cohort Visual Analytics for Post-Treatment Care

High-performance computing (HPC) systems are essential for scientific discovery and engineering innovation. However, their growing power demands pose significant challenges, particularly as systems scale to the exascale level. Prior uncore frequency tuning studies have primarily focused on conventional HPC workloads running on CPU-only systems. As HPC advances toward heterogeneous computing, integrating diverse GPU workloads on heterogeneous CPU-GPU systems, it becomes imperative to revisit and enhance uncore scaling. Our investigation reveals that uncore frequency scales down only when CPU power approaches its thermal design power (TDP), which is rare in GPU-dominant applications. As a result, modern computing systems experience unnecessary power waste. In this study, we present MAGUS, a user-transparent uncore frequency scaling runtime for heterogeneous computing. MAGUS dynamically adjusts uncore frequencies according to distinct application execution phases, effectively minimizing power waste caused by consistently using maximum uncore frequencies. Our design incorporates several key techniques, including real-time monitoring and prediction of memory accesses, intelligent handling of frequent phase transitions, and leveraging vendor-provided power management features. We evaluate MAGUS with various GPU benchmarks and applications on multiple heterogeneous systems with different CPU and GPU architectures. Experimental results demonstrate that MAGUS achieves up to 27% energy savings compared to the default settings, while maintaining a performance loss of less than 5% and an overhead of under 1%. 
Keywords: GPU workloads, heterogeneous CPU-GPU systems, uncore frequency scaling, energy efficiency, performance-power trade-offs

Minimizing Power Waste in Heterogenous Computing via Adaptive Uncore Scaling

We present the design and development of a crowdsourced, data-driven approach to quantifying the flavor of specific foods. Our approach leverages an interactive smartphone app with a client-server architecture and supports future stratified analysis based on dietary and demographic factors. This approach has the potential to document taste loss and to empower patients to describe taste loss symptoms in a more precise, standardized language. 
Index Terms: flavor, taste loss, head and neck cancer

FlavorCharter: Development of a Smartphone App for Quantifying Food Flavor

In the precision medicine paradigm, oncological treatment leverages complex ensemble datasets of similar patients to estimate the outcomes for a current patient. A key challenge is developing and deploying easy-to-understand AI predictive models for the outcomes of a specific patient, based on patient data from multiple
institutions. We describe the lessons learned from the development and deployment of an interactive dashboard to support the analysis of individual head and neck cancer patient outcomes based on cohort data. As required by the project, the dashboard design aims to handle a large client base. The dashboard combines an AI solution with a multi-view interface featuring domain-specific plots to facilitate the visual analysis of patient outcomes and to quickly stratify new patients into risk groups. A year after the successful public deployment of the dashboard, we evaluate it with clinician domain experts. We report the feedback and we reflect on the lessons learned through this experience. 
Index Terms: VA-machine intelligence for healthcare data visualization, Human-centered AI for health decision-making, Dashboard, Risk Stratification, Precision Medicine

Interactive Oncological Risk Estimator

Lessons from the Development and Deployment of an Interactive Oncological Risk Estimator

MRS is a non-invasive way of gaining information about cellular metabolism and can be used to study neurological and metabolic disorders. However, the clinical utility is limited by low signal quality, overlapping spectral peaks, and the inability to interpret and compare metabolite data. We developed an interactive visual analysis system that integrates methods for filtering, quality measurement, and comparative summaries of MRS signals, enabling researchers to more effectively evaluate and interpret spectral data. This will allow researchers to perform a more accurate evaluation of spectral data, which will help them understand the molecular processes behind the complex neurological and metabolic processes in greater depth. 
Index Terms:</sstrong> Glyph-based Visualization, Quality Metrics, Multimodal Visualization.

Dual MRS Visualization Interface. (A) Metabolite selection panel; (B) Circular plot showing ratio (radius) and quality (saturation) with summary slices; (C) Rectangular plot displaying ratio (width), quality (height), and summary heatmap.

From Peaks to Patterns

The visualization and analysis of street and pedestrian networks are important to various domain experts, including urban planners, climate researchers, and health experts. This has led to the development of new techniques for street and pedestrian network visualization, expanding possibilities for effective data presentation and interpretation. Despite their increasing adoption, there is no established design framework to guide the creation of these visualizations while addressing the diverse requirements of various domains. When exploring a feature of interest, domain experts often need to transform, integrate, and visualize a combination of thematic data (e.g., demographic, socioeconomic, pollution) and physical data (e.g., zip codes, street networks), often spanning multiple spatial and temporal scales. This not only complicates the process of visual data exploration and system implementation for developers but also creates significant entry barriers for experts who lack a background in programming. With this in mind, in this paper, we reviewed 45 studies utilizing street-overlaid visualizations to understand how they are applied in practice. Through qualitative coding of these visualizations, we analyzed three key aspects of street and pedestrian network visualization usage: their analytical purposes, the visualization approaches employed, and the data sources used in their creation. Building on this design space, we introduce StreetWeave, a declarative grammar for designing custom visualizations of multivariate spatial network data across multiple resolutions. We demonstrate how StreetWeave can be used to create various street-overlaid visualizations, enabling effective exploration and analysis of spatial data. StreetWeave is available at <a href="http://urbantk.org/streetweave">urbantk.org/streetweave</a>. 
Index Terms: Urban visual analytics, design space, street-overlaid visualization, visualization grammar.

StreetWeave: A Declarative Grammar for Street-Overlaid Visualization of Multivariate Data

The three-dimensional (3D) organization of chromatin plays a critical role in regulating gene expression and genomic processes like DNA replication, repair, and genome stability. Although these processes occur at the individual-cell level, most chromatin structure data are derived from population-averaged assays, such as Hi-C, obscuring the heterogeneity of single-cell conformations. To address this limitation, we developed a polymer physics-based modelling framework, the Sequential Bayesian Inference Framework (sBIF), that deconvolutes bulk Hi-C data to reconstruct single-cell 3D chromatin conformations. To support a broader use of sBIF, we created ChromPolymerDB, a publicly accessible, high-resolution database of single-cell chromatin structures inferred by sBIF. The database contains ~108 reconstructed 5 kb-resolution single cell structures, spanning over 60,000 genomic loci across 50 human cell types and experimental conditions. ChromPolymerDB features an interactive web interface with tools for 3D structural analysis and multi-omics integration. Users can explore associations between chromatin conformation and gene expression, epigenetic modifications, and regulatory elements. The platform also supports comparative analyses to identify structural changes across cell types, developmental stages, or disease contexts. ChromPolymerDB offers a unique resource for researchers studying the relationship between genome architecture and gene regulation, and for advancing comparative 3D genomics. ChromPolymerDB is available online at <a href="https://chrompolymerdb.bme.uic.edu/">https://chrompolymerdb.bme.uic.edu/</a>.

Graphical Abstract

ChromPolymerDB: A High-Resolution Database of Single-Cell 3D Chromatin Structures for Functional Genomics

The rapidly expanding data generated by advanced methods in biology are creating enormous challenges at the interface between humans and biological data that can be represented and analyzed visually. To meet these challenges, a blend of methodology from the visualization, bioinformatics, and biology domains is required. This methodology encompasses perception and design knowledge, algorithm design, techniques for analyzing and visualizing Big Data, statistics, and specific domain knowledge for different application problems. Powerful and integrative visualization methods are necessary, as well as methods for visually communicating the outcome of computational analytical methods and findings. This special issue aims to highlight new challenges in biological data visualization (BioVis) and State-of-the-Art solutions created by the vibrant BioVis community.

Recent Trends and Challenges in Biological Data Visualization (CGA BioVis Special Issue Guest Editosr&rsquo; Introduction)

Anxiety is one of the most widespread mental health issues, affecting nearly 30% of U.S. adults. In 2024, 43% reported feeling more anxious than the year before.
Traditional treatments often require clinical settings, which can be costly, inaccessible, or intimidating. There&rsquo;s a growing need for self-guided, engaging
tools that support emotional regulation in everyday environments. 
VR is emerging as a powerful tool in clinical anxiety treatment. At Princeton House, VR sessions led to a 35% reduction in anxiety, and CUCARD uses VR
for exposure therapy in controlled environments like classrooms. However, these tools rely on professional oversight and offer limited user agency. &ldquo;You Can Grow Here&rdquo; builds on this research by offering a non-clinical, self-directed VR experience - making emotional support more personal, immersive, and accessible. 
Our project followed an iterative design process centered on user feedback and therapeutic impact. Through frequent testing, we refined key elements such as
lighting, sound balance, and interaction flow to enhance emotional engagement. Demonstrations leveraging improv theatre were used to prototype the user journey, ensuring each interaction supported our well-being goals. Weekly meetings and collaborative tools like Unity, Figma, and Discord enabled efficient teamwork. Continuous evaluation ensured alignment with our therapeutic objectives, fostering a meaningful and emotionally restorative experience.

Overview image of the island with key locations of scenes numbered.

You Can Grow Here A Theraputic VR Journey for Anxiety Management

&ldquo;You Can Grow Here&rdquo; is an immersive VR experience developed for the CAVE2&trade: environment, aligning with the UN Sustainable Development Goal of Good Health and Well-Being. In response to the mental health challenges intensified by the COVID-19 pandemic, the project explores how interactive storytelling, ambient sound, and 3D typography can support emotional reflection and teach anxiety coping strategies. Built in Unity with custom assets from Blender and Maya, the experience differs from most clinical VR programs, allowing users to independently explore emotions, manage anxiety, and practice evidence-based calming techniques within a safe, narrative-driven space that builds emotional resilience. 
CCS Concepts: Human-centered computing - Interaction design process and methods; Interaction design theory, concepts, and paradigms.

A glimpse of the final scene, where the rainbow represents peace, renewal, and emotional grounding.

You Can Grow Here A Therapeutic VR Journey for Anxiety Management

Advanced cyberinfrastructure (CI) is undergoing disruptive changes in system architectures and application workloads. The landscape of cyberinfrastructure workloads is rapidly expanding beyond traditional computational simulations to include a hybrid mix of applications. CI facilities now host diverse high-performance systems with heterogeneous configurations, leading to a complex mix of computing, memory, and storage components. Existing CI management methods, which are heavily heuristic or manual-based, struggle with these evolving challenges. This project addresses the complex challenges of CI resource management by integrating artificial intelligence (AI) technologies with human expertise. The proposed AIMCI framework transitions from managing isolated single clusters to coordinating facility-wide management, orchestrating the entire facility as a unified pool of diverse resources for a broad spectrum of applications with various resource requirements.

Manual, static, single-cluster resource amangement - - Automated, dynamic, facility-wide resource management

OAC Core: AIMCI: Artificial Intelligence for Managing Cyberinfrastructure

MPI_Alltoallv generalizes the uniform all-to-all communication (MPI_Alltoall) by enabling the exchange of data-blocks of varied sizes among processes. This function plays a crucial role in facilitating many computational tasks, such as FFT calculations and graph mining operations. Popular MPI libraries, such as MPICH and OpenMPI, implement MPI_Alltoall using a combination of linear and logarithmic algorithms. However, MPI_Alltoallv typically relies only on variations of linear algorithms, missing the benefits of logarithmic approaches. Furthermore, current algorithms also overlook the intricacies of modern HPC system architectures, such as the significant performance gap between intra-node (local) and inter-node (global) communication. To address these problems, this paper presents two novel algorithms: Parameterized Logarithmic non-uniform All-to-all (ParLogNa) and Parameterized Linear nonuniform All-to-all (ParLinNa). ParLogNa is a tunable logarithmic time algorithm for non-uniform all-to-all, and ParLinNa is a hierarchical and tunable near-linear-time algorithm for non-uniform all-to-all. These algorithms efficiently address the trade-off between bandwidth maximization and latency minimization that existing implementations struggle to optimize. We show a performance improvement over the state-of-the-art implementations by factors of 42x and 138x on Polaris and Fugaku, respectively. 
<a href="https://doi.org/10.1145/3731545.3731590">https://doi.org/10.1145/3731545.3731590</a>

An example of ParLinNa when P = 15, N = 3, r = 2 and Q = 5.

Parameterized Algorithms for Non-uniform All-to-all

We present the development, architecture, and features of a new multi-device mHealth software platform to support near real-time remote monitoring of metabolic health and timely intervention in the treatment and survivorship of cancer patients. Our platform, mEnergy, leverages a human-centered design process, and integrates in a unified, web-based framework consumer-grade hardware - Fitbit wearable sensor devices, smartphones, and Withings smart scales. mEnergy can
aid oncologists in identifying early indicators of muscle-wasting (sarcopenia) due to sleep disturbance, insufficient weight recovery, or reduced/limited activity. The platform aims for a smooth transition into clinical practice and increased adherence to evidence-based recommendations, in particular in underserved geographical areas. This toxicity-surveillance approach based on mHealth technologies can improve treatment outcomes, quality of life, and survivorship.

Visual analysis of multi-device health data.

mEnergy: Development of a Novel mHealth Multi-device Platform for Near Real-time Remote Assessment of Energy Disorders

Efficient job scheduling is crucial in high-performance computing (HPC), balancing user demands for quick job turnaround with facility goals for high resource utilization. Traditional scheduling requires users to specify a system at job submission, which can lead to inefficiencies. A unified scheduling approach, viewing the resources within a computing facility as an integrated pool, promises improved resource use and reduced job wait times. This paper presents CQSim+, an open-source, discrete event-driven simulator tailored for symbiotic multi-resource scheduling. CQSim+ supports dynamic simulation by continuously integrating real-time data from job schedulers, enabling adaptive scheduling based on the system&rsquo;s current state. Through extensive experimentation, we demonstrate CQSim+&rsquo;s ability to enhance resource utilization and decrease job wait times in both homogeneous and heterogeneous HPC environments. 
Additionally, we present a case study that coordinates job scheduling between two production systems, illustrating how CQSim+ can effectively optimize job scheduling across distinct systems. 
CCS Concepts: Computing methodologies - Modeling methodologies; Discrete-event simulation; Real-time simulation; Simulation tools; Social and professional topics - System management 
Keywords: Multi-resource scheduling, Simulation tool, Resource management systems, High-performance computing 
<a href="https://doi.org/10.1145/3726301.3728404">https://doi.org/10.1145/3726301.3728404</a>

A time diagram of CQSim+ performing meta-scheduling for multiple systems.

CQSim+: Symbiotic Simulation for Multi-Resource Scheduling in High-Performance Computing

In high-performance computing (HPC), modern supercomputers typically provide exclusive computing resources to user applications. Nevertheless, the interconnect network is a shared resource for both inter-node communication and across-node I/O access, among co-running workloads, leading to inevitable network interference. In this study, we develop MFNetSim, a multi-fidelity modeling framework that enables simulation of multi-traffic simultaneously over the interconnect network, including inter-process communication and I/O traffic. By combining different levels of abstraction, MFNetSim can efficiently co-model the communication and I/O traffic occurring on HPC systems equipped with flash-based storage. We conduct simulation studies of hybrid workloads composed of traditional HPC applications and emerging ML applications on a 1,056-node Dragonfly system with various configurations. Our analysis provides various observations regarding how network interference affects communication and I/O traffic. 
<a href="https://doi.org/10.1145/3729424">https://doi.org/10.1145/3729424</a>

An illustration of workload replay module.

MFNetSim: A Multi-Fidelity Network Simulation Framework for Multi-Trafic Modeling of Dragonfly Systems

EVL/UIC researchers collaborate with the Discovery Partners Institute, University of Illinois System, to develop a computational framework for the visualization and analysis of transportation, weather, and public health data.

Developing an Open Computational Framework for Decision Support Across Transportation, Weather, and Public Health

Datalog, a declarative logic programming language that operates bottom-up, has experienced increasing popularity due to its natural handling of recursive queries. Its applications span diverse fields, including graph mining, program analysis, deductive databases, and neuro-symbolic reasoning. While Datalog shares similarities with SQL in using relational algebra kernels, it uniquely employs iterative execution until reaching a fixed point to support recursion. Current Datalog engines like SLOG, LogicBlox, and Soufflé work well with multi-core and multi-threaded systems, but none have yet tackled multi-node, multi-GPU architectures. Our research addresses this gap by developing the first multi-GPU, multinode Datalog engine. This advancement is particularly for high-performance computing (HPC) systems, which typically feature multiple GPUs per node. Our implementation combines MPI for inter-node communication with CUDA for GPU parallelization, enabling the processing of massive datasets in real time. We have created novel data-parallel implementations of core relational algebra operations (join), while also optimizing deduplication and tuple materialization. To handle iterative execution, we have developed two novel GPU-accelerated methods for non-uniform all-to-all data exchange. Evaluating on Argonne National Lab&rsquo;s Polaris supercomputer demonstrated our engine’s effectiveness, achieving performance improvements of up to 32× against state-of-the-art multi-node Datalog engine. 
Keywords: Datalog, Multi-GPU, Analytic Databases 
<a href="https://doi.org/10.1145/3721145.3730431">https://doi.org/10.1145/3721145.3730431</a>

First iteration of semi-naïve evaluation with local aggregation on Weakly Connected Component (WCC) query using mnmgDatalog.

Multi-Node Multi-GPU Datalog

Motivation: 
- The Large Hadron Collider (LHC) has created hundreds of petabytes of data. The High-Luminosity LHC (HL-LHC) will produce orders of magnitude more.
The HL-LHC is a major upgrade to the LHC, aiming to be operational by the end of 2030. It will produce more data in one year than the LHC produced during its first 10 years of operation. 
- Significant R&D efforts are needed to handle the incoming tide of data. Without using new technologies, the data volume of the HL-LHC will rapidly outgrow CERN&rsquo;s compute and storage capacities. This means everything is on the table, even historically disfavored methods like lossy compression.

﻿﻿Evaluating SZ3 Compressor Performance on High Energy Physics Data (poster)

In recent years, numerous hardware accelerators have been developed to meet the rising demand for machine learning (ML) workloads, and Large Language Models inference in particular. GPUs are currently the standard for ML training and inference. However, they require substantial data movements that hurt performance and increase power consumption, making systems extremely energy-intensive. In response, many companies, including Intel, AMD, and Google, as well as numerous startups such as Groq, SambaNova, Cerebras, and Graphcore, have introduced specialized accelerators for ML workloads that leverage a dataflow design, which aims to reduce data movement and thus improve both performance and power consumption. This article presents a comparative analysis of the performance and energy efficiency of various AI accelerators and GPUs for large language model (LLM) inference, using popular open-source models evaluated on both synthetic and real-world datasets.

Energy Efficiency of LLM Inference Across Various AI Accelerators (poster)

While large language models (LLMs) are increasingly used for generating parallel scientific code, most current efforts emphasize functional correctness, often overlooking performance and energy considerations. In this work, we propose LASSI-EE, an automated LLM-based refactoring framework that generates energy-efficient parallel code on a target parallel system for a given parallel code as input. 
Through a multi-stage, iterative pipeline process, LASSI-EE achieved an average energy reduction of 47% across 85% of the 20 HeCBench benchmarks tested on NVIDIA A100 GPUs. Our findings demonstrate the broader potential of LLMs, not only for generating correct code but also for enabling energy-aware programming. We also address key insights and limitations within the framework, offering valuable guidance for future improvements.

LASSI-EE: Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes (poster)

HEP workflows often consist of complex DAGs with streaming job arrivals that challenge traditional static scheduling approaches. We introduce a dynamic pilot job scheduling model that allocates HPC resources on demand when queued DAG jobs reach a configurable threshold. The simulation framework models interactions between the trace parser, DAG manager, pilot job manager, and HPC resources. The system improves scheduling responsiveness and resource utilization by reserving nodes as needed and scheduling jobs in batches based on DAG dependencies.

On-Demand Pilot Job Scheduling for Adaptive Execution of HEP Workflows on HPC Systems (poster)

Motivation: 
•	Parallel Discrete Event Simulations (PDES) offers accurate HPC simulations but is computationally intensive and slow to scale. 
•	Surrogate models can accelerate simulations, and we explore if causal insights can improve their long-term forecasting stability. 
Research Questions: 
•	Can Causal Signals hidden in HPC simulations unlock better forecasting? 
•	Can different Causal Discovery methods identify key drivers for accurate surrogate forecasting?

Temporal Causal Graph Discovery  in Complex HPC Network Traffic Simulations (poster)

Motivation: 
- Dragonfly networks suffer from inter-job interference due to shared global links. 
- Existing routing algorithms fail to account for system-wide job patterns and link congestion. 
- Modern HPC systems offer flexible job placement, allowing users to specify workload locations for optimized performance.

Preventing Workload Interference with Intelligent Routing and Flexible Job Placement Strategy on Dragonfly system (poster)

High-performance computing (HPC) job scheduling has seen promising advances with Deep Reinforcement Learning (DRL). However, challenges such as low interpretability, instability, and high computational cost hinder DRL&rsquo;s practical adoption. We explore a model-based alternative using Monte Carlo Tree Search (MCTS) to overcome these limitations. By leveraging existing HPC simulators as models for MCTS and focusing on transparent decision-making, we aim to develop a scalable and interpretable scheduling solution fit for real-world deployment.

Overcoming RL Limitations in HPC Scheduling: A Model-Based MCTS Approach for Practical Deployment (poster)

Large language model (LLM) requests vary widely: a short factual query may produce only a few dozen tokens, while multi-step reasoning or proof generation can run to tens of thousands. Because generation is autoregressive, inference time grows proportionally with output length. Current serving stacks such as vLLM and Orca schedule requests using a simple first-come-first-serve (FCFS) policy. While fair, FCFS suffers from a well-known issue in LLM serving called head-of-line blocking, where a single long request delays all shorter ones behind it - leading to inflated tail latency and wasted throughput. A classical remedy is the Shortest-Job-First (SJF) policy, which improves efficiency by serving shorter jobs first. However, SJF requires knowledge of job length in advance - information that is unavailable in LLMs until generation completes, due to their autoregressive nature. 
We introduce a prompt-sensitive LLM task scheduler based on pairwise learning method from learning-to-rank utilizing Margin Ranking Loss and show that our method significantly improves scheduling performance with minimal overhead.

From Prompts to Priorities: Pairwise Learning-to-Rank Scheduling for Low-Latency  LLM Serving (poster)

Prior uncore frequency tuning studies have primarily focused on conventional HPC workloads running on CPU-only systems. As HPC advances toward heterogeneous computing, integrating diverse GPU workloads on heterogeneous CPU-GPU systems, it becomes imperative to revisit and enhance uncore scaling. Our investigation reveals that uncore frequency scales down only when CPU power approaches its TDP (Thermal Design Power) -- a rare scenario in GPU-dominant applications -- resulting in unnecessary power waste in modern computing systems.

MAGUS: Memory-Throughput-Based Uncore Frequency Scaling on Heterogeneous Systems (poster)

AllReduce is a critical collective in both HPC and large-scale AI workloads. However, scaling it to Exascale systems presents key challenges due to inter-node communication bottlenecks and underutilization of intra-node resources like shared memory and NVLink. This work analyzes state-of-the-art AllReduce algorithms to identify inefficiencies and opportunities for hybrid strategies that explicitly separate intra- and inter-node communication. 
We introduce a preliminary algorithmic design that leverages tunable intra-node communication patterns and discuss key performance criteria, including message count and data volume. Our early results provide insight into communication trade-offs and guide the development of adaptive AllReduce implementations optimized for Exascale systems.

EMMA: Efficient Multi-node Memory-aware AllReduce Algorithms (poster)

While behavioral studies have explored the spectrum between ASD and ADHD, there is a lack of research examining this continuum through fMRI-based brain connectivity. Our motivation is to fill this gap by leveraging topological data analysis to reveal transitional patterns in functional networks underlying these conditions.

Neurological Diseases Spectrum Mapping using Topological Data Analysis (poster)

Non-uniform MPI_Alltoallv communication is critical in many high-performance computing (HPC) applications where data exchange patterns vary significantly between processes. However, existing MPI implementations rely on fixed heuristics that are not well-suited for dynamic, irregular workloads, often leading to suboptimal performance. Furthermore, there are customized non-uniform All-to-all functions having performance advantage compared to official MPI Alltoallv, under certain scenarios. Creating a framework which can accurately select the optimal algorithm will be extremely beneficial, but challenging due to its potential complexity. 
Currently, there are research works regarding the data-driven approach for tuning MPI functions, but most of them they focus on MPI functions such as MPI_Scatter, MPI_Reduce, MPI_gather, uniform MPI_Alltoall Non-uniform Alltoall is still an under-developed area. One challenge of developing a tuning framework for non-uniform Alltoall is that the blocksizes between processes are not fixed and it introduces significant additional complexity to the tuning workflow.

Performance Characterization and Tuning of Non-uniform All-to-all Data Exchanges (poster)

Bottom‐up logic programming (e.g., Datalog) runs via repeated relational‐algebra kernels - selection, projection, join, and reorder - until no new facts emerge. While GPUs promise massive parallelism for these primitives, existing engines remain CPU-bound (8–16 threads) and struggle with deduplication and growing result sets. 
We introduce a WebGPU-based hash-join pipeline that: 
- Maps each RA primitive (hash‐join, GPU radix sort, deduplication, set difference, insertion into the full relation) to its own WGSL compute shader 
- Uses dynamic, growable GPU buffers and load-factor-tuned hash tables for memory management 
- Drives a delta-only fixed-point loop, processing only newly discovered tuples each iteration

Accelerating Iterative Relational Algebra Operations with WebGPU (poster)

Research Contributions: 
- First ever Datalog engine designed for multinode multi-GPU HPC systems, outperforming state-of-the-art shared-memory, distributed-memory, and GPU-based engines 
- Introduces novel GPU-Aware communication and buffer preparation strategies for scalable recursive query evaluation 
- Supports recursive aggregation for Datalog rules using high-throughput GPU kernels

MNMGDatalog: A Scalable Multi-Node Multi-GPU Datalog Engine (poster)

Gait disorders are commonly observed in older adults, who frequently experience various issues related to walking. Additionally, researchers and clinicians extensively investigate mobility related to gait in typically and atypically developing children, athletes, and individuals with orthopedic and neurological disorders. Effective gait analysis enables the understanding of the causal mechanisms of mobility and balance control of patients, the development of tailored treatment plans
to improve mobility, the reduction of fall risk, and the tracking of rehabilitation progress. However, analyzing gait data is a complex task due to the multivariate nature of the data, the large volume of information to be interpreted, and the technical skills required. Existing tools for gait analysis are often limited to specific patient groups (e.g., cerebral palsy), only handle a specific subset of tasks in the entire workflow, and are not openly accessible. To address these shortcomings, we conducted a requirements assessment with gait practitioners (e.g., researchers, clinicians) via surveys and identified key components of the workflow, including (1) data processing and (2) data analysis and visualization. Based on the findings, we designed VIGMA, an open-access visual analytics framework integrated with computational notebooks and a Python library, to meet the identified requirements. Notably, the framework supports analytical capabilities for assessing disease progression and for comparing multiple patient groups. We validated the framework through usage scenarios with experts specializing in gait and mobility rehabilitation. VIGMA is available at <Za href="https://github.com/komar41/vigma">github.com/komar41/VIGMA</a>.
Index Terms: Gait, stroke, rehabilitation, visual analytics, data processing, multivariate data, ensemble, survey, computational notebook. 
<a href="https://doi.org/10.48550/arXiv.2504.17960">https://doi.org/10.48550/arXiv.2504.17960</a> 
<a href="https://youtu.be/rh_u81BPU94">See a VIGMA demonstration.</a>

Access to raw trial videos - is provided through a scrollable video exploration view. Users can toggle between spatiotemporal views and the video exploration view using a checkbox.

VIGMA: An Open-Access Framework for Visual Gait and Motion Analytics

AI development is shaped by academics and industry leaders - let us call them &ldquo;infuencers&rdquo; - but it is unclear how their views align with those of the public. To address this gap, we developed an interactive platform that served as a data collection tool for exploring public views on AI, including their fears, hopes, and overall sense of hopefulness. We made the platform available to 330 participants representative of the U.S. population in terms of age, sex, ethnicity, and political leaning, and compared their views with those of 100 AI infuencers identifed by Time magazine. The public fears AI getting out of control, while infuencers emphasize regulation, seemingly to defect attention from their alleged focus on monetizing A&rsquo;s potential. Interestingly, the views of AI infuencers from
underrepresented groups such as women and people of color often differ from the views of underrepresented groups in the public. 
:CCS Concepts: Human-centered computing - Empirical studies in HCI; Empirical studies in collaborative and social computing; HCI design and evaluation methods; Visualization systems and tools; Computing methodologies - Artifcial intelligence; Social and professional topics - Codes of ethics; Information systems - Crowdsourcing. 
Keywords: responsible AI, ethical AI, AI governance, empirical ethics, value alignment, AI fears, AI hopes, AI infuencers, participatory AI ethics, crowdsourcing

The frst version of our platform for collecting public views on AI encourages participants to share their honest opinions to help break down opinion bubbles (R2).

The Hall of AI Fears and Hopes: Comparing the Views of AI Influencers and those of Members of the U.S. Public Through an Interactive Platform

Mapping the extent of flood events is a necessary and important aspect of disaster management. In recent years, deep learning methods have evolved as an effective tool to quickly label high-resolution imagery and provide necessary flood extent mappings. These methods, though, require large amounts of annotated training data to create models that are accurate and robust to new flooded imagery. In this work, we present FloodTrace, a web-based application that enables effective crowdsourcing of flooded region annotation for machine learning applications. To create this application, we conducted extensive interviews with domain experts to produce a set of formal requirements. Our work brings topological segmentation tools to the web and greatly improves annotation efficiency compared to the state-of-the-art. The user-friendliness of our solution allows researchers to outsource annotations to non-experts and utilize them to produce training data with equal quality to fully expert-labeled data. We conducted a user study to confirm our application’s effectiveness in which 266 graduate students annotated high-resolution aerial imagery from Hurricane Matthew in North Carolina. Experimental results show the efficiency benefits of our application for untrained users, with median annotation time less than half the state-of-the-art annotation method. In addition, using our application’s aggregation and correction framework, flood detection models trained on crowdsourced annotations were able to achieve performance equal to models trained on fully expert-labeled annotations, while requiring a fraction of the expert&rsquo;s time. 
Keywords: Terrain visualization, machine learning, collaborative and distributed visualization, topology-based techniques 
Index Terms: Human-centered computing [Visualization]: Visualization application domains - Geographic visualization

Visualization of aggregated crowdsourced data from 45 participants.

Enabling Fast and Accurate Crowdsourced Annotation for Elevation-Aware Flood Extent Mapping

Graph drawing, or graph layout creation, is a computationally difficult challenge in visualization that involves placing the vertices of a graph into a layout that provides insight into its structure. In order to visualize large-scale graphs, effective layouts are necessary for understanding. Previous work has shown the potential for graph drawing directly in the web browser by using WebGPU, a new API that brings the full capabilities of modern GPUs to the web. Compared to the existing state-of-the-art for web-based graph visualization, which rely on CPU-based graph drawing algorithms, WebGPU-accelerated work improves performance and scalability. However, we find that existing WebGPU solutions utilize suboptimal quadtree data structures for graph drawing. In this work, we implement a modified quadtree data structure that uses a Hilbert spatial ordering for a fully parallelizable bottom-up construction algorithm in WebGPU. We utilize this data structure, along with optimizations to the quadtree traversal, to propose a massively more performant graph drawing algorithm. We evaluate the performance of our work against the existing state-of-the-art and demonstrate up to 69.5× speed-ups for layout creation of relevant graphs while enabling graph drawing for datasets of much larger size. 
Keywords: Graph layout creation, web-based, quadtree, GPU. 
Index Terms: Human-centered computing [Visualization]: Visualization techniques - Graph drawings

Example graph drawn for the Pkustk13 dataset from the SuiteSparse matrix collection, consisting of 94,893 vertices and 6,616,827 edges.

Accelerating Web-Based Graph Drawing with Bottom-Up GPU Quadtree Construction

Data visualization methodologies were intensively leveraged during the COVID-19 pandemic. We review our design experience working on a set of interdisciplinary COVID-19 pandemic projects. We describe the challenges we met in these projects, characterize the respective user communities, the goals and tasks we supported, and the data types and visual media we worked with. Furthermore, we instantiate these characterizations in a series of case studies. Finally, we describe the visual analysis lessons we learned, considering future pandemics. 
<a href="https://ieeexplore.ieee.org/document/10964487">https://ieeexplore.ieee.org/document/10964487</a>

Empowering Communities: Tailored Pandemic Data Visualization for Varied Tasks and Users

Authors: Humbert-Vidan. L., Kamel, S., Wentzel, A., Kaffey, Z., Abdelaal ,M., Spier, K. B., West, N. A., Marai, G. E., Canahuate, G., Zhang, X, Chen, M.M., Wahid, K. A., Rigert, J., Hosseinian, S., Schaefer, A. J., Brock, K. K., Chambers, M., Otun, A. O., Aponte-Wesson, R., Patel, V., Hope, A., Phan, J., Garden, A. S., Frank, S. J., Morrison, W. H., Spiott, M. T., Rosenthal, D., Lee, A., He, R., Naser, M. A., Watson, E., Hutcheson, K. A., Mohamed, A. S. R., Sandulache, V. C., van Dijk, L. V., Moreno, A. C., Urbano, T. G., Fuller, C. D., Lai, S. Y. 
Background 
Existing studies on osteoradionecrosis of the jaw (ORNJ) have primarily used cross-sectional data, assessing risk factors at a single time point. Determining the time-to-event profile of ORNJ has important implications to monitor oral health in head and neck cancer (HNC) long-term survivors. 
Methods 
Data were retrospectively obtained for a clinical observational cohort of 1129 patients (198 ORNJ cases) with HNC treated with radiotherapy (RT) at The University of Texas MD Anderson Cancer Center. A Weibull Accelerated Failure Time model was trained on previously identified dosimetric, clinical and demographic predictors. External validation was performed using an independent cohort of 265 patients (92 ORNJ cases) treated at Guy’s and St. Thomas’ Hospitals. To facilitate clinical implementation of the model, an online graphical user interface (GUI) was developed, including formal stakeholder usability testing. 
Results 
Our model identified that gender (males), pre-RT dental extractions and D25% were associated with a 38 %, 27 % and 12 % faster onset of ORNJ, respectively, with adjusted time ratios of 0.62 (p = 0.11), 0.73 (p = 0.13) and 0.88 (p < 0.005). The model demonstrated strong internal calibration (integrated Brier score of 0.133, D-calibration p-value 0.998) and optimal discrimination at 72 months (Harrell’s C-index of 0.72). 
Conclusion 
This study is the first to demonstrate a direct relationship between radiation dose and the time to ORNJ onset, providing a novel characterization of the impact of delivered dose and patient-related factors not only on the probability of a late effect (ORNJ), but the conditional risk during survivorship. 
Keywords: Osteoradionecrosis of the jaw, Time-to-event prediction models, Normal tissue complication prediction, Radiation-induced toxicity, Decision support tool, Radiotherapy, Head and neck cancers 
<a href="https://doi.org/10.1016/j.radonc.2025.110890">https://doi.org/10.1016/j.radonc.2025.110890</a>

Screenshot of WAFT-based time-to-ORNJ online calculator GUI. The user can either obtain a predicted risk of developing ORNJ at a specific time point or visually assess the time-dependency of ORNJ risk with the different covariates of the ORNJ WAFT model.

Externally validated digital decision support tool for time-to-osteoradionecrosis risk-stratification using right-censored multi-institutional observational cohorts

September 2023 marked the 50th anniversary of the Electronic Visualization Laboratory (EVL). This paper summarizes EVL&rsquo;s efforts in Visual Data Science, with a focus on the many networked, immersive, collaborative visualization and virtual reality (VR) systems and applications the Lab has developed and deployed, as
well as lessons learned and future plans. 
Index terms: Visualization, Virtual Reality, Visual Data Science, Collaboration

An EVL CAD Rendering of the DOCC Space

Immersive Analytics at the Electronic Visualization Laboratory

In consumer research, understanding consumer behavior and experiences is vital for making informed decisions about product design, innovation and marketing. Zaltman&rsqu;s Metaphor Elicitation Technique (ZMET) leverages metaphors and non-verbal communication to uncover and gain deeper insights into consumers&rsquo; thoughts and emotions. This paper introduces a novel system that enables consumer researchers (interviewers) to perform a modified version of metaphor elicitation interviews in virtual reality (VR). Consumers (participants) use 3D objects in the virtual environment to express their thoughts and emotions, instead of pictures conventionally used in a ZMET interview. The system features an asymmetric setup where the participant is immersed in VR using a head-mounted display (HMD), while the interviewer views the participant&rsquo;s perspective on a monitor. We discuss the technical and design aspects of the VR system and present the results of a user study (N = 17) that we conducted to validate the effectiveness of performing the metaphor elicitation interviews in VR. This work also explores the experiences of both participants and interviewers during the interview sessions, aiming to identify potential improvements. The qualitative and quantitative analysis of the data demonstrated how immersion, presence and embodied interaction positively affect and aid in sense-making and deeper expression of the participants&rsquo; thoughts, perspectives and emotions. 
Index Terms - Human computer interaction (HCI), virtual reality (VR), collaborative virtual environments, asymmetric VR, VR applications, embodied interaction, presence, immersion, perception and cognition, deep metaphors, consumer research, Zaltman&rsquo;s metaphor elicitation technique (ZMET).

Areas in the virtual home environment (A) Exercise area, (B) Outdoor space - summer and (C) Outdoor space - winter.

Enhancing Consumer Insights through VR Metaphor Elicitation

Patient-Reported Outcomes (PRO) consist of information provided directly by the patients about their health status including symptom ratings. PROs are commonly used in clinical practice to support clinical decision-making and have recently been incorporated into machine learning models to improve risk prediction. In this work, we aim to evaluate whether the inclusion of a patient stratification based on 12-month post-treatment predicted Patient Reported Outcomes improves risk prediction of radiation-induced toxicity and overall survival for head and neck cancer patients. A bidirectional long-short term memory (Bi-LSTM) recurrent neural network was used to model the longitudinal PRO data and to predict symptom ratings 12 months post-treatment. Patients were stratified using hierarchical clustering over the LSTM-predicted data. A logistic regression model was trained to predict Xerostomia at 12 months and a Cox regression model to predict overall survival. Results show that the inclusion of symptom burden clusters derived from the predicted Patient Reported Outcomes improves radiation-induced toxicity and overall survival prediction for head and neck cancer patients. 
Index Terms: Patient reported outcomes (PRO), deep learning, patient clustering, regression, survival analysis, xerostomia. 
<a href="https://doi.org/10.1109/JBHI.2024.3515092">https://doi.org/10.1109/JBHI.2024.3515092</a>

RMSEs at M12 for all 22 symptoms between Bi-LSTM prediction and actual testing data.

PRO-Based Stratification Improves Model Prediction for Toxicity and Survival of Head and Neck Cancer Patients

Hummingbird is an innovative, award-winning performance engaging participants in active storytelling that bridges live theater and collaborative interaction through virtual reality (VR). Hummingbird&rsquo;s story celebrates courage and coming of age through the eyes of a gutsy teen who must outsmart her mother&rsquo;s egotistic boss and survive a dangerous new technology in a live, immersive adventure. Developed at the University of Illinois Chicago by faculty and over 30 students from the departments of Computer Science and Design in partnership with professional theater producers, directors, actors, videographers and composers, this project advanced interdisciplinary collaboration, provided a unique learning environment and broadened the research experience for several cohorts of students. Over 500 people attended Hummingbird’s performances at the Tony Award-winning Goodman Theatre&rsquo;s New Stages Festival, Chicago Children&rsquo;s Theater and SIGGRAPH 2022 in Vancouver, Canada, with over 200 active VR participants. In each performance, five VR participants actively collaborate with each other and a lead actor within the VR adventure, contributing problem-solving, collaboration and teamwork, while a greater audience simultaneously follows the virtual performance aspects on a large video wall in real-time. Discussion sessions and audience evaluations followed each performance, informed the Hummingbird team on script, design and interactivity to improve future performances. Through qualitative analysis of audience experiences and insights from our collaboration, we discuss key considerations and design recommendations for integrating VR with live theater. Hummingbird demonstrates how VR can revolutionize theatrical storytelling by enabling traditional theater to narrate epic stories that were once considered too ambitious for traditional stage by extending live theater and making VR accessible to a broader audience. This project profit theater and interdisciplinary research institutions to increase opportunities for cross-disciplinary student education. 
CCS CONCEPTS: Computing methodologies; Computer Graphics; Graphics systems and interfaces; Virtual reality; Applied computing; Arts and humanities; Performing arts 
KEYWORDS Virtual reality, Storytelling, Theater, Multi-user, Performance, Interdisciplinary Collaboration, Design Process

VR participants interact in the Orchard scene while observers watch their interaction on the large screen.

Hummingbird: Live Theater Adventure Empowering Collaboration in Virtual Reality

Research in visual computing (VC) focuses on integrating visual representations of datasets with computing systems, thus leveraging the complementary abilities of humans and machines. The resulting VC systems may include machine learning (ML) techniques, which have remarkable abilities in terms of processing vast amounts of data. However, creating VC+ML systems that can operate on spatio-temporal data is a largely unexplored, difficult topic. Challenges in this area include: capturing domain characterizations for spatial VC+ML problems, in particular when serving both ML model builders and ML model clients; creating explainable VC+ML models that can operate on spatial data, including measures that can capture spatial structure similarity; designing visual encodings for ML models that use spatial information and that enhance the modeler and client understanding of the model; and measuring the effect of deploying such VC+ML systems in practice. br>
Based on several multi-year collaborations, in this dissertation I will document several instances where spatial and explainable VC+ML solutions are required, and what role spatiality plays in these situations. Throughout this process, I extract and describe design requirements for explainable spatial VC+ML systems, and I identify necessary VC advances, including spatially-aware similarity measures and spatial visual explainability. First, I construct spatially-aware similarity measures to support ML problems in precision oncology, patient risk stratification in data mining, explainable interactive regression modeling for hypothesis testing in social science, and digital twin interfaces for cancer treatment planning. Second, I introduce novel encodings for 3-dimensional and geospatial data and explore how
to integrate such encodings with ML model explanations. Finally, I implement and evaluate several VC+ML systems and several resulting models that are explicitly designed to integrate domain-specific structural spatial information and unsupervised ML. Throughout this process, I also articulate design lessons related to model actionability.

Development of the organ diagram used to encode organ-specific values. (Left) Organs that are located in the center of the head (Right) Bilateral organs that occur on both the left and right sides of the head.

Visual Computing Design for Explainable, Spatially-aware Machine Learning

With the decreasing cost of consumer display technologies making it easier for universities to have larger displays in classrooms, and the ubiquitous use of online tools such as collaborative whiteboards for remote learning during the COVID-19 pandemic, combining the two can be useful in higher education. This is especially true in visually intensive classes, such as data visualization courses, that can benefit from additional &ldquo;space to teach,&rdquo; coined after the &ldquo;space to think&rdquo; sense-making idiom. In this paper, we reflect on our approach to using SAGE3, a collaborative whiteboard with advanced features, in higher education to teach visually intensive classes, provide examples of activities from our own visually-intensive courses, and present student feedback. We gather our observations into usage patterns for using content-rich canvases in education.

A SAGE3 session can be shared across multiple locations: some locations may serve multiple users co-located in front of a large wall, while others may serve an individual with varied types of displays.

Space to Teach: Content-Rich Canvases for Visually-Intensive Education

Over the past decade, several urban visual analytics systems and tools have been proposed to tackle a host of challenges faced by cities, in areas as diverse as transportation, weather, and real estate. Many of these tools have been designed through collaborations with urban experts, aiming to distill intricate urban analysis workflows into interactive visualizations and interfaces. However, the design, implementation, and practical use of these tools still rely on siloed approaches, resulting in bespoke systems that are difficult to reproduce and extend. At the design level, these tools undervalue rich data workflows from urban experts, typically treating them only as data providers and evaluators. At the implementation level, they lack interoperability with other technical frameworks. At the practical use level, they tend to be narrowly focused on specific fields, inadvertently creating barriers to cross-domain collaboration. To address these gaps, we present Curio, a framework for collaborative urban visual analytics. Curio uses a dataflow model with multiple abstraction levels (code, grammar, GUI elements) to facilitate collaboration across the design and implementation of visual analytics components. The framework allows experts to intertwine data preprocessing, management, and visualization stages while tracking the provenance of code and visualizations. In collaboration with urban experts, we evaluate Curio through a diverse set of usage scenarios targeting urban accessibility, urban microclimate, and sunlight access. These scenarios use different types of data and domain methodologies to illustrate Curio&rsquo;s flexibility in tackling pressing societal challenges. Curio is available at <a=href"urbantk.org/curio.">urbantk.org/curio</a>. 
Index Terms - Urban analytics, urban data, spatial data, dataflow, provenance, visualization framework, visualization system.

Using Curio to build a dataflow to analyze the shadow impact of a proposed building in Boston.

Curio: A Dataflow-Based Framework for Collaborative Urban Visual Analytics

Unsupervised video anomaly detection (UVAD) aims to detect abnormal events in videos without any annotations. It remains challenging because anomalies are rare, diverse, and usually not well-defined. Existing UVAD methods are purely data-driven and perform unsupervised learning by identifying various abnormal patterns in videos. Since these methods largely rely on the feature representation and data distribution, they can only learn salient anomalies that are substantially different from normal events but ignore the less distinct ones. To address this challenge, this paper pursues a different approach that leverages data-irrelevant prior knowledge about normal and abnormal events for UVAD. We first propose a new normality prior for UVAD, suggesting that the start and end of a video are predominantly normal. We then propose normality propagation, which propagates normal knowledge based on relationships between video snippets to estimate the normal magnitudes of unlabeled snippets. Finally, unsupervised learning of abnormal detection is performed based on the propagated labels and a new loss re-weighting method. These components are complementary to normality propagation and mitigate the negative impact of incorrectly propagated labels. Extensive experiments on the ShanghaiTech and UCF-Crime benchmarks demonstrate the superior performance of our method. 
Funding: NSF award ECCS-2400900

Learning Anomalies with Normality Prior for Unsupervised Video Anomaly Detection

Deep learning approaches have made significant success in single-view 3D reconstruction, but they often rely on expensive 3D annotations for training. Recent efforts tackle this challenge by adopting an analysis-by-synthesis paradigm to learn 3D reconstruction with only 2D annotations. However, existing methods face limitations in both shape reconstruction and texture generation. This paper introduces an innovative Analysis-by-Synthesis Transformer that addresses these limitations in a unified framework by effectively modeling pixel-to-shape and pixel-to-texture relationships. It consists of a Shape Transformer and a Texture Transformer. The Shape Transformer employs learnable shape queries to fetch pixel-level features from the image, thereby achieving high-quality mesh reconstruction and recovering occluded vertices. The Texture Transformer employs texture queries for non-local gathering of texture information and thus eliminates incorrect inductive bias. Experimental results on CUB-200-2011 and ShapeNet datasets demonstrate superior performance in shape reconstruction and texture generation compared to previous methods. 
Funding: NSF award CNS-1828265, NSF award ECCS-2400900

Analysis-by-Synthesis Transformer for Single-View 3D Reconstruction

In the biomedical domain, taxonomies organize the acquisition modalities of scientific images in hierarchical structures. Such taxonomies leverage large sets of correct image labels and provide essential information about the importance of a scientific publication, which could then be used in biocuration tasks. However, the hierarchical nature of the labels, the overhead of processing images, the absence or incompleteness of labeled data, and the expertise required to label this type of data impede the creation of useful datasets for biocuration. From a multi-year collaboration with biocurators and text-mining researchers, we derive an iterative visual analytics and active learning strategy to address these challenges. We implement this strategy in a system called BI-LAVA—Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analytics. BILAVA leverages a small set of image labels, a hierarchical set of image classifiers, and active learning to help model builders deal with incomplete ground-truth labels, target a hierarchical taxonomy of image modalities, and classify a large pool of unlabeled images. BI-LAVA&rsquo;s front end uses custom encodings to represent data distributions, taxonomies, image projections, and neighborhoods of image thumbnails, which help model builders explore an unfamiliar image dataset and taxonomy and correct and generate labels. An evaluation with machine learning practitioners shows that our mixed human-machine approach successfully supports domain experts in understanding the characteristics of classes within the taxonomy, as well as validating and improving data quality in labeled and unlabeled collections. 
CCS Concepts: Human-centered computing -> Visual analytics; Computing methodologies -> Machine learning algorithms

Interaction during the case study: COVID-19 Image Modality Labeling.

BI-LAVA: Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analytics

Over the past decade, there has been a significant increase in the development of visual analytics systems dedicated to addressing urban issues. These systems distill intricate urban analysis workflows into intuitive, interactive visual representations and interfaces, enabling users to explore, understand, and derive insights from large and complex data, including street-level imagery, street networks, and building geometries. Developing urban visual analytics systems, however, is a challenging endeavor that requires considerable programming expertise and interaction between various multidisciplinary stakeholders. This situation often leads to monolithic and isolated prototypes that are hard to reproduce, combine, or extend. Concurrently, there has been an increase in the availability of general and urban-specific toolkits, frameworks, and authoring tools that are open source and abstract away the need to implement low-level visual analytics functionalities. This paper provides a hierarchical taxonomy of urban visual analytics systems to contextualize how they are usually designed, implemented, and evaluated. We develop this taxonomy across three distinct levels (i.e., dimensions, categories, and tags), juxtaposing visualization with analytics, data, and system dimensions. We then assess the extent to which current open-source toolkits, frameworks, and authoring tools can effectively support the development of components tailored to urban visual analytics, identifying their strengths and limitations in addressing the unique challenges posed by urban data. In doing so, we offer a roadmap that can guide the effective employment of existing resources and chart a pathway for developing and refining future systems. 
Keywords: Visual Analytics, Visualization Toolkits, Visualization Grammars, Visualization Authoring, Urban Visual Analytics

Assessing the Landscape of Toolkits, Frameworks, and Authoring Tools for Urban Visual Analytics Systems

This study addresses the prevalent issue of missing data in patient-reported outcome datasets, particularly focusing on head and neck cancer patient symptom ratings sourced from the MD Anderson Symptom Inventory. Given that many data mining and machine learning algorithms necessitate complete datasets, the accurate imputation of missing data as an initial step becomes crucial. In this study we propose, for the first time, the use of collaborative filtering for imputing missing head and neck cancer patient symptom ratings. Two configurations of collaborative filtering, namely patient-based and symptom-based, leverage known ratings to infer the missing ones. Additionally, this study compares the performance of collaborative filtering with alternative imputation methods such as Multiple Imputation by Chained Equations, Nearest Neighbor Imputation, and Linear interpolation. Performance is compared using Root Mean Squared Error and Mean Absolute Error metrics. Findings demonstrate that collaborative filtering is a viable and comparatively superior approach for imputing missing patient symptom data. 
Keywords: Head and Neck Cancer, Imputation, Collaborative Filtering 
<a href="https://doi.org/10.1007/978-3-031-68309-1_20">https://doi.org/10.1007/978-3-031-68309-1_20</a>

Collaborative Filtering for the Imputation of Patient Reported Outcomes

September 2023 marks the 50th anniversary of the Electronic Visualization Laboratory (EVL) at University of Illinois Chicago (UIC). EVL&rsquo;s introduction of the CAVE Automatic Virtual Environment in 1992, the first widely replicated, projection-based, walk-in, virtual-reality (VR) system in the world, put EVL at the forefront of collaborative, immersive data exploration and analytics. However, the journey did not begin then. Since its founding in 1973, EVL has been developing tools and techniques for real-time, interactive visualizations - pillars of VR. But EVL&rsquo;s culture is also relevant to its successes, as it has always been an interdisciplinary lab that fosters teamwork, where each person&rsquo;s expertise contributes to the development of the necessary tools, hardware, system software, applications, and human interface models to solve problems. Over the years, as multidisciplinary collaborations evolved and advanced scientific instruments and data resources were distributed globally, the need to access and share data and visualizations while working with colleagues, local and remote, synchronous and asynchronous, also became important fields of study. This paper is a retrospective of EVL&rsquo;s past 50 years that surveys the many networked, immersive, collaborative visualization and VR systems and applications it developed and deployed, as well as lessons learned and future plans. 
Article appears in the journal PRESENCE: Virtual and Augmented Reality, in the Special Issue on Immersive Visualization Laboratories - Past, Present and Future, edited by S. Su and W. Sherman, <a href="https://direct.mit.edu/pvar/issue/volume/33">https://direct.mit.edu/pvar/issue/volume/33</a>. 
Article copyrighted by Massachusetts Institute of Technology (MIT). 
DOI: <a href="https://doi.org/10.1162/pres_a_00421">https://doi.org/10.1162/pres_a_00421</a>

EVL faculty, students, and collaborators 1975.

Electronic Visualization Laboratory&rsquo;s 50th Anniversary Retrospective: Look to the Future, Build on the Past

Meetings and conversations today are multimodal in nature, including the usage of diverse technologies in the form of text, video, images, voice, AI agents, and also immersive technologies. If it were possible to replay important multimodal conversations in their spatial and chronological contexts, it would benefit several audiences, such as - researchers to study and analyze multimodal conversations, novices in any field who could learn from experts, experts who have missed being part of crucial conversations. However, recreating such multimodal interactions requires integrating all of the modalities used in the conversation. Immersive environments provide a convenient setting to present data with its spatial and temporal referents through the use of situated analytics. We present MuSA (Multimodal Situated Analytics), an immersive environment prototype, as a way to recreate multimodal conversations for exploration and analysis in immersive environments. Our development pipeline consisted of various stages such as data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We conducted two comprehensive user studies to evaluate the usability, user adoption, and space usage during the exploration of multimodal conversations with our application. In the first study, we explore conversations between seated participants (n=12), and in the second, we explore conversations between non-seated moving participants (n=13). We share insights gained during the development phase, as well as empirical results and feedback collected through both user studies.

Depiction of an analyst investigating a multimodal conversation that includes participants and other entities of the conversation through MuSA in Mixed Reality (HoloLens2).

Multimodal Situated Analytics (MuSA) for Analyzing Conversations in Extended Reality

SAGE3 (the Smart Amplified Group Environment) is an open-source collaboration software platform designed for real-time interactive information and visualization sharing. Seamlessly complementing video conferencing tools like Zoom, Microsoft Teams, and Google Meets, it enhances group environments for information-rich collaboration. With SAGE3, users can collaborate on their laptops, meeting room projectors, and on large tiled display walls. SAGE3 on large display spaces has proven to enable people to think, brainstorm, discover and reach decisions with greater speed, accuracy, comprehensiveness and confidence. SAGE3 provides a large collaborative canvas for sharing and editing of media, data, code, visualizations, dashboards, and screen-shares. SAGE3 is ideally suiting for collaborative data science, project meetings, and classroom education scenarios. <a href="https://sage3.sagecommons.org/">https://sage3.sagecommons.org/</a>

SAGE3: Smart Amplified Group Environment for harnessing the data revolution (poster)

Digital twin models are of high interest to Head and Neck Cancer (HNC) oncologists, who have to navigate a series of complex treatment decisions that weigh the efficacy of tumor control against toxicity and mortality risks. Evaluating individual risk profiles necessitates a deeper understanding of the interplay between different factors such as patient health, spatial tumor location and spread, and risk of subsequent toxicities that can not be adequately captured through simple heuristics. To support clinicians in better understanding tradeoffs when deciding on treatment courses, we developed DITTO, a digital-twin and visual computing system that allows clinicians to analyze detailed risk profiles for each patient, and decide on a treatment plan. DITTO relies on a sequential Deep Reinforcement Learning digital twin (DT) to deliver personalized risk of both long-term and short-term disease outcomes and toxicity risk for HNC patients. Based on a participatory collaborative design alongside oncologists, we also implement several visual explainability methods to promote clinical trust and encourage healthy skepticism when using our system. We evaluate the efficacy of DITTO through quantitative evaluation of performance and case studies with qualitative feedback. Finally, we discuss design lessons for developing clinical visual XAI applications for clinical end users. 
Index Terms: Medicine; Machine Learning; Application Domains; High Dimensional data; Spatial Data; Activity Centered Design

Overview of DITTO.

DITTO: A Visual Digital Twin for Interventions and Temporal Treatment Outcomes in Head and Neck Cancer

Though the object detection performance on standard benchmarks has been improved drastically in the last decade, current object detectors are often vulnerable to domain shift between the training data and testing images. Domain adaptation techniques have been developed to adapt an object detector trained in a source domain to a target domain. However, they assume that the target domain is known and fixed and that a target dataset is available for training, which cannot be satisfied in many real-world applications. To close this gap, this paper investigates fully test-time adaptation for object detection. It means to update a trained object detector on a single testing image before making a prediction, without access to the training data. Through a diagnostic study of a baseline self-training framework, we show that a great challenge of this task is the unreliability of pseudo labels caused by domain shift. We then propose a simple yet effective method, termed the IoU Filter, to address this challenge. It consists of two new IoU-based indicators, both of which are complementary to the detection confidence. Experimental results on five datasets demonstrate that our approach could effectively adapt a trained detector to various kinds of domain shifts at test time and bring substantial performance gains. Code is available at https://github.com/XiaoqianRuan1/IoU-filter. Code is available at https://github.com/ XiaoqianRuan1/IoU-filter. 
Funding:: The COMPaaS DLV project (NSF award CNS-1828265)

Fully Test-time Adaptation for Object Detection

Unsupervised semantic segmentation (USS) aims at partitioning an image into semantically meaningful segments by learning from a collection of unlabeled images. The effectiveness of current approaches is plagued by difficulties in coordinating representation learning and pixel clustering, modeling the varying feature distributions of different classes, handling outliers and noise, and addressing the pixel class imbalance problem. This paper introduces a novel approach, termed Imbalance-Aware Dense Discriminative Clustering (IDDC), for USS, which addresses all these difficulties in a unified framework. Different from existing approaches, which learn USS in two stages (i.e., generating and updating pseudo masks, or refining and clustering embeddings), IDDC learns pixel-wise feature representation and dense discriminative clustering in an end-to-end and self-supervised manner, through a novel objective function that transfers the manifold structure of pixels in the embedding space of a vision Transformer (ViT) to the label space while tolerating the noise in pixel affinities. During inference, the trained model directly outputs the classification probability of each pixel conditioned on the image. In addition, this paper proposes a new regularizer, based on the Weibull function, to handle pixel class imbalance and cluster degeneration in a single shot. Experimental results demonstrate that IDDC significantly outperforms all previous USS methods on three real-world datasets, COCO-Stuff-27, COCO-Stuff-171, and Cityscapes. Extensive ablation studies validate the effectiveness of each design. Our code is available at https://github.com/MY-LIU100101/IDDC. Our code is available at https://github.com/MY-LIU100101/IDDC. 
Keywords: Unsupervised semantic segmentation · Imbalance-Aware Dense Discriminative Clustering · End-to-end training ·
Deep clustering 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Imbalance-Aware Discriminative Clustering for Unsupervised Semantic Segmentation

Studies have proven the effectiveness of mindfulness and forest-immersion therapy as an option for stress reduction. However, what happens when an individual is seeking stress relief but the outdoors is unavailable due to unpredictable weather or even a lack of accessibility? The purpose of this research project is to observe the effects of guided scenic meditation in a virtual reality environment on physiological stress. Additionally, it performs a platform comparison by deploying the same meditative experience and testing its effectiveness using a room-scale immersive environment and a desktop computer, as desktops ($400 - $1000) are far more affordable and widely available compared to room-scale immersive environments ($1 million+). Using each platform, twenty participants were asked to engage with a meditative application that simulates the sights, sounds, and smells of a forest environment. Based on the objective data collected, the participants experienced a decrease in both heart rate and blood pressure, demonstrating the effectiveness of the simulation in reducing physiological stress. However, the room-scale immersive environment proved to be more successful in reducing the heart rate than the desktop computer. Furthermore, according to the participants, their subjectively perceived stress levels decreased, resulting in a far more relaxed state than before. The results showcase that a virtual simulation of a real environment actively contributes to stress reduction and can be used in cases when the outdoors is unavailable. The platform comparison demonstrates that the product of the research - a guided scenic meditation application that simulates a forest - is accessible and does not require a million-dollar room-scale display in order to be effective in reducing stress.

The guided scenic meditation application deployed to the CAVE2, pictured on the left, and the desktop, pictured on the right.

The Effects of Virtual Reality Environments on Physiological Stress: A Platform Comparison Between Room-Scale Displays and Desktop Computers (MS Thesis Project)

This project was performed during the COVID-19 pandemic. It presents a visual computing framework (MOTIV) for analyzing moral rhetoric on social media around controversial topics such as Stay-at-Home orders. Using Moral Foundation Theory, it proposes a methodology for deconstructing and visualizing the when, where, and who behind each of these moral dimensions as expressed in microblog data. The project was developed in collaboration with experts from language processing, communications, and causal inference. Our approach integrates microblog data with multiple sources of geospatial and temporal data, and leverages unsupervised machine learning (generalized additive models) to support collaborative hypothesis discovery and testing. 
Abstract: We present a visual computing framework for analyzing moral rhetoric on social media around controversial topics. Using Moral Foundation Theory, we propose a methodology for deconstructing and visualizing the when, where, and who behind each of these moral dimensions as expressed in microblog data. We characterize the design of this framework, developed in collaboration with experts from language processing, communications, and causal inference. Our approach integrates microblog data with multiple sources of geospatial and temporal data, and leverages unsupervised machine learning (generalized additive models) to support collaborative hypothesis discovery and testing. We implement this approach in a system named MOTIV. We illustrate this approach on two problems, one related to Stay-at-home policies during the COVID-19 pandemic, and the other related to the Black Lives Matter movement. Through detailed case studies and discussions with collaborators, we identify several insights discovered regarding the different drivers of moral sentiment in social media. Our results indicate that this visual approach supports rapid, collaborative hypothesis testing, and can help give insights into the underlying moral values behind controversial political issues. 
Supplemental Material: <a href="https://osf.io/ygkzn/?view_only=6310c0886938415391d977b8aae8b749">https://osf.io/ygkzn/?view_only=6310c0886938415391d977b8aae8b749</a>

MOTIV visualization of moral framing on social media in 2020.

MOTIV: Visual Exploration of Moral Framing in Social Media

Sunlight and shadow play critical roles in how urban spaces are utilized, thrive, and grow. While access to sunlight is essential to the success of urban environments, shadows can provide shaded places to stay during the hot seasons, mitigate heat island effect, and increase pedestrian comfort levels. Properly quantifying sunlight access and shadows in large urban environments is key in tackling some of the important challenges facing cities today. In this paper, we propose Dee Umbra, a novel computational framework that enables the quantification of sunlight access and shadows at a global scale. Our framework is based on a conditional generative adversarial network that considers the physical form of cities to compute high-resolution spatial information of accumulated sunlight access for the different seasons of the year. We use data from seven different cities to train our model, and show, through an extensive set of experiments, its low overall RMSE (below 0.1) as well as its extensibility to cities that were not part of the training set. Additionally, we contribute a set of case studies and a comprehensive dataset with sunlight access information for more than 100 cities across six continents of the world. Deep Umbra is available at <a href="https://urbantk.org/shadows">https://urbantk.org/shadows</a>. 
Index Terms - Urban computing, Urban analytics, Sunlight access, Shadow, Generative adversarial networks.

Example where shadows are cast across tiles.

Deep Umbra: A Generative Approach for Sunlight Access Computation in Urban Spaces

e-JUST - Environmental Justice using Urban Scalable Toolkit, a state-of-the-art GIS-based web platform, is being developed to inform local and regional decision-making regarding air quality and heat in the Chicago metropolitan area, utilizing Earth science and high-resolution climate model outputs. e-JUST contributes to mitigating risks associated with extreme heat and air pollution by improving decision-making related to heat and air quality stressors in the Chicago metropolitan region. e-JUST technology architecture involves ReactJS, Vega-Lite, and Urban Toolkit's functionalities. e-JUST integrates a variety of meteorological and air quality datasets. Among these are NASA&rsquo;s remote sensing observations, such as GOES 16 and ECOSTRESS, that measure surface temperatures at 100 m and hourly intervals. The environmental data layers include high-resolution (1 km and higher) outputs from urban climate models related to temperature, NO2 and PM2.5, ozone, and other pollutants. To account for demographic data, we use the 2020 US Census. Our data fusion approaches include further downscaling environmental data layers to street scale using machine learning algorithms for spatiotemporal interpolation, enabling more accurate predictions at scales relevant to communities. e-JUST&rsquo;s statistical view includes a heat matrix that illustrates correlations between several environmental and socioeconomic variables at a grid point in time, as well as a scatterplot that allows users to compare observed data and forecasts of two different variables. Overall, this toolkit provides information for the near past and near-real-time forecasting to aid in decision-making and emergency management preparedness for Chicago communities. Additionally, our platform is designed to provide users with easy access to data, enabling them to view and analyze data immediately. The e-JUST toolkit is anticipated to benefit a wide audience, ranging from the general public to academic researchers.

e-JUST - Environmental Justice using Urban Scalable Toolkit

Similarity detection seeks to identify similar, but distinct items over multivariate datasets. Often, similarity cannot be defined computationally, leading to a need for visual analysis, such as in cases with ensemble, computational, patient cohort, or geospatial data. In this work, we empirically evaluate the effectiveness of common visual encodings for multivariate data in the context of visual similarity detection. We conducted a user study with 40 participants to measure similarity detection performance and response time under moderate scale (16 items) and large scale (36 items). Our analysis shows that there are significant differences in performance between encodings, especially as the number of items increases. Surprisingly, we found that juxtaposed star plots outperformed superposed parallel coordinate plots. Furthermore, color-cues significantly improved response time, and attenuated error at larger scales. In contrast to existing guidelines, we found that filled star plots (Kiviats) outperformed other encodings in terms of scalability and error.

Similarity detection over multivariate data using five visual encodings (clockwise order from top left): Kiviat diagrams, Lines-Only glyphs, Color-cue Kiviat diagrams, Parallel Coordinate Plots, and Color-cue Parallel Coordinate Plots.

Kiviat Defense: An Empirical Evaluation of Visual Encoding Effectiveness in Multivariate Data Similarity Detection

Biocuration is the process of analyzing biological or biomedical articles to organize biological data into data repositories using taxonomies and ontologies. Due to the expanding number of articles and the relatively small number of biocurators, automation is desired to improve the workflow of assessing articles worth curating. As figures convey essential information, automatically integrating images may improve curation. In this work, we instantiate and evaluate a first-in-kind, hybrid image+text document search system for biocuration. The system, MouseScholar, leverages an image modality taxonomy derived in collaboration with biocurators, in addition to figure segmentation, and classifiers components as a back-end and a streamlined front-end interface to search and present document results. We formally evaluated the system with ten biocurators on a mouse genome informatics biocuration dataset and collected feedback. The results demonstrate the benefits of blending text and image information when presenting scientific articles for biocuration. 
Index Terms - document search, biocuration

MouseScholar interface.

MouseScholar: Evaluating an Image+Text Search System for Biocuration

Rayleigh-Benard (RB) convection occurs in a fluid layer when there is a temperature difference between the top and bottom boundaries, resulting in the spontaneous formation of convective motion.  In this study, a direct numerical simulation has been performed aimed at elucidating the impact of introducing dispersed droplets into a single-phase flow on the heat transfer dynamics within this flow.  The dimensionless parameters considered in this study encompass a Prandtl number of 4, a Weber number of 6000, and a Rayleigh number of 10^8.  Dispersed droplets were incorporated over a range spanning from 0 to 50%.

Effects of employing liquid-liquid emulsions on heat transfer within a turbulent Rayleigh-Benard convection

In the realm of Computational Fluid Dynamics (CFD), the demand for memory and computation resources is extreme, necessitating the use of leadership-scale computing platforms for practical domain sizes. This intensive requirement renders traditional checkpointing methods ineffective due to the significant slowdown in simulations while saving state data to disk. As we progress towards exascale and GPU-driven High-Performance Computing (HPC) and confront larger problem sizes, the choice becomes increasingly stark: to compromise data fidelity or to reduce resolution. To navigate this challenge, this study advocates for the use of in situ analysis and visualization techniques. These allow more frequent data &ldquo;snapshots&rdquo; to be taken directly from memory, thus avoiding the need for disruptive checkpointing. We detail our approach of instrumenting NekRS, a GPU-focused thermal-fluid simulation code employing the spectral element method (SEM), and describe varied in situ and in transit strategies for data rendering. Additionally, we provide concrete scientific use cases and report on runs performed on Polaris, Argonne Leadership Computing Facility’s (ALCF) 44 Petaflop supercomputer and Jülich Wizard for European Leadership Science (JUWELS) Booster, Jülich Supercomputing Centre’s (JSC) 71 Petaflop High Performance Computing (HPC) system, offering practical insight into the implications of our methodology.

Side view visualization of a RBC case.

Scaling Computational Fluid Dynamics: In Situ Visualization of NekRS using SENSEI

SAGE3, the newest and most advanced generation of the Smart Amplified Group Environment, is an open-source software designed to facilitate collaboration among scientists, researchers, students, and professionals across various fields. This tutorial aims to introduce attendees to the capabilities of SAGE3, demonstrating its ability to enhance collaboration and productivity in diverse settings, from co-located office collaboration to remote collaboration to both at once, with diverse displays, from personal laptops to large-scale display walls. Participants will learn how to effectively use SAGE3 for brainstorming, data analysis, and presentation purposes, as well as installation of private collaboration servers and development of custom applications. 
CCS CONCEPTS: Human-centered computing, Collaborative and social computing systems and tools; Open source software; Visualization systems and tools; Interactive systems and tools. 
KEYWORDS: Large Displays, Space to Think, Collaboration, Visualization, Data Science, Data Analysis, Computational Narratives

SAGE3 for Interactive Collaborative Visualization, Analysis, and Storytelling

Applications in immersive environments have gained popularity for training, learning, and recreational tasks. Due to the increasing availability of sensors and data-capturing devices, research is extending the use of immersive environments to support visual analytics processes, including sensemaking and strategic immersion for interaction and task completion. In this work, we propose Personal Situated Analytics (PSA) framework to embed users into recorded meetings with support for multiple degrees of immersion in the Reality-Virtuality spectrum. Our proposed framework encompasses various stages such as tracking, data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We evaluate this framework on a data analysis scenario between human subjects and a conversational AI agent. Our pilot study (n=12) using this framework compares user experiences when using two different devices: Hololens2 and Quest2.

A user embedded in a conversation using Personal Situated Analytics in Virtual Reality as virtual avatars.

Using Personal Situated Analytics (PSA) to Interpret Recorded Meetings

Personalized head and neck cancer therapeutics have greatly improved survival rates for patients, but are often leading to understudied long-lasting symptoms which affect quality of life. Sequential rule mining (SRM) is a promising unsupervised machine
learning method for predicting longitudinal patterns in temporal data which, however, can output many repetitive patterns that are difficult to interpret without the assistance of visual analytics. We present a data-driven, human-machine analysis visual system
developed in collaboration with SRM model builders in cancer symptom research, which facilitates mechanistic knowledge discovery in large-scale, multivariate cohort symptom data. Our system supports multivariate predictive modeling of post-treatment symptoms
based on during-treatment symptoms. It supports this goal through an SRM, clustering, aggregation back end, and a custom front
end to help develop and tune the predictive models. The system also explains the resulting predictions in the context of therapeutic
decisions typical in personalized care delivery. We evaluate the resulting models and system with an interdisciplinary group of modelers and head and neck oncology researchers. The results demonstrate that our system effectively supports clinical and symptom research. 
Index Terms: Temporal Data; Life Sciences; Mixed Initiative Human-Machine Analysis; Data Clustering and Aggregation

Longitudinal symptom analysis and prediction for head and neck patients (ICC treatment).

Roses Have Thorns: Understanding the Downside of Oncological Care Delivery Through Visual Analytics and Sequential Rule Mining

While cities around the world are looking for smart ways to channel new advances in data collection, management, and analysis to address their day-to-day problems, the complex nature of urban issues and the overwhelming amount of available structured and unstructured data have posed significant challenges in translating these efforts into actionable insights. In the past few years, urban visual analytics tools have significantly helped tackle these challenges. When analyzing a feature of interest, an urban scientist must transform, integrate and visualize different thematic (e.g., sunlight, flows) and physical (e.g., buildings, sidewalks) layers, oftentimes across multiple spatial and temporal scales. However, integrating and analyzing these layers requires expertise in different fields, increasing development time and effort. This makes the entire visual data exploration and system implementation hard for experienced programmers and also sets a high entry barrier for urban experts outside of computer science. With this in mind, in this paper, we present the Urban Toolkit (UTK), a flexible and extensible visualization framework that enables the easy authoring of web-based visualizations through a new high-level grammar specifically built with common urban use cases in mind. In order to facilitate the integration and visualization of different urban data, we also propose the concept of knots to merge thematic and physical urban layers. We evaluate our approach through use cases and a series of interviews with experts and practitioners from different domains, including urban accessibility, urban planning, architecture, and climate science. UTK is available at urbantk.org. 
Index Terms: Urban visual analytics, Urban analytics, Urban data, Visualization toolkit.

UTK abstracts spatial joins through the use of knots.

The Urban Toolkit: A Grammar-based Framework for Urban Visual Analytics

Weather forecasting is essential for decision-making and is usually performed using numerical modeling. Numerical weather models, in turn, are complex tools that require specialized training and laborious setup and are challenging even for weather experts. Moreover, weather simulations are data-intensive computations and may take hours to days to complete. When the simulation is finished, the experts face challenges analyzing its outputs, a large mass of spatiotemporal and multivariate data. From the simulation setup to the analysis of results, working with weather simulations involves several manual and error-prone steps. The complexity of the problem increases exponentially when the experts must deal with ensembles of simulations, a frequent task in their daily duties. To tackle these challenges, we propose ProWis: an interactive and provenance-oriented system to help weather experts build, manage, and analyze simulation ensembles at runtime. Our system follows a human-in-the-loop approach to enable the exploration of multiple atmospheric variables and weather scenarios. ProWis was built in close collaboration with weather experts, and we demonstrate its effectiveness by presenting two case studies of rainfall events in Brazil. 
Index Terms - Weather visualization, Ensemble visualization, Provenance management, WRF visual setup

Building, managing, and analyzing a Weather Research and Forecasting (WRF) model ensemble using PROWIS to study an extreme rainfall event in Maricá (RJ, Brazil) in April 2022.

ProWis: A visual approach for building, managing, and analyzing weather simulation ensembles in runtime

New web technologies have enabled the deployment of powerful GPU-based computational pipelines that run entirely in the web browser, opening a new frontier for accessible scientific visualization applications. However, these new capabilities do not address the memory constraints of lightweight end-user devices encountered when attempting to visualize the massive data sets produced by today&rsquo;s simulations and data acquisition systems. In this paper, we propose a novel implicit isosurface rendering algorithm for interactive visualization of massive volumes within a small memory footprint. We achieve this by progressively traversing a wavefront of rays through the volume and decompressing blocks of the data on-demand to perform implicit ray-isosurface intersections. The progressively rendered surface is displayed after each pass to improve interactivity. Furthermore, to accelerate rendering and increase GPU utilization, we introduce speculative ray-block intersection into our algorithm, where additional blocks are traversed and intersected speculatively along rays as other rays terminate to exploit additional parallelism in the workload. Our entire pipeline is run in parallel on the GPU to leverage the parallel computing power that is available even on lightweight end-user devices. We compare our algorithm to the state of the art in low-overhead isosurface extraction and demonstrate that it achieves 1.7×–5.7× reductions in memory overhead and up to 8.4× reductions in data decompressed.

Interactive full-resolution isosurface visualization of the 2048×2048×1920 Richtmyer-Meshkov (R-M) data set in the browser.

Speculative Progressive Raycasting for Memory Constrained Isosurface Visualization of Massive Volumes

Using immersive environments for training, education, and recreation is becoming increasingly popular among individuals. In recent years these environments have also been used to facilitate data analytics processes involving interaction and task completion. In this work, we discuss Personal Situated Analytics (PSA) a cross-platform framework to embed users into recorded conversations with support for multiple degrees of immersion in the Reality-Virtuality spectrum. Our development pipeline consisted of several stages: tracking, data capturing, data cleaning, data synchronization, prototype building, and deploying the final product to end-user hardware. We evaluate PSA by analyzing data from a recorded meeting involving human participants and a visual conversational AI agent. Our pilot study (n = 12) using this framework compares user experiences for MR and VR environments. 
Index Terms: Immersive Analytics, Situated Analytics, Visualization techniques, Human-centered computing

PSA is an immersive MR/VR framework for experiencing and analyzing recorded conversations and events. We see two participants represented as virtual avatars engaged in a conversation.

PSA: A Cross-Platform Framework for Situated Analytics in MR and VR

Estimating the human posture from an image or a video is a fundamental task in computer vision. It not only enhances other vision tasks like action recognition, person re-identification, and virtual try-on but also facilitates many real-world applications including robotics, healthcare, sports, and retail. An effective and efficient human pose estimation system can enable robots to acquire skills from demonstrations, support physical therapists in diagnosing and rehabilitating patients, assist sports analysts and coaches in tracking and training athletes, and empower retailers to establish employee-free stores. Thanks to the advancement of deep learning and the availability of large-scale datasets, the performance of state-of-the-art human pose estimation methods has drastically improved in recent years. These approaches can accurately estimate postures in daily activities and sports. However, several challenges persist. For example, (1) it remains challenging to estimate postures that occur rarely or are entirely absent in the training data; (2) handling complex scenarios, such as crowded environments, motion blur, low-light conditions, and occlusions, is still a formidable task; (3) there is a growing need to develop efficient models that can estimate human poses in real-time or on low-power devices; (4) exploring novel applications for human pose estimation that can bring societal benefits or transform industries is an exciting avenue of research. This special issue presents a collection of high-quality papers on human pose estimation and its applications. They cover a wide range of topics, from fundamental human pose models with improved accuracy and efficiency to applications in human behavior understanding and assessment, sign language recognition, and multi-camera calibration. Through this special issue, we would like to open the path toward more discussion between practitioners and researchers in this very important yet challenging field of human pose estimation.

Guest editorial: special issue on human pose estimation and its applications

Semi-Supervised Temporal Action Localization (SS-TAL) aims to improve the generalization ability of action detectors with large-scale unlabeled videos. Albeit the recent advancement, one of the major challenges still remains: noisy pseudo labels hinder efficient learning on abundant unlabeled videos, embodied as location biases and category errors. In this paper, we dive deep into such an important but understudied dilemma. To this end, we propose a unified framework, termed Noisy Pseudo-Label Learning, to handle both location biases and category errors. Specifically, our method is featured with (1) Noisy Label Ranking to rank pseudo labels based on the semantic confidence and boundary reliability, (2) Noisy Label Filtering to address the class-imbalance problem of pseudo labels caused by category errors, (3) Noisy Label Learning to penalize inconsistent boundary predictions to achieve noise-tolerant learning for heavy location biases. As a result, our method could effectively handle the label noise problem and improve the utilization of a large amount of unlabeled videos. Extensive experiments on THUMOS14 and ActivityNet v1.3 demonstrate the effectiveness of our method. The code is available at github.com/kunnxia/NPL.

Learning from noisy pseudo labels for semi-supervised temporal action localization

* Authors: 
Belcaid, M., Leigh, J., Theriot, R., Kirshenbaum, N., Tabalba, R., Rogers, M. University of Hawai’i at Manoa, Honolulu, HI 
Johnson, A., Brown, M., Renambot, L., Long, L., Nishimoto, A., University of Illinois Chicago, Chicago, IL 
North C., Harden, J., Virginia Tech, Blacksburg, VA 
Translational software research bridges the gap between scientific innovations and practical applications, driving impactful societal advancements. However, developing such software is challenging due to interdisciplinary collaboration, technology adoption, and post-funding sustainability. This article presents the experiences and insights of the Scalable Adaptive Graphics Environment (SAGE) team, which has spent two decades developing translational, cross-disciplinary, collaboration tools to benefit computational science research. With a focus on SAGE and its next-generation iterations, we explore the inherent challenges in translational research, such as fostering cross-disciplinary collaboration, motivating technology adoption, and ensuring post-funding product sustainability. We also discuss the roles of funding agencies, policymakers, and academic institutions in promoting translational research. Although the journey is fraught with challenges, the societal impact and satisfaction derived from translational research underscore its significance in the broader scientific landscape. This article aims to encourage further conversation and the development of effective models for
translational software projects.

The process that defines domain science, computer science (CS), and sustainable translational research.

Reflecting on the Scalable Adaptive Graphics Environment Team’s 20-Year Translational Research Endeavor in Digital Collaboration Tools

Recent advances in precision medicine for treating Oropharyngeal cancer (OPC) have led to significant improvements in survival for patients. However, OPC survivors often are left with severe side effects that result from damage to vital tissues in the head and neck caused by secondary radiation received during treatment. These side effects can persist for months after treatment cessation, and diminish the patients’ quality of life. Existing guidelines for radiation therapy plans identify dose limits to individual organs with the goal to avoid or minimize permanent tissue damage and their related toxicity. However, existing tools fail to account for either the three-dimensional nature of dose distributions or the fact that critical organs often work as a system that can have multiple failure points. 
We present an unsupervised approach for stratifying patients based on three-dimensional dose distributions to multiple organs and evaluate their association with long-term patient-reported symptoms. We apply this method to predicting severe patient-reported symptoms 6 months after ceasing radiation therapy for OPC patients. For actionable interpretability, we introduce a rule-mining method that simplifies our strata into a set of dose thresholds, and demonstrate that these thresholds outperform existing dose treatment guidelines. 
Keywords: Radiation therapy; Oropharynx cancer; Head and Neck cancer; Stratification; Symptom Burden; Quality of Life

Results of a chi-squared test between covariates and membership in each set of clusters for each outcome.

Multi-Organ Spatial Stratification of 3-D Dose Distributions Improves Risk Prediction of Long-Term Self-Reported Severe Symptoms in Oropharyngeal Cancer Patients Receiving Radiotherapy: Development of a Pre-Treatment Decision Support Tool.

Motivation: Figures in biomedical papers communicate essential information with the potential to identify relevant documents in biomedical and clinical settings. However, academic search interfaces mainly search over text fields. 
Results: We describe a search system for biomedical documents that leverages image modalities and an existing index server. We integrate a problem-specific taxonomy of image modalities and image-based data into a custom search system. Our solution features a front-end interface to enhance classical document search results with image-related data, including page thumbnails, figures, captions, and image-modality information. We demonstrate the system on a subset of the CORD-19 document collection. A quantitative evaluation demonstrates higher precision and recall for biomedical document retrieval. A qualitative evaluation with domain experts further highlights our solution’s benefits to biomedical search. 
Availability and implementation: A demonstration is available at <a href="https://runachay.evl.uic.edu/scholar">https://runachay.evl.uic.edu/scholar</a>. Our code and image models can be accessed via <a href="https://github.com/uic-evl/bio-search">https://github.com/uic-evl/bio-search</a>. The dataset is continuously expanded. 
Image caption: Search interface displaying the results of a boolean query. (A) The color-coded legend displays the image modalities in the first level of the taxonomy. (B) The search bar, with a keyword box showing a boolean query, a time filter, and an image modalities filter limiting results to documents containing \textit{radiology} or \textit{fluorescence microscopy} figures. (C) The metadata of a result includes the document title, year, venue, authors, and the hits on the abstract and full text. Below is the count of figures and modalities in the document. (D) The enhanced document surrogate shows the image content, including the page thumbnail, extracted figure, subfigures and modalities, and caption. 
<a href="https://doi.org/10.1093/bioadv/vbad095">https://doi.org/10.1093/bioadv/vbad095</a>

Search interface displaying the results of a boolean query.

Enhancing biomedical search interfaces with images

Objective: Evaluate the effectiveness of machine learning tools that incorporate spatial information such as disease location and lymph node metastatic patterns-of-spread, for prediction of survival and toxicity in HPV+ oropharyngeal cancer (OPC). 
Materials & methods: 675 HPV+ OPC patients that were treated at MD Anderson Cancer Center between 2005 and 2013 with curative intent IMRT were retrospectively collected under IRB approval. Risk stratifications incorporating patient radiometric data and lymph node metastasis patterns via an anatomically-adjacent representation with hierarchical clustering were identified. These clusterings were combined into a 3-level patient stratification and included along with other known clinical features in a Cox model for predicting survival outcomes, and logistic regression for toxicity, using independent subsets for training and validation. 
Results: Four groups were identified and combined into a 3-level stratification. The inclusion of patient stratification in predictive models for 5-yr Overall survival (OS), 5-year recurrence free survival, (RFS) and Radiation associated dysphagia (RAD) consistently improved model performance measured using the area under the curve (AUC). Test set AUC improvements over models with clinical covariates, was 9 % for predicting OS, and 18 % for predicting RFS, and 7 % for predicting RAD. For models with both clinical and AJCC covariates, AUC improvement was 7 %, 9 %, and 2 % for OS, RFS, and RAD, respectively. 
Conclusion: Including data-driven patient stratifications considerably improve prognosis for survival and toxicity outcomes over the performance achieved by clinical staging and clinical covariates alone. These stratification generalize well to across cohorts, and sufficient information for reproducing these clusters is included. 
Keywords: Head & Neck cancer, Statistical and research methods, Machine learning, Radiation oncology, Health Informatics, Imaging, Risk stratification, Oropharyngeal cancer, Clustering, Radiomics

Average lymph node involvement and clinical staging categories of the 3 derived lymph node + radiomics clusters across both the training and validation datasets.

Spatially-aware clustering improves AJCC-8 risk stratification performance in oropharyngeal carcinomas

Honorable Mention in the EuroVis Best Paper Category 
Exploratory analysis of the chemical space is an important task in the field of cheminformatics. For example, in drug discovery research, chemists investigate sets of thousands of chemical compounds in order to identify novel yet structurally similar synthetic compounds to replace natural products. Manually exploring the chemical space inhabited by all possible molecules and chemical compounds is impractical, and therefore presents a challenge. To fill this gap, we present ChemoGraph, a novel visual analytics technique for interactively exploring related chemicals. In ChemoGraph, we formalize a chemical space as a hypergraph and apply novel machine-learning models to compute related chemical compounds. It uses a database to find related compounds from a known space and a machine learning model to generate new ones, which helps enlarge the known space. Moreover, ChemoGraph highlights interactive features that support users in viewing, comparing, and organizing computationally identified related chemicals. With a drug discovery usage scenario and initial expert feedback from a case study, we demonstrate the usefulness of ChemoGraph. 
Keywords: chemical space exploration, cheminformatics, multipartite graphs, data visualization 
CCS Concepts: Applied computing → Chemistry 
<iframe width="560" height="315" src="https://www.youtube.com/embed/kVpz6TDjQMw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

An overview of ChemoGraph.

ChemoGraph: Interactive Visual Exploration of the Chemical Space

Most real-world networks are both dynamic and multivariate in nature, meaning that the network is associated with various attributes and both the network structure and attributes evolve over time. Visualizing dynamic multivariate networks is of great significance to the visualization community because of their wide applications across multiple domains. However, it remains challenging because the techniques should focus on representing the network structure, attributes, and their evolution concurrently. Many real-world network analysis tasks require the concurrent usage of the three aspects of the dynamic multivariate networks. In this paper, we analyze current techniques and present a taxonomy to classify the existing visualization techniques based on three aspects: temporal encoding, topology encoding, and attribute encoding. Finally, we survey application areas and evaluation methods; and discuss challenges for future research. 
CCS Concepts Human-centered computing → Graph drawings