V-NeuroStack is a web application which uses 3-D time stack approach to identify patterns in the neuronal activity and to understand functional relationships between neurons. Our research group aims to understand the auditory corticothalamic activity in the mouse. We used GCaMP6s calcium imaging to capture spontaneous activity of cortical neurons. V-NeuroStack uses a point cloud to visualize the behavior of neurons during spontaneous activity while also having the capability to drill down on a neuron at a single timestep. In V-NeuroStack we assemble the 2-D movie frame data captured during spontaneous activity to visualize it as a 3-D time stack that the user can manipulate using given controls. Additionally, for every cross-sectional frame of the 3-D time stack, we generate a 2-D projection of the data at any time step which may then be explored for detailed analysis of the activity. A line graph depicting the intensities over the entire set of timesteps can be generated by clicking on a neuron in the 2-D projection. Button and slider controls can be used to change the portion of data used for visualization and a user interactive pointing device can be used to manipulate the 3-D time stack. Multiple data files are fed to our application: a timeseries file contains calcium intensities for each neuron at each timestep, an XYcoordinates file contains the location of each neuron on the image. Finally a file for each timestep identifies clusters of like-responding neurons; a Python script is used to extract the color associated with each cluster for the visualization. To generate the point cloud, V-NeuroStack uses the same x and y value for each neuron and populates a new z value to stack the neuronal activity in time. The result is a tube-like structure generated for each neuron for some or all available timesteps in the data. V-NeuroStack is written in JavaScript. In conclusion, we have implemented a tool to view patterns during spontaneous activity in mouse brain tissue that cannot be captured using a single 2-D view of the data. A visible pattern may either show like-colored neurons that belong to a particular cluster during a certain period of the activity or help in identifying strong correlation between the neurons for a given set of timesteps. An accompanying 2D image for any timestep can be viewed to gain greater detail of the neuronal activity. This flexibility in exploring patterns in firing of neurons has the potential to open up further research questions on understanding spontaneous as well as other forms of neuronal activity in any brain region.

V NeuroStack Application Shows time stacks for 139 neurons, with controls, cross section of one timestep and intensities for Cell 1.

V-NeuroStack: 3-D time stacks for finding patterns in spontaneous activity of neurons in mouse brain slice

Animal behavior research is becoming an increasingly data-driven field, enabled by advancements in GPS tracking. Rather than directly observe movement and behavior in small numbers of animals, over short time-spans and in small areas, researchers can use GPS collars to track many animals, over large areas and long time spans. These large datasets have the potential to provide rich information about animal behavior. However, this tracking data needs to be integrated with geospatial data about the environment in order to put the movements and estimated behaviors into context. We present an immersive visualization which integrates high resolution topological data from Barro Colorado Island, a 4000 acre island located in the Panama Canal, with GPS tracking data from close to two dozen primates captured over several months. Our application leverages parallelization for rapid filtering of the movement data, allowing researchers to explore the data in a large-scale, immersive environment (CAVE2). We present this work in order to explore the possibility of creating virtual field environments from data, to bring the field into the lab and enable big data animal behavior research. 
Index Terms: Human-centered computing, Visualization, Visualization application domains, Scientific visualization 
<a href="https://doi.org/10.1109/LDAV48142.2019.8944350">DOI: 10.1109/LDAV48142.2019.8944350</a>
Support provided by National Science Foundation award CNS-1456638.

An aerial view of the island with the point cloud canopy.

Bringing the Field into the Lab: Large-Scale Visualization of Animal Movement Trajectories within a Virtual Island

There is a steadily growing interest in leveraging ecosystems of digital devices that go beyond a single desktop for visual data analysis and exploration. However, multi-device ecologies pose several challenges as information and tasks are scattered among separate devices and displays. To understand the challenges associated with multi-device environments for information visualization, we performed an exploratory study designed to examine how users employ different tools to perform different kinds of activities in approaching a visual analysis task. Previous work examined three factors (users, tools, and tasks) independently. We study the synthesis of these factors. To do this, we adopted a hybrid analysis approach that focuses on three different aspects: users, tools, and tasks. We believe this analysis will help us identify associated challenges and better inform design goals in developing multi-device tools for visual data analysis. 
Keywords: Collaborative visual analytics, exploratory analysis, multi-device ecology

Illustration of the study setup.

Towards Understanding Collaborative Visual Data Analysis in Multi-Device Environments

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE&trade; and the LCD-based successor CAVE2&trade;. Each of these has distinct advantages and limitations based on how they&rsquo;re used. This work explores preserving the high resolution and sense of presence of CAVE2 environments in full stereoscopic mode by using a see-through augmented reality HMD to expand the user&rsquo;s field of regard beyond the physical display walls. In our explorative study, we found that in a visual search task in a stereoscopic CAVE2, the addition of the HoloLens to expand the field of regard did not hinder the performance or accuracy of the participant, but promoted more physical navigation which in post-study interviews participants felt aided in their spatial awareness of the virtual environment. 
CCS CONCEPTS: Human-centered computing, Human computer interaction (HCI); Mixed / augmented reality; Virtual reality. 
KEYWORDS; Augmented Reality; Virtual Reality; Display Wall; Immersive Displays; Presence

HoloLens view inside CAVE2

Extending Virtual Reality Display Wall Environments Using Augmented Reality

In today&rsquo;s Big Data era, data scientists require new computational instruments in order to quickly analyze large-scale datasets using complex codes and quicken the rate of scientific progress. While Federally-funded computer resources, from supercomputers to clouds, are beneficial, they are often limiting - particularly for deep learning and visualization - as they have few Graphics Processing Units (GPUs). GPUs are at the center of modern high-performance computing and artificial intelligence, efficiently performing mathematical operations that can be massively parallelized, speeding up codes used for deep learning, visualization and image processing, more so than general-purpose microprocessors, or Central Processing Units (CPUs). The University of Illinois at Chicago is acquiring a much-in-demand GPU-based instrument, COMPaaS DLV - COMposable Platform as a Service Instrument for Deep Learning &amp; Visualization, based on composable infrastructure, an advanced architecture that disaggregates the underlying compute, storage, and network resources for scaling needs, but operates as a single cohesive infrastructure for management and workload purposes. We are experimenting with a small system and learning a great deal about composability, and we believe COMPaaS DLV users will benefit from the varied workflow that composable infrastructure allows. 
Keywords: distributed systems, testbed implementation &amp; deployment, composable infrastructure, deep learning, visualization

COMPaaS DLV: Composable Infrastructure for Deep Learning in an Academic Research Environment

Multi-device environments present new opportunities for collaborative visual data analysis and sense-making by utilizing each device&rsquo;s strengths and capabilities. However, one of the associated challenges with visual data analysis in multi-device environments is the sharing of visual components across devices. We present a framework developed on top of SAGE2 platform for cross-device collaborative visual data exploration. As part of our framework, we contribute to the concept of rapid development and assembling of visualizations that can span multiple devices of different modalities. It provides the users with an environment for visualization compositions that delegate the rendering to the target device, allowing them to augment their large display workspace with portable devices for further exploration territories. Facilitated by its intuitive visualization composition pipeline, users with no programming skills such as data analysts can enhance their analytical scope with no coding barriers. We describe the framework, its implementation with a use case, and the rationale behind its design. 
Keywords: Collaboration, Multi-device environment, Visual Data Analysis

PolyVis: Cross-Device Framework for Collaborative Visual Data Analysis

SAGE (the Scalable Adaptive Graphics Environment) and its successor SAGE2 (the Scalable Amplified Group Environment) are operating systems for managing content across wideband display environments. This paper documents the prevalent usage patterns of SAGE-enabled display walls in support of the e-Science enterprise, based on nearly 15 years of observations of the SAGE community. These patterns will help guide e-Science users and cyberinfrastructure developers on how best to leverage large tiled display walls, and the types of software services that could be provided in the future. 
Keywords: tiled display wall, immersive analytics, visualization, human-centered computing, computer-supported cooperative work

The e-Science Research, Development and Training Enterprise.

Usage Patterns of Wideband Display Environments In e-Science Research, Development and Training

When speaking to second language learners, talkers often reduce their rate of speech to assist their listeners&rsquo; understanding and comprehension. This study grants English as a Second Language subjects fine-grained, real-time control over the playback rates of lengthy audio tracks of conversational speech, and tests the subjects&rsquo; listening comprehension at their desired playback speeds and at unmodified speeds. We find evidence that slower playback rates are preferred, but no evidence that such playback rates affect listener comprehension. 
Index Terms: Perception of prosody; ESL; human-computer interaction

The Effects of Time Expansion on English as a Second Language Individuals

In head and neck (HN) cancer treatment, radiotherapy (RT) plans assigned to a given patient depend in large measure on the tumor location with respect to surrounding organs at risk (OARs). With the emergence of Big Data repositories, RT data from patients with
similar tumor locations could be used to predict the RT plan for a new patient, as opposed to clinicians or institutional memory alone. 
We propose an automated technique to evaluate RT plan similarity between HN cancer patients who have previously received radiotherapy, based on similarities in their tumor location with respect to OARs. We use this technique to predict and evaluate the RT dose distribution across organs in a new HN patient, using similar patients as a prior. 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Cohort-Based Spatial Similarity Can Predict Radiotherapy Dose Distribution

Imaging is an integral tool in radiation oncology, starting from initial diagnosis to follow-up post-therapy. In head and neck
cancer treatment, toxicity and tumor control depend on the tumor locational geometry and systematic predictable spread of disease to
affected lymph nodes (LN) levels. However, a rigorous methodology for integrating imaging spatial information into treatment outcome
prediction models remains an unmet need. We hypothesized that integrating tumor and LN spatial information derived from routine diagnostic images can aid in prediction model development for both oncologic outcomes and post-therapy toxicity. 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Correlating Toxicity Outcomes with Spatial Patterns of Lymph Node Metastasis for Oropharyngeal Cancer Patients

Objectives: We sought to model chronic radiation-associated dysphagia (RAD) in patients given intensity-modulated radiation therapy (IMRT) for oropharyngeal squamous cell cancer (OPSCC) as a function of age and dose to non-target swallowing muscles. 
Methods: We reviewed 300 patients with T1-T4 N0-3 M0 OPSCC given definitive IMRT with concurrent chemotherapy. Chronic RAD was defined as aspiration or stricture on videoflouroscopy/endoscopy, gastrostomy tube, or aspiration pneumonia at >=12 months after IMRT. Doses to auto segmented regions of interest (ROIs; inferior, middle and superior constrictors, anterior and posterior digastrics, mylo/geniohyoid complex, intrinsic tongue, and gengioglossus) were obtained from DICOM-RT plans and dose-volume histograms. The probability of chronic RAD as a function of mean ROI dose, stratified by age (<50, 50–59, 60–69, or >=70 years), was estimated with logistic probability models and subsequent unsupervised nonlinear curves. 
Results: Chronic RAD was observed in 34 patients (11%). Age was a significant correlate of chronic RAD, both independently and with dose for all muscle groups examined. Distinct muscle-specific dose-response profiles were observed as a function of age (e.g., 5% of patients in their 50 s [but 20% of those 70 + ] who received 60 Gy to the superior constrictor had chronic RAD). This effect was stable across all observed muscle ROIs, with a false discovery rate-corrected p < 0.05, for all dose/muscle/age models, suggesting that including age as a covariate improves modeling of chronic RAD.<bt> 
Conclusions: Age at treatment moderates the probability of chronic RAD after chemo-IMRT for OPSCC, with aging muscles showing lower dose thresholds. Uniform dose constraints may not predict toxicity in older patients. 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Example of auto-segmented ROIs on axial, sagittal, and coronal views with dose wash overlay.

Chronic Radiation-Associated Dysphagia in Oropharyngeal Cancer Survivors: Towards Age-Adjusted Dose Constraints for Deglutitive Muscles

We describe a visual computing approach to radiation therapy (RT) planning, based on spatial similarity within a patient cohort. In radiotherapy for head and neck cancer treatment, dosage to organs at risk surrounding a tumor is a large cause of treatment toxicity. Along with the availability of patient repositories, this situation has lead to clinician interest in understanding and predicting RT outcomes based on previously treated similar patients. To enable this type of analysis, we introduce a novel topology-based spatial similarity measure, T-SSIM, and a predictive algorithm based on this similarity measure. We couple the algorithm with a visual steering interface that intertwines visual encodings for the spatial data and statistical results, including a novel parallel-marker encoding that is spatially aware. We report quantitative results on a cohort of 165 patients, as well as a qualitative evaluation with domain experts in radiation oncology, data management, biostatistics, and medical imaging, who are collaborating remotely. 
Keywords: Biomedical and Medical Visualization, Spatial Techniques, Visual Design, High-Dimensional Data 
Funding: NIH Award #NCI-R01CA225190: QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data (Marai, PI) 
NIH, National Cancer Institute #1R01CA214825-01: SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment (Marai, PI)

Screen shot of visual interface for cohort-based radiation therapy (RT) prediction results.

Cohort-based T-SSIM Visual Computing for Radiation Therapy Prediction and Exploration

Background: In virtual reality (VR) applications such as games, virtual training, and interactive neurorehabilitation, one can employ either the first-person user perspective or the third-person perspective to perceive the virtual environment; however, applications rarely offer both perspectives for the same task. We used a targeted-reaching task in a large-scale virtual reality environment (N = 30 healthy volunteers) to evaluate the effects of user perspective on the head and upper extremity movements, and on user performance. We further evaluated how different cognitive challenges would modulate these effects. Finally, we obtained the user-reported engagement level under the different perspectives. 
Results: We found that first-person perspective resulted in larger head movements (3.52 ± 1.3m) than the third-person perspective (2.41 ± 0.7m). First-person perspective also resulted in more upper-extremity movement (30.08 ± 7.28m compared to 26.66 ± 4.86m) and longer completion times (61.3 ± 16.4s compared to 53 ± 10.4s) for more challenging tasks such as the &ldquo;flipped mode&rdquo;, in which moving one arm causes the opposite virtual arm to move. We observed no significant effect of user perspective alone on the success rate. Subjects reported experiencing roughly the same level of engagement in both first-person and third-person perspectives (F(1.58) = 0.9, P = .445). 
Conclusion: User perspective and its interaction with higher-cognitive load tasks influences the extent of movement and user performance in a virtual theater environment, and may influence the choice of the interface type (first or third person) in immersive training depending on the user conditions and exercise requirements. 
Keywords: User perspective, Engagement, CAVE2

View of the CAVE2, displaying the third-person perspective of a task. 72 high-resolution displays are arranged in 18 columns in a circular fashion, providing an area of 320 degrees of screen space with a resolution of 36 Megapixels for each user eye.

User perspective and higher cognitive task-loads influence movement and performance in immersive training environments

Promoting a wider range of contribution types can facilitate healthy growth of the visualization community, while increasing the intellectual diversity of visualization research papers. In this article, we discuss the importance of contribution types and summarize contribution types that can be meaningful in visualization research. We also propose several concrete next steps we can and should take to ensure a successful launch of the contribution types. 
We as a visualization research community increasingly see the need for and use of visualization not only for exploring or analyzing data, but also for communicating insights extracted from the data or for explaining concepts and phenomena in the real world. However, we seem to be stuck in a narrow scope when evaluating contributions that a visualization research project can and should
make. For example, papers presenting novel interaction techniques for visualization often appear in non-visualization conferences after (or with the fear of) being turned away from visualization venues, being labeled as &ldquo;out of scope.&rdquo; We suggest that the current lens of five paper types used by several top visualization conferences is a narrow view of possible research contributions that hinders us from demonstrating and realizing the full potential of visualization research. It may well be inhospitable to the people from other research communities or industries who successfully leverage visualizations in their research and practice. 
In this article, we discuss a broader range of contribution types than is currently accepted, with aims to promote healthy growth of the visualization community, to increase intellectual diversity, and to better demonstrate the value and impact of visualization research. We fully acknowledge the value of the visualization paper types, particularly with respect to the paper writing and reviewing process. However, we also note that paper types are frequently applied as prescriptive: they tend to specify much of the paper structure and may not be well-suited to papers with less common contributions (e.g., those outlining new problem spaces) or papers that do not comply with the existing structures (e.g., practitioner-oriented applications). We assert that the lens of contribution types can give a broader scope to the types of papers that are considered acceptable (e.g., exciting and emerging research that does not fit exactly into a paper type) and can offer more transient and flexible criteria to evaluate visualization research. 
After discussing the importance of contribution types, we describe five examples of contribution types that are rarely seen in visualization conferences. We offer the descriptions of these contribution types along with example papers, as well as a broader collection of contribution types that are seen more widely. We also argue that looking at visualization research through the contribution type lens would support the intellectual evolution and expansion of our community, opening up the possibility of seeing new contribution types we may not yet have imagined. We conclude with the discussion of concrete next steps the visualization community can take to ensure a successful implementation and application of the research contribution types.

Broadening Intellectual Diversity in Visualization Research Papers

Biological network figures are ubiquitous in the biology and medical literature. On the one hand, a good network figure can quickly provide information about the nature and degree of interactions between items, and enable inferences about the reason for those
interactions. On the other hand, good network figures are difficult to create. In this paper, we outline 10 simple rules for creating biological network figures for communication, from choosing layouts, to applying color or other channels to show attributes, to the use of layering and separation. These rules are accompanied by illustrative examples. We also provide a concise set of references and additional resources for each rule.

Two representations of proteins involved in glioblastoma multiforme (GBM).

10 simple rules to create biological network figures for communication

Research has shown that compliance to interactive training in tasks such as movement therapy following injuries, stroke and major surgeries is sub-optimal. Efforts are being made to make the rehabilitation process a more engaging experience to motivate patients to adhere to therapy. The use of games and virtual reality has gained interest among researchers and clinicians as it may cost less than clinical therapy and can be made portable for use at home. However, without the encouragement of a physical therapist, motivating patients may be difficult. In this paper, we propose a virtual slot machine game and hypothesize that it can potentially make individuals increase time devoted to therapy. Since this game requires minimal skill for gameplay, patients can simply reach a target to trigger a reel spin. We propose that the anticipation and uncertainty of this game will capture patients’ interest and motivate them to play longer. This preliminary study with able-bodied individuals may be used to tune the testing designs for future efforts on patient populations that need to repetitively practice arm movements. 
Keywords - rehabilitation, exercises, games, slot machines

A user testing the game and the interface.

Virtual Slots Game for Rehabilitation Exercises

This research investigates a novel approach to supporting classroom learning communities through the use of proxemic interaction and ambient visualizations. Specifically, community knowledge is embedded within the physical space of the classroom, with the aim of mediating opportunistic inter-group interactions, instigated through proximity and shared artifacts. This approach entails decomposing the community knowledge-base into a collection of independent thematic sub-stores, and then conceptually distributing those sub-stores to mapped, demarcated locations around the classroom, called &ldquo;Knowledge Places&rdquo;. This necessitates physical movement among and proximity to those places in order for students to contribute to or otherwise access their peers&rsquo; contributions to the emerging knowledge-base. The present research studies the materialization of Knowledge Places over the course of ten weeks within a sixth-grade life science curriculum, with topics of food webs and ecosystems. 
Keywords: Embodied Interaction, proxemic Interaction, ambient visualization, RFID, indoor location, Ubiquitous Computing; Internet Of Things; IOT 
ISBN: 978-1-4503-6690-8

Students composing contributions at ecosystems on group tablets.

Knowledge Places: Embedding Knowledge in the Space of the Classroom

Through the use of open data portals, cities, districts and countries are increasingly making available energy consumption data. These data have the potential to inform both policymakers and local communities. At the same time, however, these datasets are large and complicated to analyze. We present the activity-centered-design, from requirements to evaluation, of a web-based visual
analysis tool to explore energy consumption in Chicago. The resulting application integrates energy consumption data and census data, making it possible for both amateurs and experts to analyze disaggregated datasets at multiple levels of spatial aggregation and to compare temporal and spatial differences. An evaluation through case studies and qualitative feedback demonstrates that this visual analysis application successfully meets the goals of integrating large, disaggregated urban energy consumption datasets and of supporting analysis by both lay users and experts. 
Keywords: Interactive Visualization; Visual Design; Sustainability

Interactive, web-based, open-source energy consumption explorer for the city of Chicago.

Visual Analysis of a Smart City&rsquo;s Energy Consumption

This article provides a 25 year-long perspective on Immersive Analytics through the lens of first-in-kind technological advancements introduced at the Electronic Visualization Laboratory, University of Illinois at Chicago, along with the challenges and lessons learned from multiple Immersive Analytics projects. 
The Electronic Visualization Lab (EVL) at the University of Illinois at Chicago has been at the forefront of Virtual Reality research since 1992, when it introduced the CAVE&trade; Automatic Virtual Environment, the first projection-based virtual reality system in the world. Since then, the lab has developed a wide range of immersive systems and technologies to be used for visualization and visual analysis tasks, such as PARIS&trade;, ImmersaDesk&trade;, CAVE2&trade;, and SAGE2&trade;. These technologies paved the way for the resurgence in virtual reality in the past few years, and lead to the introduction of Immersive Analytics - which investigates how new interaction
and display technologies can be used to support analytical reasoning by immersing the users in data.

CAVE2 immersive exploration of dark matter formation in the universe. While little information can be shown on a typical desktop display, the high-resolution tiled environment lets analysts examine details without losing the context of the larger dataset.

Immersive Analytics Lessons from the Electronic Visualization Laboratory: a 25 Year Perspective

Animation studios render 3D scenes using a technique called path tracing which enables them to create high quality photorealistic frames. Path tracing involves shooting 1000&rsquo;s of rays into a pixel randomly (Monte Carlo) which will then hit the objects in the scene and, based on the reflective property of the object, these rays reflect or refract or get absorbed. The colors returned by these rays are averaged to determine the color of the pixel. This process is repeated for all the pixels. Due to the computational complexity it might take 8-16 hours to render a single frame. We implemented a neural network-based solution to reduce the time it takes to render a frame to less than a second using a generative adversarial network (GAN), once the network is trained. The main idea behind this proposed method is to render the image using a much smaller number of samples per pixel than is normal for path tracing (e.g., 1, 4, or 8 samples instead of, say, 32,000 samples) and then pass the noisy, incompletely rendered image to our network, which is capable of generating a high-quality photorealistic image. 
Keywords: image denoising, rendering, deep learning

Gaussian noise image (left), denoised image (middle) and ground truth photorealistic image (right).

Image Denoising Using A Generative Adversarial Network

Hybrid virtual reality environments allow analysts to choose how much of the screen real estate they want to use for Virtual Reality (VR) immersion, and how much they want to use for displaying different types of 2D data. We present the use-based design and evaluation of an immersive visual analytics application for cosmological data that uses such a 2D/3D hybrid environment. The application is a first-in-kind immersive instantiation of the Activity-Centered-Design theoretical paradigm, as well as a first documented immersive instantiation of a details-first paradigm based on scientific workflow theory. Based on a rigorous analysis of the user activities and on a details-first paradigm, the application was designed to allow multiple domain experts to interactively analyze visual representations of spatial (3D) and nonspatial (2D) cosmology data pertaining to dark matter formation. These hybrid data are represented at multiple spatiotemporal scales as time-aligned merger trees, pixel-based heatmaps, GPU-accelerated point clouds and geometric primitives, which can further be animated according to simulation data and played back for analysis. We have demonstrated this multi-scale application to several groups of lay users and domain experts, as well as to two senior domain experts from the Adler Planetarium, who have significant experience in immersive environments. Their collective feedback shows that this hybrid, immersive application can assist researchers in the interactive visual analysis of large-scale cosmological simulation data while overcoming navigation limitations of desktop visualizations. 
Keywords: immersive analytics, virtual reality environments, hybrid environments, astronomy, data visualization 
Funding: NSF CAREER IIS-1541277, NSF CNS-1625941, CRA-W REU

Collaboratively examining in an immersive environment the formation of dark matter halos.

DarkSky Halos: Use-Based Exploration of Dark Matter Formation Data in a Hybrid Immersive Virtual Environment

The most common sensor arrangement of 360 panoramic video cameras is a radial design where a number of sensors are outward looking as in spokes on a wheel. The cameras are typically spaced at approximately human interocular distance with high overlap. We present a novel method of leveraging small form-factor camera units arranged in stereo pairs and interleaved to achieve a fully panoramic view with fully parallel sensor pairs. This arrangement requires less keystone correction to get depth information and the discontinuity between images that have to be stitched together is smaller than in the radial design. The primary benefit for this arrangement is the small form factor of the system with the large number of sensors enabling a high resolving power. We highlight mechanical considerations, system performance and software capabilities of these manufactured and tested imaging units. One is based on the Raspberry Pi cameras and a second based on a 16 camera system leveraging 8 pairs of 13-megapixel AR1335 cell phone sensors. In addition, several different variations on the conceptual design were simulated with synthetic projections to compare stitching difficulty of the rendered scenes.

(a) StarCAM on cart with calibration target, (b) StarCAM capture infrastructure, (c) StarCAM capture computer

StarCAM - A 16K stereo panoramic video camera with a novel parallel interleaved arrangement of sensors

Without considering the pose of the viewer there are problems with displaying stereo correctly at the top and bottom poles of a 4 Pi steradian panorama. This has led to a commonly held position that one cannot handle stereo adequately at the poles of a panorama. In the tracked case, where the pose of one viewer&rsquo;s head is known, it is possible to improve the stereo experience for that viewer utilizing variations on standard VR display methods. It is also possible to display an approximately correct projection without knowing the viewer&rsquo;s pose. There are practical issues with the projections used and the camera placement and stitching methods that affect the quality of stereo disparity, especially at the poles. This paper visually analyzes issues in the display of stereo panoramas with standard VR display methods on both HMDs and panel or projection-based walk-in VR displays.

Equirectangular projection of left image of the stereo pairs

The Quality of Stereo Disparity in the Polar Regions of a Stereo Panorama

We discuss the design and operation of an Epipolar-Plane Imaging light-field camera array and embedded-computation system designed for panoramic operation. Built from a planar configured EPI module, this is a hybrid capable of imaging in 360º x 3D. We discuss the geometry and constraints of EPI imaging, its connection with the design and structuring of the acquisition array, and the architecture of the EPIModule forming the basis for our panoramic capture. Finally, we show an arrangement of modules on a geodesic - the Truncated Icosahedron of football/soccer - which happens to suit the optical and mechanical properties of the first EPIModule we have built. Results are preliminary; more will be presented for the conference publication.

An EPIModule: camera head PCB at left, processor PCB at right.

EPIModules on a geodesic: Toward 360-degree light-field imaging

Visualization research often seeks designs that first establish an overview of the data, in accordance to the information seeking mantra: &ldquo;Overview first, zoom and filter, then details on demand&rdquo;. However, in computational fluid dynamics (CFD), as well as in other domains, there are many situations where such a spatial overview is not relevant or practical for users, for example when the experts already have a good mental overview of the data, or when an analysis of a large overall structure may not be related to the specific, information-driven tasks of users. Using scientific workflow theory and, as a vehicle, the problem of viscous finger evolution, we advocate an alternative model that allows domain experts to explore features of interest first, then explore the context around those features, and finally move to a potentially unfamiliar summarization overview. In a model instantiation, we show how a computational back-end can identify and track over time low-level, small features, then be used to filter the context of those features while controlling the complexity of the visualization, and finally to summarize and compare simulations. We demonstrate the effectiveness of this approach with an online web-based exploration of a total volume of data approaching half a billion seven-dimensional data points, and report supportive feedback provided by domain experts with respect to both the instantiation and the theoretical model. 
Index Terms - theory, visualization design, details-first model, discourse paper, computational fluid dynamics 
<a href="https://youtu.be/q-_KM_oRgxk">See a video on this project.</a>