&ldquo;Hearts and Minds: The Interrogations Project&rdquo; is a Virtual Reality art installation developed using a novel method for direct output of the Unity-based virtual reality projects into CAVE2&trade; [Febretti et al. 2013] environment. This artwork incorporates original research and technological innovation in an adaptation of veterans&rsquo; testimonies detailing US military interrogations in Iraq during the American counter-insurgency campaign in the early 2000s. It uses VR technology to immerse participants in the minds of people who experienced torture and interrogation during the war to understand its current social and psychological consequences. The powerful content of this artwork focuses on the impact of war and trauma on veterans, and utilizes the power of VR as a medium to evoke empathy, understanding and awareness. This work was developed at the Electronic Visualization Lab in Chicago through a unique cross-disciplinary international collaboration between artists, scientists, and researchers from five different Universities. The methods developed for this project allow hands-on education of virtual reality by letting students create their own virtual environments and exhibit them in the CAVE2 quickly. These methods have been recently adapted by Design and Computer Science courses at the University of Illinois at Chicago.

A scene of Hearts and Minds in the CAVE2 environment.

The Battle for Hearts and Minds: Interrogation and Torture in the Age of War. An Adaptation for Oculus Rift.

The main focus of this research is to evaluate the combination of the CAVE2&trade; and the HTC VIVE to improve the collaborative tasks. The thesis describes background work conducted as part of an internship with Mechdyne, Inc., which provides the groundwork for the core research. 
We are living in a very interesting period for virtual reality (VR). The general press called 2016 &ldquo;the year of the VR&rdquo;. In fact, they talked about the new Head Mounted Display (HMD) being available for the first time to the general public. They manage to produce advances in HMD and making it more affordable (such as HTC, Oculus, and Sony). VR exists since the 1950&rsquo;s and takes different forms other than an HMD. For example, the CAVE&trade; is a VR device. Its screens are not in front of the eyes of the user - rather it is projected on large screens. While HMD is the opposite. CAVE is very expensive to manufacture and requires big spaces. This type of VR is more fitting for industrial use. On the other hand, HMD is also suitable for industrial use because it has high-quality engagement among participants, it is less expensive to produce, and it is at parity with CAVE&rsquo;s standard. Both devices have advantages and disadvantages. Thus, it would be interesting to combine both devices to complement each other. 
VR has a lot of applications. One of the applications that arouses great interests among companies is its collaborative application. It connects people in various locations to work together. With the advent of globalization, many big companies have an international presence and their subsidiaries are located in various countries. In order to bring them together, they need an effective network of communication. Essentially, companies such as Microsoft with Windows Holographic or MiddleVR with Improov3 are developing their own platforms and devices to connect people in various locations in a more efficient way. 
With this in mind, the objective of this thesis works is to develop a synergy among people by combining two VR devices to explore the efficiencies and capabilities. 
Keywords: HTC VIVE, CAVE2, collaboration, NASA TLX, Virtual Reality, EVL, Mechdyne, HoloLens, OmegaDesk, Cyber-Commons, Unity, interaction, HMD, Oréal, virtual shopper

Virtual Shopper VIVE application.

Telecollaboration Task Between CAVE2 and HTC VIVE

The highly rewarding experience of electronic music persists throughout our daily lives. Our immediate environments are replete with events that emit sounds that are extremely complex. Electronic music engages with the listening habits we take for granted in our everyday lives, and reveals how intricate they can be. Inspired by such intricacies, I have conducted a series of listening experiments with 80 participants over the course of three years to explore the cognition of electronic music. In this article, I will first present the method and the results of this experiment, including a categorical analysis of mental associations evoked by different works of electronic music. Next, I will offer a discussion of narrativity in electronic music supported by these results and diverse perspectives on narrativity from a number of disciplines. I will then construct a definition of gestures as narrative units in electronic music in relation to events in the environment. In doing so, I will bring together various theories on electronic music with not only the findings of the current study but also existing research on auditory cognition.

Categorical distribution of real-time descriptors by piece.

Imagining through Sound: An experimental analysis of narrativity in electronic music

Real world group-to-group collaboration often occurs between partially distributed interdisciplinary teams, with each discipline working in a unique environment suited for its needs. Groupware must be flexible so that it can be incorporated into a variety of workspaces in order to successfully facilitate this type of mixed presence collaboration. We have developed two new techniques for sharing and synchronizing multi-user applications between heterogeneous large-scale shared displays. The first new technique partitions displays into a perfectly mirrored public space and a local private space. The second new technique enables user-controlled partial synchronization, where different attributes of an application can be synchronized or controlled independently. This paper presents two main contributions of our work: 1) identifying deficiencies in current groupware for interacting with data during mixed presence collaboration, and 2) developing two multi-user data synchronization techniques to address these deficiencies and extend current collaborative infrastructure for large-scale shared displays. 
Author Keywords: Mixed presence collaboration; computer-supported cooperative work; large-scale displays; multi-user interaction; data synchronization. 
ACM Classification Keywords: H.5.3. Information Interfaces and Presentation: Group and Organization Interfaces: Computer-supported cooperative work, synchronous interaction.

Mixed presence collaboration using the dataduplication technique. The yellow highlight in each photo shows the shared portal on each shared display with synchronized applications and pointer icons.

Synchronized Mixed Presence Data-Conferencing Using Large-Scale Shared Displays

Classic visual analysis relies on a single medium for displaying and interacting with data. Large-scale tiled display walls, virtual reality using head-mounted displays or CAVE systems, and collaborative touch screens have all been utilized for data exploration and analysis. We present our initial findings of combining numerous display environments and input modalities to create an interactive multi-modal display space that enables researchers to leverage various pieces of technology that will best suit specific sub-tasks. Our main contributions are 1) the deployment of an input server that interfaces with a wide array of interaction devices to create a single uniform stream of data usable by custom visual applications, and 2) three real world use cases of leveraging multiple display environments in conjunction with one another to enhance scientific discovery and data dissemination. 
Author Keywords: Multiple Display Environments; multi-user interaction; collaboration; input devices; large-scale displays; virtual reality; multi-touch screens; motion capture. 
ACM Classification Keywords: H.5.2. User Interfaces: Input devices and strategies, user centered design.

Collaborators gather in front of a large-scale display to share research findings and utilize a multitouch digital whiteboard to brainstorm next steps to advance their work.

Interactive Multi-Modal Display Spaces for Visual Analysis

InfoVis novices struggle with visualization construction. Even with the aid of visualization software, such users may face challenges when translating their questions into appropriate visual encodings, or interactively refining the representation to achieve a desired result. 
A &lsquo;conversational interface&rsquo; which maintains a dialog with the user through natural language and gestures, could allow users to engage in repeated cycles of visualization generation and modification, asking questions directly through speech. 
In this poster we present a prototype conversational visual data analysis system. Our prototype was developed from a corpus consisting in 15-subjects engaging in exploratory data visualization with a simulated conversational interface. It features 1) speech to visualization pipeline, 2) classification system to divide utterances into major types, 3) history manager and knowledge-base.

Articulate2: Toward a Conversational Interface for Visual Data Exploration

Assuming that the topological space containing all possible brain states forms a very high-dimensional manifold, this paper proposes an unsupervised manifold learning framework to reconstruct and visualize this manifold using EEG brain connectivity data acquired from a group of healthy volunteers. Once this manifold is constructed, the temporal sequence of an individual&rsquo;s EEG activities can then be represented as a trajectory or thought chart in this space. Our framework first applied graph dissimilarity space embedding to the temporal EEG connectomes of 20 healthy volunteers, both at rest and during an emotion regulation task (ERT), followed by local neighborhood reconstruction then nonlinear dimensionality reduction (NDR) in order to reconstruct and embed the learned manifold in a lower-dimensional Euclidean space. We showed that resting and ERT thought charts represent distinct trajectories, and that the manifold resembles dynamical systems on the torus. Additionally, new trajectories can be inserted on-line via out-of-sample embedding, thus providing a novel data-driven framework for classifying brain states, with potential applications in neurofeedback via real-time thought chart visualization. 
Keywords: Thought chart; Graph dissimilarity embedding; Nonlinear dimensionality reduction; EEG connectome; Emotion regulation 
DOI: 10.1007/978-3-319-47103-7_15

An illustration of a typical ERT session. A fixation point is on display before each trial, then followed by either a neutral or negative picture on the screen. An audio instruction will ask test subjects to maintain, reappraise or stay neutral.

Thought Chart: Tracking Dynamic EEG Brain Connectivity with Unsupervised Manifold Learning

Radiomics involves the application of image processing algorithms to define a series of quantitative image characteristics. We sought to develop a clinically usable radiomics signature for primary oropharyngeal cancers (OPC), using the gross tumor volume (GTV) regions of interest (ROIs) to derive a &ldquo;phenotypic profile&rdquo; associated with time to local recurrence (LR). 
Support: 
National Institutes of Health: 
NCI-R01-CA214825, &ldquo;SMART-ACT: Spatial Methodologic Approaches for Risk Assessment and Therapeutic Adaptation in Cancer Treatment&rdquo; 
NCI-R01CA225190,&ldquo;QuBBD: Precision E –Radiomics for Dynamic Big Head & Neck Cancer Data&rdquo;

Head and neck cancer treatment and toxicity Big Data collected at U Texas MD Anderson Cancer Center; jointly analyzed at EVL, MDACC, UMN and Iowa.

Development of a Predictive Quantitative Contrast Computed Tomography-Based Feature (Radiomics) Profile for Local Recurrence in Oropharyngeal Cancers

Site-specific data visualization installations have distinct conditions of data collection, data analysis, audience interaction, and data archiving. This paper describes features of five data visualization projects related to their successful staging within different contexts.

A photo showing a gallery visitor contributing data to the Pockets Full of Memories installation.

Data in context: Conceptualizing site-specific visualization projects.

Node-link diagrams are widely used for visualizing relational data in a wide range of fields. However, in many situations it is useful to provide set membership information for elements in networks. We present BranchingSets, an interactive visualization technique that uses visual encodings similar to Kelp Diagrams in order to augment traditional node-link diagrams with information about the categories that both nodes and links belong to. BranchingSets introduces novel user-driven methods to procedurally navigate the graph topology and to interactively inspect complex, hierarchical data associated with individual nodes. Results indicate that users find the technique engaging and easy to use. This is further confirmed by a quantitative study that compares the effectiveness of the visual encodings used in BranchingSets to other techniques for displaying set membership within node-link diagrams, finding our technique more accurate and more efficient for facilitating interactive queries on networks containing nodes that belong to multiple sets.

A prototype application that uses the BranchingSets technique to facilitate the navigation of a complex, hierarchical biological pathway dataset where both nodes and links are members of one or more sets.

BranchingSets: Interactively Visualizing Categories on Node-Link Diagrams

This paper introduces a user-centered Augmented Reality (AR) approach for publishing 2D media archives as interactive content. We discuss the relevant technical considerations for developing an effective application for public outdoor AR experiences that leverage context-specific elements in a challenging, real-world environment. Specifically, we show how a classical marker-less approach can be combined with mobile sensors and geospatial information in order apply our knowledge of the surroundings to the experience itself. Our contributions provide the enabling technology for Chicago 0,0 Riverwalk, a novel app-based AR experience that superimposes historical imagery onto matching views in downtown Chicago, Illinois along an open, pedestrian waterfront located on the bank of the Chicago River. Historical photographs of sites along the river are superimposed onto buildings, bridges, and other architectural features through image-based AR tracking, providing a striking experience of the city’s history as rooted in extant locations along the river. 
Index Terms: H.5.1 [Information interfaces and presentation (e.g. HCI)]: Multimedia Information Systems-Artificial, augmented and virtual realities

A group of people testing the Riverwalk AR application with both smartphone and tablet devices along the Chicago River.

Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences

The goal of our research is to support full-fledged dialogue between a user and a system that transforms the user queries into visualizations. So far, we have collected a corpus where users explore data via visualizations; we have annotated the corpus for user intentions; and we have developed the core NL-to-visualization pipeline.

User examining crime data on tiled display.

Towards a Dialogue System that Supports Rich Visualizations of Data

In this paper, we investigate the role affect plays in electronic music listening. By referring to a listening experiment conducted over the course of three years, we explore the relation between affect and diegetic affordances (i.e. those of the spatiotemporal universes created by electronic music). We will compare existing perspectives on affect with the psychologist James Gibson&rsquo;s model of affordances in the context of an electronic music practice. We will conclude that both the sounds themselves and the diegetic affordances of these sounds may elicit affective reactions, and that further study into the relation between diegesis, affordance, and affect may contribute to a better understanding of what we hear in electronic music.

Diegetic Affordances and Affect in Electronic Music

With the advent of new hardware and software technologies, virtual reality has gained a significant momentum recently. VR design tools, such as game engines, have become much more accessible and are being used in a variety of applications ranging from physical rehabilitation to immersive art. These tools, however, offer a limited set of tools for audio processing in 3D virtual environments. Furthermore, they are platform-dependent due to performance requirements and feature separate editing and rendering modes, which can be limiting for sonic VR implementations. To address these, we introduce a novel web-based system that makes it possible to compose and control the binaural rendering of a dynamic open-space auditory scene. Developed within a framework of well-established theories on sound, our system enables a highly detailed bottom-up construction of interactive virtual soundscapes by offering tools to populate navigable sound fields at various scales (i.e. from sound cones to 3D sound objects to sound zones). Based on modern web technologies, such as WebGL and Web Audio, our system operates on both desktop computers and mobile devices. This enables our system to be used for a variety of mixed reality applications, including those where users can simultaneously manipulate and experience a virtual soundscape.

A screenshot of our user interface on a desktop computer displaying an object with two cones and a motion trajectory being edited.

A Web-based System for Designing Interactive Virtual Soundscapes

Computer-modeled characters resembling real people sometimes elicit cold, eerie feelings. This effect, called the uncanny valley, has been attributed to uncertainty about whether the character is human or living or real. Uncertainty, however, neither explains why
anthropomorphic characters lie in the uncanny valley nor their characteristic eeriness. We propose that realism inconsistency causes anthropomorphic characters to appear unfamiliar, despite their physical similarity to real people, owing to perceptual narrowing. We further propose that their unfamiliar, fake appearance elicits cold, eerie feelings, motivating threat avoidance. In our experiment, 365 participants categorized and rated objects, animals, and humans whose realism was manipulated along consistency reduced and control transitions. These data were used to quantify a Bayesian model of categorical perception. In hypothesis testing, we found reducing realism consistency did not make objects appear less familiar, but only animals and humans, thereby eliciting cold, eerie feelings. Next, structural equation models elucidated the relation among realism inconsistency (measured objectively in a two-dimensional Morlet wavelet domain inspired by the primary visual cortex), realism, familiarity, eeriness, and warmth. The fact that reducing realism consistency only elicited cold, eerie feelings toward anthropomorphic characters, and only when it lessened familiarity, indicates the role of perceptual narrowing in the uncanny valley. 
doi:10.1167/16.11.7

Objects, animals, and humans constitute the low, intermediate, and high anthropomorphism groups, respectively. The right half of a photograph of each entity is shown beside the left half of its 3-D computer model.

Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley

This essay reformulates the question of human augmentation as a problem of advanced human-machine communication, theorizing that such communication implies robust artificial intelligence and necessitates understanding the relational role new technologies play in human-machine communication. We focus on the questions, &ldquo;When do electronic tools cease to be &rsquo;simply&rsquo; tools, and become meaningfully part of ourselves,&rdquo; and, &ldquo;When might we think of these tools as augmenting our selves, rather than simply amplifying our capabilities?&rdquo; These questions, already important to the medical and rehabilitative fields, loom larger with increasing commodification of pervasive augmentation technologies, and indicate the verge on which human-machine
communication now finds itself. Through analyses of human and machine agency, mediated through a theory of close human-machine communication, we argue that the critical element in discussions of human-machine communication is an increase in sense of agency, extending the traditional human-computer interface dictum to provide an internal locus of control. 
Keywords 
Human Augmentics, Human-Machine Communication, Human-Computer Interaction, Agency, Sense of Agency

Communication, Machines & Human Augmentics

Electromagnetic fields are formed through complex interactions between outer space, the Sun, our Earth, its atmosphere, and the built environment. Our communications technology makes use of them to enable the transmission of information at local, global, and even extraterrestrial scales. Mastery of electromagnetic fields made the information age possible, and the constant acceleration of innovation in communication technology continues to profoundly influence contemporary culture. Because wireless communication takes advantage of ranges of the electromagnetic spectrum that we are unable to sense physically, it is necessary to translate these signals into a domain that human beings can more readily perceive in order to understand them. Not only is it relevant to see the signals themselves (which is commonly done through instruments such as oscilloscopes or spectrum analyzers), it is also important to understand what kinds of information are being transmitted via these signals and the sociopolitical mechanisms that define and govern their use. 
This article introduces a series of artworks that explore new creative opportunities made possible via low-cost sensors (which can be used to home in on a small region of the spectrurm within limited ranges) and state-of-the-art receivers (which are able to capture terabytes of spectrum data per second from great distances.

Data Improvisations, the IEEE VIS Arts Program exhibition. Traversing the Spectrum Radio Streams and Aerial Modulations were first presented at this exhibition October 2015.

Spectral Landscapes: Visualizing Electromagnetic Interactions

The thesis work presented here focuses on the optimization of the similarity computation among nodes for spatio-temporal networks. The study case in this work is the computation of spatio-temporal networks on brain. The aim of the first part of the work presented is to built a GPU-based computation system able to speed up the networks definition. In the second part of the work we developed an interactive tool to visualize the networks computed using the GPU-based method developed in this first part. 
Nowadays, with the increase of computational analysis in sciences such as biology and neuroscience, the computational aspect is the most challenging one. Scientists need tools able to process large amounts of data simultaneously. Previously, scientific computations were performed using clusters of computers, but this type of infrastructure is very expensive and complex to build. The work in this thesis is part of a greater project that has the aim to apply spatio-temporal networks inference techniques to perform network analysis studies in different fields. One of the problems of spatio-temporal network applications is the computational time, and it becomes impractical to keep developing studies when it takes a long time to analyze and compute the results. We aim to help the researchers to get results in a reasonable amount of time, so they can be focused more on their studies. There are many ways to speed up the computation process, one of these is to exploit the paradigm of parallel computation using Graphic Processor Unit (GPU). GPUs represent a low-cost solution to this problem since the level of parallelization they have is really high and GPUs hardware match exactly the requirements for the problem addressed. With the first part of this thesis work we present a GPU solution and we evaluate the performance of spatio-temporal networks definition. 
The second part of the thesis work uses the results computed in the first part to propose an interactive visualization tool for the spatio-temporal networks computed. With a decrease in computation time, it is easier to achieve a comparison between the results. The proposed visualization tool allows for the changing of parameters at run-time and visualizing the differences in the resulting networks. This process is made possible thanks to a great work of pre-computation of all the data that must to be visualized using the GPU-based system developed. This second part of the work goals to improve the understanding of the results computed by the algorithm. In this study, we focus on brain network computation and visualization. 
We mentioned that this thesis work is composed by two different parts. One of them is used to compute the similarity measure needed to define the DN. While, the other aims to visualize interactively the DN computed, leaving one degree of freedom to the
final user. One of the goals of this work is the creation of a pipeline that goes from the collected data to the visualization of the result, which in our case are the networks computed previously. In fact the two piece of software developed are part of the pipeline that defines the framework we present. This framework want to avoid the dependability problem given by that the different software are not able to communicate among each others.

Network comparison, it shows the comparison between two different data session loaded in the system, each one is represented by a color in the network.

GPU Acceleration and Interactive Visualization for Spatio-Temporal Networks

Thanks to technological advances, low-priced tiled touch displays have the potential to radically change the way we work together in collaborative environments, such as a classroom or a meeting. Whiteboards are an example device that could be replaced by tiled touch displays. Whiteboards are commonly used in classroom environments to communicate content, improve the teaching experience, and sketch diagrams or illustrations to better explain complex concepts. They are also very frequently used in meetings or brainstorming sessions, in order to share ideas between a group of people. In recent years, whiteboards have been increasingly replaced by projectors or very large non-interactive displays. These new technologies bear specific advantages, such as better readability through the use of prepared slides, and improved online access to teaching materials. However, they also introduce a certain rigidity to the teaching process: when a student poses an unexpected question, a quick answer or diagram cannot be promptly sketched. 
This thesis proposes an electronic whiteboard for tiled touch displays. An electronic whiteboard can combine the advantages of an electronic device with the fast paced interaction provided by a regular whiteboard, enhancing both solutions with new functions. In this application, users can perform classic whiteboard actions such as drawing or writing free-handed on the screen using their finger or a stylus. They can use brushes to paint, and they can erase content. However, unlike on traditional whiteboards, users can also relocate content on the screen, insert figures and drawings, and save or load previous sessions. Moreover, everything can be shared between all the users, and more people can interact simultaneously, improving the collaborative task. 
Creating an electronic whiteboard application poses a number of problems that will be analyzed and solved in the following sections, contributing to the literature in the human-computer interaction, computer graphics and software engineering fields. The main graphics and software problems analyzed are the networked architecture of the application, which needs to run on multiple displays; the integration of the whiteboard application with other display-enabled applications; and the management of multiple simultaneous inputs. We also deal with the problem of rendering the same graphic elements on different sized displays, and how
to keep the solution lightweight enough to be sent over a network without creating a major latency problem. The interaction problems are the interface design and the analysis of how people behave when collaborating on a shared touch enabled display. We also analyze how to deal with remotely controlled collaboration, since the whole application can be shared among both co-located and remote screens. In this context, we evaluate remote interaction using mobile touch displays, and we conduct a small scale user study to analyze the advantages of multi-touch displays over conventional keyboard and mouse configurations.

Single user using SAGEBoard.

SAGEBoard: a Whiteboard for Large Multitouch Displays

People are generally enthusiastic of innovative technologies. One of the most important witnesses of this enthusiasm is the reaction of the audience in the iPhone 2007 Steve Jobs presentation when he showed the pinch gesture to zoom . People literally said &lsquo;wow&rsquo; and after a while clapped, completely fascinated by what they were seeing. LVDs or compositions of them are one of these technologies that people appreciated in the past and continue to appreciate in the current days. Just imagine how in living rooms the televisions are becoming every year larger. Also in office desks are present larger monitors or more of them side by side used in order to increase the number of open program windows. 
The main characteristics of LVDs are a ratio where the horizontal dimension is three or four times the vertical one, and the width that is at least three meters. These dimensions make very inefficient their usage in everyday tasks, or at least using it through applications developed for a normal desktop pc. Nevertheless, many fields start to have a strong interest in this kind of device. Companies had an interest on employing them for meetings or presentation to customers. Studies understood how their huge resolution was potentially able to show an enormous quantity of information in a more efficient way with respect to all the methods used in the past. A map is an interesting example of the higher efficiency of LVDs; a high-resolution map fits better in a display with very high resolution. Many scientists can simultaneously analyze the map, and easily discuss about points of the map having these points in front of themselves. 
The efficiency of the scientists&srquo; actions may be further increased by the introduction of a system that allows the scientists to directly manipulate with their hands and fingers what they are observing. Here comes the need of a LVD with multi-touch capabilities. Such a feature, in addition to being fascinating and attracting, increases enormously the degrees of freedom of the interaction between the human and the LVD. Another interesting example useful to appreciate the potential usefulness of this device is a group of scientists during the analysis of a molecule model. The bigger size of a molecule model on a LVD and the possibility to rotate and zoom the model aid scientists on their analysis and discussions. 
LVDs are a technology relatively new that bring numerous challenges together with their potentiality. Adding multi-touch interaction increases the number of the previous challenges that computer scientists and developers have to tackle and to find a solution for. Some of these challenges are related to: 
- Input Detection: how users should interact with the display, which is the most efficient input device to interact with the display among single mouse, multiple mice, single-touch, multi-touch, pointer, if input devices influence the user collaboration; 
- Human Interaction: how the usage of a normal operating system is comfortable on an LVD, how users dispose windows, how users exploit the display space and if there are less used area, how the position of the users with respect to the LVDs influences the previous factors, how users collaborate in a shared context; 
- Data Visualization: how to show data and information, how to interact, navigate, consume and produce data; 
- Application Context: in which context applications have to live, so if an application has to be a normal operating system program or if has to live in a dedicated context; 
- Application Development: how to organize an application interface for this large resolution, which are the guidelines to follow for LVD application development, if the normal desktop application development has the same characteristics of LVD application development, if there are specific library dedicated to application for LVD. 
The past and current research is directed mainly towards the first three points of the previous list. Data visualization on LVDs is probably the most researched field of the previous points. Different techniques dedicated to different kind of data can be employed to allow users to consume more effi- ciently the information. There is much research also in the human-interaction field with these devices that generated a interesting conclusions. However, these conclusions are somewhat sparse, sometimes divergent and consequently it is difficult to exploit efficiently the information. This is a clear symptom that LVDs are still a very young study subject and much effort has still to be done in order to create a consistent literature. The same phenomenon is observable for input detection for LVDs, even though there are less studies on this direction. 
Instead, the research seems to be absent for the application context and the application development for LVDs. There are no directives or guidelines suggesting how to develop applications for very large and high resolution displays. This consequently leads developer to create a dedicated solution for each problem. Moreover, there are not any dedicated software environment to them. There are some attempts on making browsers the main container for LVD applications . In fact, internet page interfaces can be adapted to whatever resolution without a big effort. However there are no common ways or guidelines to organize an interface or create functions for LVD applications, and every small piece has to be created from scratch by the developer. 
In this thesis we will focus on the human interaction with LVDs.

The interface of the second toy example. A pressure on an image on the sidebar displays the image on the central container.

Large Multi-Touch Vertical Displays in Multi-User Competitive Tasks

In this paper, we describe a novel multimedia system for networked musical collaboration. Our system, called Monad, offers a 3D virtual environment that can be shared by multiple participants to collaborate remotely on a musical performance. With Monad, we explore how various features of this environment in relation to game mechanics, network architecture, and audiovisual aesthetics can be used to mitigate problems inherent to networked musical performance, such as time delays, data loss, and reduced agency of users. Finally, we describe the results of a series of qualitative user studies that illustrate the effectiveness of some of our design
decisions with two separate versions of Monad. 
Author Keywords: Networked music performance; virtual environments; game mechanics; audiovisual performance systems. 
ACM Classification: H.5.5 [Information Interfaces and Presentation] Sound and Music Computing|Systems, H.5.2 [Information Interfaces and Presentation] User Interfaces|Auditory (non-speech) feedback, H.5.2 [Information Interfaces and Presentation] Multimedia Information Systems

A public performance with Monad v0.2 with local players in Istanbul and remote players from Vancouver and Berlin.

Networked Virtual Environments as Collaborative Music Spaces

PORTAL is an interactive performance tool that uses a laser projector to visualize computer-generated audio signals. In this paper, we firrst offer an overview of earlier work on audiovisual and laser art that inspired the current project. We then discuss our own implementation, focusing not only on the technical issues related to the use of a laser projector in an artistic context, but also on the aesthetic considerations in dealing with the translation of sounds into visuals, and vice versa. We provide detailed descriptions of our hardware implementation, our software system, and its desktop and mobile interfaces, which are made available online. Finally, we offer the results of a user study we conducted in the form of an interactive online survey on audience perception of the relationship between analogous sounds and visuals, which was explored as part of our performance practice. 
Author Keywords: Audiovisual art, sound-image interaction, multimodality, laser projection
ACM Classification: H.5.5 [Information Interfaces and Presentation] Sound and Music Computing, H.5.2 [Information Interfaces and Presentation] User Interfaces

A photo from a PORTAL performance.

PORTAL: An Audiovisual Laser Performance System

This project&rsquo;s purpose was to investigate the ability of virtual reality to teach music. An application was developed for the CAVE2&trade; and Microsoft HoloLens, and their ability to teach music effectively was analyzed. Both the applications teach the user how to play &lsquo;Mary Had a Little Lamb&rsquo; on the viola, and each virtual reality platform had it&rsquo;s own benefits and drawbacks. An user interface was created for both platforms that shows the user the correct bow direction, tempo, fingering, and sheet music. Additional features included adjustable tempo and a metronome that can be turned on and off. The application was helpful since the user could get visual cues from the graphics while still keeping an eye on his/her instrument. Future goals for
the application include songs that can be uploaded by the user, microphone input, more instruments, and better graphics. The results of this research show that music instruction is a viable possibility in a virtual reality environment and will spark developers and musicians to work together to further explore this field.

Trackers placed on viola for CAVE2 version of the music instruction application.

Music Instruction in a Virtual/Augmented Reality Environment Using the CAVE2 and Microsoft HoloLens

&ldquo;Hearts and Minds: The Interrogations Project&rdquo; is an interactive performance made for the CAVE2&trade; large-scale 320-degree panoramic virtual reality environment that describes veterans&rsquo; testimonies about military interrogations in Iraq during the American counter-insurgency campaign. The project is based on interviews of American soldiers and on their actual testimonies. The project was achieved through technical innovation, cross-disciplinary and international collaboration. It was developed using a novel method for direct output of the Unity-based virtual reality projects into the CAVE2 environment. Other portable versions of the work were developed to reach new audiences across educational, arts and public arenas which include (1) personal computer version navigable using Xbox 360 controller; (2) web-based version available for free download; (3) Oculus Rift immersive virtual reality HMD version; (4) mobile version of the project for Apple iPad (in progress). This paper describes the development and compares the interaction experiences across platforms. 
Keywords: Virtual reality, Navigation, Interaction, Immersive environments, 3D user interface, CAVE2&trade;

Presentation of the project using the Xbox controller at Litteraturhuset Oslo, Norway

From CAVE2&trade; to Mobile: Adaptation of Hearts and Minds Virtual Reality Project Interaction

In this paper we introduce TimeArcs, a novel visualization technique for representing dynamic relationships between entities in a network. Force-directed layouts provide a way to highlight related entities by positioning them near to each other. Entities are brought closer to each other (forming clusters) by forces applied on nodes and connections between nodes. In many application domains, relationships between entities are not temporally stable, which means that cluster structures and cluster memberships also may vary across time. Our approach merges multiple force-directed layouts at different time points into a single comprehensive visualization that provides a big picture overview of the most significant clusters within a user-defined period of time. TimeArcs also supports a range of interactive features, such as allowing users to drill-down in order to see details about a particular cluster. To highlight the benefits of this technique, we demonstrate its application to various datasets, including the IMDB co-star network, a dataset showing conflicting evidences within biomedical literature of protein interactions, and collocated popular phrases obtained from political blogs. 
Categories and Subject Descriptors (according to ACM CCS): H.5.2 [Information Interfaces and Presentation]: User
Interfaces-Graphical user interfaces

TimeArcs visualization for interactions around OPSD protein. (1) and (2) are conflicting evidences of OPSD and K+ interaction in 2003 and 2012 publications.

TimeArcs: Visualizing Fluctuations in Dynamic Networks

Adjacency matrices can be a useful way to visualize dense networks. However, they do not scale well as the network size increases due to limited screen space, especially when the number of rows and columns exceeds the pixel height and width of the screen. We introduce a new scalable technique, MultiLayerMatrix, to visualize very large matrices by breaking them into multiple layers. In our technique, the top layer shows the relationships between different groups of clustered data while each sub-layer shows the relationships between nodes in each group as needed. This process can be applied iteratively to create multiple sub-layers for very large datasets. We illustrate the usefulness of MultiLayerMatrix by applying it to a network representing similarity measures between 2,048 characters in the Asteraceae taxonomy, a rich dataset that describes characteristics of species of flowering plants.We also discuss the scalability of our technique by investigating its effectiveness on a large synthetic dataset with 20,000 columns by 20,000 rows.

Visualizing 2048 nodes in a regular adjacency matrix (left) and in a MultiLayerMatrix of two layers: The middle panel shows the first layer, and the right panel shows an example of the second layer.

MultiLayerMatrix: Visualizing Large Taxonomic Datasets

PURPOSE 
To develop methods for rapid and simultaneous design, testing, and management of multiple clinical decision support (CDS) features to aid nurse decision-making. 
METHODS 
We used quota sampling, think-aloud and cognitive interviews, and deductive and inductive coding of synchronized audio-video data and archival libraries. 
FINDINGS 
Our methods and organizational tools allowed us to rapidly improve the usability, understandability, and usefulness of CDS in a generalizable sample of practicing nurses. 
CONCLUSIONS 
The method outlined allows the rapid integration of nursing terminology-based electronic health record data into routine workflow and holds strong potential for improving patient outcomes. 
IMPLICATIONS FOR NURSING PRACTICE 
The methods and organizational tools for the development of multiple CDS system features can be used to translate knowledge into practice.

The Hands Ecosystem

Toward a More Robust and Efficient Usability Testing Method of Clinical Decision Support for Nurses Derived from Nursing Electronic Health Record Data

We present the design of a scalable visual tool for the analysis of high-throughput network traffic, power consumption and cluster-based system resource allocation. The tool explores the possibility of visualizing remote and distributed resources over large distances by using SAGE2, a collaborative framework for large scale display systems. Programmable networks and distributed cloud resources allow accessibility to large amounts of infrastructure data. The tool needs to present the information in an easy-to-understand format for users who do not have any prior systems or network administration experience, but still have an interest in power consumption and network traffic - such as university and city officials. We evaluate the effectiveness of the application
through user testing and feedback. 
Keywords: network traffic, power consumption, cluster-based system resource allocation, analysis, large displays.

Network switch data.

Developing a Scalable SNMP Monitor

Commercial software, such as Google Hangouts, Skype, and WebEx, has made significant strides to enhance remote collaboration between single users. Other systems, such as Oblong&rsquo;s Mezzanine, have enabled groups to collaborate over distance, but only when the display environment is cloned in all locations. These configurations are not always practical in the real world since collaboration is often interdisciplinary, with each discipline having a unique work environment suited for its needs. I have used SAGE2, the Scalable Amplified Group Environment, as a platform to implement a scalable solution for viewing and
interacting with arbitrary content, and developed and tested various techniques for synchronizing data across multiple sites. SAGE2 is the successor to SAGE, the Scalable Adaptive Graphics Environment, which is a middleware to display and interact with an assortment of data-intensive information from multiple sources on displays of arbitrary size. In other words, SAGE2 allows collaborators to use any Scalable Resolution Shared Display (SRSD) from a single monitor to a large tiled display wall to act as a seamless window manager for applications such as a high-resolution image viewer, interactive mapping software, 3D model
viewer, and multi-user notepad. 
Unlike video and audio, data-conferencing with fully synchronized content may not always be ideal. Groups or individuals may be analyzing different portions of the same data, and therefore desire unsynchronized interactions in certain situations. I have conducted user studies comparing three techniques of data-conferencing. The first technique was data-pushing, sending unsynchronized documents between locations, along with two video streams – one of the collaborators and one of the SRSD. The second technique was data-duplication, where one section of the SRSD contained fully synchronized versions of all data-conferencing content and a second portion of the SRSD contained local unsynchronized copies of the data-conferencing content. The final technique was to use advanced data-synchronization options, where collaborators chose which aspects of each shared application were synchronized and which were controlled independently. These studies aimed to answer three main research questions: 
1) Does providing continuously synchronized applications improve the quality of collaboration and awareness of the remote team? 
2) Can synchronizing applications provide remote teams the same quality of collaboration as local teams? 
3) Does the size of a shared display have an effect on the quality of collaboration and awareness of the remote team? 
While I used SAGE2 to develop and test these principles, the applications of the knowledge gained can broadly apply to data-conferencing software in general.

Mapping application used in the formal user study. Multiple users could interact simultaneously to manipulate the map, or add/remove markers.

Data-Intensive Remote Collaboration using Scalable Visualizations in Heterogeneous Display Spaces

In this paper, we investigate methods for improving networked musical collaboration by introducing game mechanics to musical performance. We present a novel audiovisual performance system, where multiple users contribute to an over-the-network performance that is mediated by a variety of game-like rules. User evaluations and audience feedback collected in both private performance and public concert contexts indicate the potential of game-like music environments for improving musical interaction and listener attention in networked computer music performances. 
Author Keywords: Networked music performance; game mechanics; audiovisual performance systems. 
ACM Classification Keywords: H.5.5 [Information interfaces and presentation.

A screencast from a performance with Monad v0.1 [6], which utilized a client-driven rewarding mechanism.

Using Game Mechanics to Facilitate Networked Musical Collaboration

Distractions is a kinetic sculpture that brings together sonification and visualization in a single physical artifact. It utilizes inaudible frequencies to visualize invisible electromagnetic signals, such as those from cellphone, Wi-Fi and radio communications, that surround us in daily life. This way, the piece highlights some of our routine sources of distraction in their raw forms. By picking up the electromagnetic signals in the exhibition space, the sculpture visualizes the signals to and from the mobile devices that are brought into this space by the visitors. The sub frequency vibrations, which oscillate the structure according to these intentional or unintentional communications, not only represent the distractions the artist faces on a regular basis, but also intercept the dialogue between the audience and the inert steady state of the sculpture. The piece therefore interacts with the audience through the distractions that draw them away from the experience of the artwork. 
Author Keywords: Kinetic sculpture; sonification; depth image; radio frequency data; visualization; CNC 
ACM Classification Keywords: H.5.1 [Information interfaces and presentation (e.g., HCI)]: Multimedia Information Systems

The MDF framwework of the sculpture being milled with a CNC machine.

Distractions: Interactive Visualizations on a Kinetic Sculpture Using Infrasound Vibrations

This paper presents a novel approach for the design of creative location-based mixed reality applications. We introduce a framework called DigitalQuest that simplifies adding geolocated virtual content on top of real-world camera input. Unlike previous work, which relies solely on markers or image pattern recognition, we define a &ldquo;mirror world&rdquo; that facilitates interactive mixed reality. DigitalQuest consists of an editor that allows users to easily add their own content as desired and a mobile application that loads content from a server based on the location of the device. Each piece of virtual content can be organized through the editor so that it appears only in certain circumstances, allowing a designer to determine when and where a virtual object is attached to a real-world location. We have used our editor to create a series of futuristic scavenger hunts in which participating teams must solve puzzles in order to access new virtual context appearing in a mixed reality environment via a mobile phone application. In this paper, we introduce our editor and present an example scavenger hunt game, Morimondo, that was built using it. Specifically, we describe our technique to utilize camera and motion sensors on the mobile phone to enable an appropriate level of user engagement within this game. We are able to obtain realistic augmentations with accurate positioning by leveraging sensor fusion and through the use of filters that compensate for sensor noise, using image processing only for error correction or in special situations. The initial success of this project leads us to believe that DigitalQuest could be used to design a wide range of creative multi-user mixed reality applications. 
Index Terms: H.5.1 [Information interfaces and presentation (e.g., HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; H.5.3 [Group and Organization Interfaces]: Collaborative computing-Games.

A sample camera view of the mobile application showing the user arriving in proximity of a virtual object. The on-screen buttons provide user interaction; the upper-left bar provides a score and the current GPS accuracy.

DigitalQuest: A Mixed Reality Approach to Scavenger Hunts

A range of platforms facilitate the design of virtual environments. Most commonly, game engines, such as Unity and Unreal, are used for developing and simulating virtual realities. However, such platforms are primarily oriented towards visual design and provide only limited audio functionality. Furthermore, these platforms utilize separate editing and rendering modes, and operate on desktop platforms. Making use of existing research into the development of interactive virtual soundscapes, we introduce a novel user interface that enables the rapid design of both virtual sonic environments, and the assets (i.e. sound objects and sound zones) contained within them. 
Specifically, our UI offers the following contributions: 
<ul>
<li>provides a 3D design environment specific to sonic virtual realities, with specialized components such as sound objects and sound zones;</li> 
<li>offers both interactive and parametric control over the attributes of such components, enabling a precise authoring of highly-detailed environments;</li> 
<li>introduces a multi-cone model for creating 3D sound objects with complex propagation characteristics;</li> 
<li>enables adding dynamism to objects via hand drawn motion trajectories that can be edited in 3D;</li> 
<li>makes it possible to design virtual sound fields at various scales using multiple view and attribute windows;</li> 
<li>offers a unified interface for the design and the simulation of such realities, allowing the user to modify a sound field in real-time;</li> 
<li>operates on the web-browser so that it supports mobile devices, which therefore makes it possible for the user to simultaneously explore and edit augmented sonic realities.</li>
</ul>

Designing and Controlling Virtual Sonic Environments Using a Browser-based 3DUI

Node Kara is an audiovisual mixed reality installation by the artist Anıl Camci. It offers a body-based interaction using a 3D imaging of the exhibition space. With a blurred audiovisual scene in its steady-state, it invites visitors to interact with the multiplicity of layers it holds within. By moving in the exhibition space, the visitors can deblur both the sound and the visuals as they get immersed in the piece. Rather than a detection of discrete events or gestures, Node Kara relies on fluid interactions with continuous visual layers and stochastic audio. This way, the piece reacts to each visitor in a unique way. A video preview of Node Kara can be seen at <a href="https://vimeo.com/anilcamci/nodekara">https://vimeo.com/anilcamci/nodekara</a> (password: nodekara). 
Index Terms: H.5.1 [Information interfaces and presentation (e.g., HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; H.5.2 [Information Interfaces and Presentation]: User Interfaces-Auditory (non-speech) feedback.

Four layers of Node Kara superimposed onto each other with a representation of the visitor’s depth image highlighted.

Node Kara: An Audiovisual Mixed Reality Installation

We describe the lessons learned from three recent Immersive Analytics projects which take place at the Electronic Visualization Lab. These successful collaborative projects use the CAVE2 immersive hybrid environment. All three projects visualize multifaceted scientific data and benefit from interdisciplinary collaborations with experts across application domains. We further outline the challenges and opportunities ahead for the field of Immersive Analytics. 
Index Terms: K.6.1 [Immersive Analytics]: Virtual Reality-Interdisciplinary collaborations; K.7.m [Technology]: Displays-CAVE2

Young researchers examine, as a group, large scale cosmology data in CAVE2.

Interdisciplinary Immersive Analytics at the Electronic Visualization Laboratory: Lessons Learned and Upcoming Challenges

Empirical evaluation methods for visualizations have traditionally focused on assessing the outcome of the visual analytic process as opposed to characterizing how that process unfolds. There are only a handful of methods that can be used to systematically study how people use visualizations, making it difficult for researchers to capture and characterize the subtlety of cognitive and interaction behaviors users exhibit during visual analysis. To validate and improve visualization design, it is important for researchers to be able to assess and understand how users interact with visualization systems under realistic scenarios. This article presents a methodology for modeling and evaluating the behavior of users in exploratory visual analysis. We model visual exploration using a Markov chain process comprising transitions between mental, interaction, and computational states. These states and the transitions between them can be deduced from a variety of sources, including verbal transcripts, videos and audio recordings, and log files. This model enables the evaluator to characterize the cognitive and computational processes that are essential to insight acquisition in exploratory visual analysis and reconstruct the dynamics of interaction between the user and the visualization system. We illustrate this model with two exemplar user studies and demonstrate the qualitative and quantitative analytical tools it affords. 
Keywords: Exploratory visual analysis, visual exploration, evaluation, insight-based evaluation

Modeling and evaluating user behavior in exploratory visual analysis.

Understanding how brain regions are interconnected is an important topic within the domain of neuroimaging. Advances in non-invasive technologies enable larger and more detailed images to be collected more quickly than ever before. These data contribute to create what is usually referred to as a connectome, that is, a comprehensive map of neural connections. The availability of connectome data allows for more interesting questions to be asked and more complex analyses to be conducted. In this paper we present a novel web-based 3D visual analytics tool that allows user to interactively explore the intrinsic geometry of the connectome. That is, brain data that has been transformed through a dimensionality reduction step, such as multidimensional scaling (MDS), Isomap, or t-distributed stochastic neighbor embedding (t-SNE) techniques. We evaluate our tool through a series of real-world case studies, demonstrating its effectiveness in aiding domain experts for a range of neuroimaging tasks.

This figure shows the main view of the web-based 3D visual analytics tool that allows user to interactively explore the intrinsic geometry of the connectome.

Intrinsic Geometry Visualization for the Interactive Analysis of Brain Connectivity Patterns

In situ analysis has been proposed as a promising solution to glean faster insights and reduce the amount of data to storage. A critical challenge here is that the reduced dataset is typically located on a subset of the nodes and needs to be written out to storage. Data coupling in multiphysics codes also exhibits a sparse data movement pattern wherein data movement occurs among a subset of nodes. We evaluate the performance of data movement for sparse data patterns on the IBM Blue Gene/Q supercomputing system &ldquo;Mira&rdquo; and identify performance bottlenecks. We propose a multipath data movement algorithm for sparse data patterns based on an adaptation of a maximum flow algorithm together with a breadth-first search that fully exploits all the underlying datapaths and I/O nodes to improve data movement. We demonstrate the efficacy of our solutions through a set of micro benchmarks and application benchmarks on Mira scaling up to 131,072 compute cores. The results show that our approach achieves up to 5X improvement in achievable throughput compared with the default mechanisms. 
Keywords: multiple paths, sparse data movement, topology-aware aggregation, data-intensive, Blue Gene/Q

Efficacy of the approaches to move the data to /dev/null on the I/O nodes.

Improving Sparse Data Movement Performance Using Multiple Paths on the Blue Gene/Q Supercomputer

In this paper, we present SAGE2, a software framework that enables local and remote collaboration on Scalable Resolution Display Environments (SRDE). An SRDE can be any configuration of displays, ranging from a single monitor to a wall of tiled flat-panel displays. SAGE2 creates a seamless ultra-high resolution desktop across the SRDE. Users can wirelessly connect to the SRDE with their own devices in order to interact with the system. Many users can simultaneously utilize a drag-and-drop interface to transfer local documents and show them on the SRDE, use a mouse pointer and keyboard to interact with existing content that is on the SRDE and share their screen so that it is viewable to all. SAGE2 can be used in many configurations and is able to support many communities working with various types of media and high-resolution content, from research meetings to the creative session to education. 
SAGE2 is browser-based, utilizing a web server to host content, WebSockets for message passing and HTML with JavaScript for rendering and interaction. Recent web developments, with the emergence of HTML5, have allowed browsers to use advanced rendering techniques without requiring plug-ins (canvas drawing, WebGL 3D rendering, native video player, etc.). One major benefit of browser-based software is that there are no installation requirements for users and it is inherently cross-platform. A user simply needs a web browser on the device he/she wishes to use as an interaction tool for the SRDE. This lowers considerably the barrier of entry to engage in meaningful collaboration sessions. 
Keywords: Scalable Resolution Display Environments, Window manager, Web-based, Collaboration, Multi-user interaction, Large-scale displays, Application development

Co-located collaborative session using SAGE2 - ten users interacting with a SRDE tiled wall.

SAGE2: a Collaboration Portal for Scalable Resolution Displays

RehabJim is a Unity3D application developed on behalf of researchers at a major Rehabilitation Institute in order to explore the opportunities that Virtual Reality may offer to biomechanical and neurological rehabilitation. In particular, our work focuses on arm actions performed while standing by patients post-stroke. We leverage a 3D immersive environment, augmented with Kinect interaction. Our approach follows a third person perspective of the patient&rsquo;s body, and employs two cartoon-style avatars - the patient and their therapist. The user is asked to reach to computer-generated virtual objects with her hands. Our evaluation with users and a domain expert shows that this type of environment leads to an engaging, enjoyable experience that can encourage patients to perform a wide variety of whole-body motions. 
Keywords: rehabilitation avatars, virtual worlds, virtual reality, CAVE2&trade;

Patient in the CAVE2 interactive with RehabJim. The shadows cast on the ground help the user perceive depth in the 3D environment.

RehabJim: A Third Person Approach To Virtual Reality Biomechanical Rehabilitation

Despite laser scan 3D point cloud acquisition has greatly improved over the next few years, the process of creating 3D large scale city models is still quite expensive and not straightforward. At the same time, nowadays services such as Google Street View provide a vast amount of geo-registered panoramic imagery, guaranteeing a decent resolution for dense locations at zero cost. Our idea is indeed to leverage this free information provided by Google Street View in order to obtain a cheap and automatizable 3D reconstruction of an urban area, by extracting the depth information related to the great number of panoramic images available online. 
ACM Classification Keywords 
H.5.m. Graphical user interfaces; Computer Graphics; Visualization 
Author Keywords 
Google Street View; Point Clouds; 3D reconstruction

Combination of multiple panoramas along a street.

3D City Reconstruction From Google Street View

Study of the behavior of individual members in communities of dynamic networks can help neuroscientists to understand how interactions between neurons in brain networks change over time. Visualization of those temporal features is challenging, especially for networks embedded within spatial structures, such as brain networks. In this article, the authors present the design of SwordPlots, an interactive multi-view visualization system to assist neuroscientists in their exploration of dynamic brain networks from multiple perspectives. Their visualization helps neuroscientists to understand how the functional behavior of the brain changes over time, how such behaviors are related to the spatial structure of the brain, and how communities of neurons with similar functionality evolve over time. To evaluate their application, they asked neuroscientists to use SwordPlots to examine four different mouse brain data sets. Based on feedback, their visualization design can provide neuroscientists with the ability to gain new insights into the properties of dynamic brain networks.

A SwordPlot includes multiple integrated components for representing time-series data and the community membership of a node in a dynamic network.

SwordPlots: Exploring Neuron Behavior within Dynamic Communities of Brain Networks

This paper presents a guideline for visualization designers who want to choose appropriate techniques for enhancing tasks involving multidimensional projection. Specifically, we adopt a user-centric approach in which we take user perception into consideration. Here, we focus on projection techniques that output 2D or 3D scatterplots that can then be used for a range of common data analysis tasks, which we categorize as pattern identification tasks, relation-seeking tasks, membership disambiguation tasks, or behavior comparison tasks. Our user-centric task categorization can be used to effectively guide the organization of multidimensional data projection layouts. Moreover, we present real-world examples that demonstrate effective choices made by visualization designers faced with complex datasets requiring dimensionality reduction. 
Keywords: Multidimensional data analysis, task taxonomy, multidimensional data projection, user-centric evaluation.

Comparison of INFOVIS04 document data set map using Neighbor Joining and LSP projections: Four dierent topics of information visualization are identied by coloring points.

Choosing Visualization Techniques for Multidimensional Data Projection Tasks: A Guideline with Examples

Simulation and modeling of turbulent flow, and of turbulent reacting flow in particular, involve solving for and analyzing time-dependent and spatially dense tensor quantities, such as turbulent stress tensors. The interactive visual exploration of these tensor quantities can effectively steer the computational modeling of combustion systems. In this article, the authors analyze the challenges in dense symmetric-tensor visualization as applied to turbulent combustion calculation; most notable among these challenges are the dataset size and density. They analyze, together with domain experts, the feasibility of using several established tensor visualization techniques in this application domain. They further examine and propose visual descriptors for volume rendering of the data. Of these novel descriptors, one is a density-gradient descriptor which results in Schlieren-style images, and another one is a classification descriptor inspired by machine-learningtechniques. The result is a hybrid visual analysis tool to be utilized in the debugging, benchmarking and verification of models and solutions in turbulent combustion. The authors demonstrate this analysis tool on two example configurations, report feedback from combustion researchers, and summarize the design lessons learned. 
&copy; 2016 Society for Imaging Science and Technology. 
[DOI: 10.2352/J.ImagingSci.Technol.2016.60.1.010404]