This document outlines the research, implementation, and aesthetic philosophies of Meridianus Divinitus. Meridianus Divinitus is an artistic reinterpretation of the body energetic. It was created by taking data from existing charts which outline the flows of chi through out the human body for use in training acupuncture and acupressure, augmented by personal impressions attained by tactile remote sensing. This information was used to create a system of rules for particle creation and flow. Simple sets of spatial locations were artistically interpreted into five separate video animations, based loosely on the ancient five &ldquo;sacred&rdquo; elements (in Western cultures): fire, water, air, earth, and spirit. These works culminated in a four-channel gallery installation, juxtaposing four animations of archetypal classic elements projected in the four cardinal directions.

Fire Element

Meridianus Divinitus

ParaView is one of the most widely used scientific tools that supports parallel visualization of large scale data. The Scalable Adaptive Graphics Environment (SAGE) is a graphics middleware that enables real-time streaming of ultra-high resolution visual content from distributed visualization resources to scalable tiled displays connected by ultra-high-speed networks. Integrating these two technologies enables visualization of large-scale data at an extremely high resolution to be displayed on distantly located scalable tiled displays. The benefits, limitations, and future directions for this approach will be discussed. 
Categories and Subject DescriptorK 
I.3.2 [Computer Graphics]: Graphics Systems - Distributed/network graphics 
General Terms 
Experimentation, Performance 
Keywords 
Large-scale data, remote visualization, ultra-high resolution visualization, ParaView, SAGE

100 million pixel tiled display at EVL displaying two ParaView sessions.

Remote Visualization of Large Scale Data for Ultra-High Resolution Display Environments

While many visualization tools exist that offer sophisticated functions for charting complex data, they still expect users to possess a high degree of expertise in wielding the tools to create an effective visualization. This poster presents Articulate, an attempt at a semi-automated visual analytic model that is guided by a conversational user interface. The goal is to relieve the user of the physical burden of having to directly craft a visualization through the manipulation of a complex user-interface, by instead being able to verbally articulate what the user wants to see, and then using natural language processing and heuristics to semi-automatically create a suitable visualization. 

Index Terms: H.5.2 [Information Interfaces and Presentation]: User Interfaces - Graphical user interfaces

Yiwen Sun presenting poster to Dr. Jock Mackinlay at VAST 2009

Articulate: a Conversational Interface for Visual Analytics

We introduce SocioScape, an interactive tool for the visual exploration of spatially referenced, dynamic social networks. The tool combines a novel depiction methodology that illustrates the evolution of social groups with a spatio-temporal visualization showing the movement of these groups in the environment. The depiction technique used by SocioScape provides an alternative to graphs, allowing domain scientists to easily explore the evolution of the social structure and examine the role of the environment in shaping the social behavior of populations. 

Keywords: Information visualization, social networks, data exploration 

Index Terms: H.2.8 [Database Management]: Interactive data exploration and discovery; H.5.2 [Information Interfaces and Representation (HCI)]: Graphical user interfaces

SocioScape Poster for VAST 2009

SocioScape - a Tool for Interactive Exploration of Spatio-Temporal Group Dynamics in Social Networks

The field of Surgery has seen two major paradigm shifts - from Open to Laparoscopic and then from Laparoscopic to Robotic. During this transition, newer more advanced technologies and devices have been added to the already existing Operating Rooms. But in spite of the technical advancements in the equipment used and the way Surgery is performed, the Operating Room itself has remained unaltered over the ages. As a consequence, our traditional Operating Rooms have succumbed to lack of space, cluttered cables, overcrowding, poor connectivity and data access making them an inefficient workspace. Our project &ldquo;OR of the Future&rdquo; hinges on addressing these issues while aiming to change the way we think of Operating Rooms. Imaging and Visualization is one crucial aspect of this otherwise broad scoped project. 

In the current Operating Rooms, different members of the team view information pertaining to them on small independent screens. Due to this independent visualization scheme, working as a team and coordinating with each other at different stages of the procedure becomes problematic. Since the nurses, the Assistant Surgeon and the Anesthesiologist do not see exactly what the Surgeon sees; it becomes difficult for them to prepare for the &ldquo;next step&rdquo; in a timely fashion. With this problem in hand, we hope to find a solution with the proposed instrument, the Visualization Wall.

Visualization Wall for OR of the Future

Everything from image capture, modification, creation, and output, either on displays or on printers has been rapidly migrating from traditional photographic and print techniques to computer-based techniques. It is inevitable, except in perhaps artistic circumstances, that within 10 years all imaging will have switched over to using computing rather than traditional film and print-based media. While the traditional role of displays has been for the viewing of television programming, over the past 10 years displays have increasingly become portals into virtual environments. It is therefore important to understand the current state of the art in image display technology so that one can begin to anticipate how this technology in the future will evolve, and how best to take advantage of that evolution once it matures. This chapter focuses on the issues of image display and how new advanced displays that are being used mainly in research and higher education today may become commonplace in the next 5-10 years.

Advances in Computer Displays (chapter)

This paper describes the first experiment on Multi-Rail and MultiLane technologies using global networks. These technologies leverage end-host and network parallel resources, e.g., processor cores and lambda paths, to achieve large bandwidths.

Prototype of implemented terabit LAN NIC

Achieving Large Bandwidth by Leveraging Parallelism in End-Hosts and Networks

This thesis presents SocioScape, an interactive visualization tool that embodies a methodology for the visual analysis of spatial and temporal group dynamics in social networks. The methodology introduces a novel visual representation technique suitable for dynamic social networks. This representation provide an advantage over dynamic graphs by explicitly illustrating the evolution of social groups and association choices made by actors over time. The representation is combined with a well-established technique for depicting spatio-temporal data, allowing analysts to investigate the effect of the physical positioning of actors and their movement in the environment on their social behavior. This integration also facilitates the investigation of potential hypotheses that explain the emergence of the observed social structure structure. A case study demonstrates the usefulness of the tool. 

The primary contributions of this thesis include: 
<ol>
<li>A novel visual representation method for dynamic social networks that departs from traditional graph-based visualizations, revealing the evolution of social groups and association choices that actors make over time.</li>
<li>A methodology that integrates abstract representation of social interactions with a spatio-temporal visualization to allow analysis of the role of environment in shaping the underlying social structure.</li>
<li>A case study in which the methodology was used by behavioral ecologists to explore the social behavior of two animal populations of endangered species.</li>
</ol>

Screenshot of SocioScape showing the movement and social structure of Grevy&rsquo;s zebra herds

SocioScape - Spatio-temporal Visual Analysis of Group Dynamics in Social Networks

Museums and researchers both have a need to display and interact with high-resolution data. There is great interest in using multi-touch tables, most of which are projector-based. However LCD-displays are becoming an attractive alternative to projectors because of their low cost of maintenance and image clarity. While touch screen overlays have been available for LCD panels for some time, these overlays were only able to register at most two touches. In contrast our project&rsquo;s goal was to create a touch table that is both LCD-based and can register an arbitrary number touches. This paper will provide the technical details of how to build such displays, one of which was HD (1080p) resolution while the other was 4- Megapixel resolution. This paper will also discuss how we plan to extend this technique to develop a multitouch autostereoscopic display.

30-inch multi-touch table. The display bottom LED array is raised so show large LED&rsquo;s.

Design Challenges in Creating a High Resolution LCD-based Multi-touch Display

Middle school students were asked to critique three representations of DNA nucleotide strands - a paper-and-pencil static representation, a tangible representation using &ldquo;pop beads&rdquo; to represent nucleotides, and a computer simulation - that they used in design tasks within an instructional unit on nanoscale self-assembly. Student responses were coded into six response categories. Responses generally focused on visual and interactive affordances of representations rather than issues of enjoyment or task difficulty. Positive responses outnumbered criticisms, with visual affordances dominating negative responses. Distribution among response categories varied little across representational forms, but the simulation elicited the strongest positive response, with the paper-and-pencil representation drawing the strongest criticisms. This study provides tentative support for the explicit introduction of representational critique generally, and further, for the use of multiple representations as stimuli for such activities, especially when the phenomenon are from domains in which the learners lack direct experience or prior understandings.

Subject Drawings of Nucleotide Chains

Response Characteristics of Middle School Learners&rsquo; Critiques of Nanoscale Phenomena Representations

Many inter-related planetary height map and surface image map data sets exist, and more data are collected each day. Broad communities of scientists require tools to compose these data interactively and explore them via real-time visualization. While related, these data sets are often unregistered with one another, having different projection, resolution, format, and type. We present a GPU-centric approach to the real-time composition and display of unregistered-but-related planetary-scale data. This approach employs a GPGPU process to tessellate spherical height fields. It uses a render-to-vertex-buffer technique to operate upon polygonal surface meshes in image space, allowing geometry processes to be expressed in terms of image processing. With height and surface map data processing unified in this fashion, a number of powerful composition operations may be uniformly applied to both. Examples include adaptation to non-uniform sampling due to projection, seamless blending of data of disparate resolution or transformation regardless of boundary, and the smooth interpolation of levels of detail in both geometry and imagery. Issues of scalability and precision are addressed, giving out-of-core access to giga-pixel data sources, and correct rendering at scales approaching one meter.

A view of the U.S. showing the NED, BMNG, and SRTM data sets, totaling 115GB

Planetary-Scale Terrain Composition

This paper summarizes the design rationale and classroom experience surrounding a middle school instructional unit on a nanoscale phenomenon: the self-assembly of DNA strands as applied to the problem of virus detection. The instructional design was motivated by a strategy for integrating new nanoscale content into an existing curricular unit, situating learner activity within a design framework, and using multiple representations of nanoscale phenomena, both as media for the design activities and as the object of student critique. Students in a middle school classroom showed significant pre-post gains both in their understandings of domain concepts and in their ability to articulate affordances and constraints of representations and models.

Students use PDAs surrounding dynamic simulated &ldquo;cauldron&rdquo; to design and attach candidate &ldquo;catcher&rdquo; strands

Introducing nanoscale phenomena to middle school learners using a DNA design activity

Does usability affect long term user engagement in computer games, or are other factors more influential? This paper explores this issue, discussing an evaluation study that measured the relevance of usability versus playability factors for long-term user engagement in eight commercial games. 

Keywords: Interactive game, engagement, usability, playability, heuristic evaluation, user testing, correlation 

ACM Classification Keywords: H5.2. User Interfaces - Evaluation / Methodology

Workflow for Usability / Playability Factors

Usability, Playability, and Long-Term Engagement in Computer Games

In this paper, we propose an immersive empathic design method and used it to create an interactive high-resolution core visualization system for real-world geological core drilling expeditions. A high domain knowledge barrier makes it difficult for a person from outside this field to imagine the user experience simply through observation. The globally distributed nature of the core drilling community imposes further design constraints. We used this approach to embed a computer scientist trained as a junior core technician. This process allowed the developer to experience authentic user activities and enabled the design of an innovative system for solving real-world problems. This approach made the best use of precious co-located opportunities, overcame the initial domain knowledge barrier, and established a trust relationship between the developer and the domain scientists. The system designed through this approach formed a sustainable and adaptive foundation that the domain scientists can build on. Through in-situ deployment, observation and interview evaluations from on-going expeditions, we present the advantages of this process. 

Keywords: HCI, Visualization, Empathic Design

Dr. Franco Talarico with the CoreWall setup in McMurdo station, Antarctica

Design an Interactive Visualization System for Core Drilling Expeditions Using Immersive Empathic Method

For many areas of science and engineering, visualization has become more than a convenient tool: it has become a necessity for interpreting the enormous amounts of data being produced by large-scale instrumentation, experiments and simulations. One of the most creative - and useful - ways of presenting scientific data is the large-scale tiled display. Ranging from tabletop versions to fully immersive 3D stereo virtual reality environments, these displays enable users to explore features that may be hard or impossible to identify with conventional, personal computers. 

Concluding the first session of the SciDAC 2007 conference in Boston, SciDAC Program Director Dr. Michael Strayer commented that one of the major challenges facing the SciDAC multidisciplinary teams was &ldquo;developing new visualization and data management systems to support moving, storing, analyzing, manipulating, and visualizing multipetabyes of scientific data and objects.&rdquo; 

Scientific visualization indeed has become a critical tool in understanding large datasets and the behavior of complex systems. And a special form of scientific visualization - large-scale tiled displays - is enabling scientists and engineers to explore a wide range of applications, from structural engineering of large buildings to medical studies of extremely small cellular structures. 

Tiled displays are not new. During the early 1990s a number of planar tiled displays were built and used for scientific studies. Several factors limited their wide acceptance, however. They were expensive, large, and hence unsuitable for the typical office environment. To address this situation, the California Institute for Telecommunications and Information Technology (Calit2) at the University of California - San Diego (UCSD), and the Electronic Visualization Laboratory (EVL) at the University of Illinois - Chicago (UIC) initiated the OptIPuter project. 

The goal of the OptIPuter project is to develop innovative methods for scientists to explore very large remote data objects in an interactive and collaborative fashion. It is an ambitious goal, and the developers have enjoyed remarkable success. Without a doubt the highlight is the &ldquo;OptIPortal&rdquo; - a commodity-priced, easily replicable display, with tens to hundreds of megapixels, which can be used by individual researchers or collaborative groups.

Tellurion, a real-time planetary-scale data visualization developed by EVL PhD graduate Robert Kooima, is shown here on the Calit2 OptIPortal

Visualizing Science: The OptIPuter Project

Several studies have analyzed the link between mental dysfunctions and eye movements, using eye tracking techniques to determine where a person is looking, that is, the fixations. In this paper, we present a novel methodology to improve current diagnosis and evaluation methods of attention disorders. We have developed and tested several data-mining methodologies suitable for the automatic analysis and visualization of eye tracking data. In particular three novel methods of classification of subjects are proposed: (i) a method that uses Expectation Maximization to classify according to statistical likelihood of fixations locations; (ii) a procedure based on the Levenshtein distance method to compare sequences of fixations; and (iii) a method based on the analysis of the transitions frequencies of fixations between regions. Results of evaluation of classification accuracy are finally presented.

Eye movement for a single image for a five seconds lapse of time: (a) a control subject; (b) a subject diagnosed with ADHD

Automatic Analysis of Eye Tracking Data for Medical Diagnosis

Visualization has proven its value in scientific advances by helping scientists gain insight from their data and verify scientific computations. The amount of scientific data collected from sensors and simulations can easily be on the order of petabytes of data. Visualization of this largescale data requires cluster computing, and more often than not distributed computing over high-speed networks, as the size of the data exceeds the capacity of the average computing clusters, and the data may not even reside locally. To view visualizations of large-scale data at or near native resolution, scalable tiled display walls are increasingly being used for scientific visualization. 

In this context, the Scalable Adaptive Graphics Environment (SAGE) has been developed to support large-scale data visualization in a distributed visualization environment that includes ultrahigh resolution scalable tiled displays. It is a specialized middleware that enables real-time streaming of extremely high-resolution graphics and high-definition video from remotely distributed rendering and storage clusters to scalable display walls over ultra high-speed networks. This dissertation extends SAGE to support distant collaboration between multiple endpoints. 

In the SAGE framework, each visualization application streams its rendered pixels to the virtual high-resolution frame buffer of SAGE, allowing users to freely move, resize and overlap the application windows on the display. Every window movement or resize operation requires dynamic and non-trivial reconfigurations of the involved graphics streams. These reconfigurations become even more complex when SAGE is required to support multiple collaboration endpoints with different tiled display configurations and application window layouts. Visualcasting is a new SAGE network service to address this problem using a high-speed bridging system that receives pixel streams from rendering clusters and that duplicates and sends them to each end-point. This enables distant collaboration among international researchers in scalable display environments. Using the Visualcasting service, collaborators can share their visualizations and interact with each other through high-definition video conferencing in the SAGE Framework.

Multi-point HD video conferencing using Visualcasting

Visualcasting: Scalable Real-time Image Distribution in Ultra-High Resolution Display Environments

This document outlines the research, theory and shared scholarly pursuit for Critical Gameplay. Critical Gameplay is a design technology derived from the application of critical design on the development of electronic games. The goal of this design practice is to highlight social phenomenon through the creation of games. In particular the practice, as applied here is fundamentally involved in the investigation of gameplay mechanics as they relate to learned perspective on general operations. 

Critical Gameplay is also the name of the gallery exhibition associated with this research.

Still from Critical Gameplay

Critical Gameplay

The current inability to realistically simulate and test dynamic user interfaces of mobile device prototypes is one of the prime limiting factors to successful product launches. Current processes provide very little insight into inter-relationships among physical form of the product, the human-machine interface and the applications that run on those devices. In this paper, we present a technique that provides a common platform where product designers, user interface developers and software programmers can interactively visualize and test their new concepts by exploring and touching a 3D virtual reality prototype using Augmented Reality and Haptics. This simulation tool can enhance or even replace traditional prototyping and facilitate testing of the prototype at various points of the design cycle. 

Keywords: Augmented Reality, Product Design, Virtual Reality Prototyping, User Interface, Haptics, Simulation

Functional Virtual Reality Prototype of a PDA interacted with a virtual stylus

Applying Augmented Reality and Haptics to Evaluate Dynamic Prototypes of Mobile Devices

This paper focuses upon the research and development of a prototype dental simulator for training of periodontal procedures. By the use of virtual reality and haptics technology, the periodontal simulator allows trainees to learn performing diagnosis and treatment of periodontal diseases by visualizing a three-dimensional virtual human mouth and feeling real tactile sensations while touching the surface of teeth, gingiva, and calculi with virtual dental instruments. Since periodontics requires dentists to depend primarily on tactile sensations to perform diagnostic and surgical procedures, the use of haptics is unquestionably crucial for a realistic periodontal simulator. The haptics-based virtual reality periodontal training simulator has been validated by a experiment conducted by the College of Dentistry at University of Illinois at Chicago (UIC) with faculty members and dental students, which demonstrates the scientific contribution and usefulness of the simulator as a vital part of the curriculum of the Department of Periodontics at UIC. 

Keywords:Virtual reality, Simulation, Training, Haptics, Dentistry, Periodontics

Periodontal simulator showing calculi being removed with scaler

Haptics-based virtual reality periodontal training simulator

This paper describes the design, development and user study of a text-to-speech system for reviewing email messages. The system retrieves messages, converts them to audio files, and facilitates transferring the messages to a portable listening device. This study reviews the efficacy of such a system and compares the users&rsquo; ability to comprehend spoken audio messages verses written messages. According to the study the system was successful, but the user interest in the design was only moderate. 

Keywords: Universal Accessibility; Text-to-Speech; Universal Design; Email; Portable Communication Device; Portable Entertainment Device; User Study

gReader Interface

gReader: A Universally Designed, Device-Independent Email Client

Cognitive research has revealed learning techniques more effective than those utilized by the traditional art history lecture survey course. Informed by these insights, the author and fellow graduate researchers at the University of Illinois at Chicago designed a &ldquo;serious&rdquo; computer game demo, Art Thief, as a potential model for a learning tool that incorporates content from art history. The game design implements constructed learning, simulated cooperation and problem solving in a first-person, immersive, goal-oriented mystery set within a virtual art museum.