Synesthesia is defined as a unique neuroperceptual event identified in one out of three million people in which a sensory input, such as music, is perceived as a different sensory mode. Examples are seeing triangles when one listens to Mozart or tasting points when one eats artichokes. These phenomena are particularly intriguing as they relate to current research on neurological networking / interaction. The prototype developed in this project may serve as a stimulus to design innovative methods to assess, diagnose, treat, and rehabilitate those with brain injury, such as trauma, strokes, and so on. This could, in turn, stimulate efforts to build optimal environments in which dendrite regrowth / new networking is encouraged and enhanced. In this project the researchers are utilizing individual biosignals-heart beat, breathing, muscle movement-to create and drive a virtual reality environment. First they acquire a participant’s heartbeat and then feed it back as a recognizable sound. Then they switch signals (auditory to visual). The project director’s interest in brain trauma results from her own automobile accident and resulting injuries.

Within the immersive, real-time virtual reality environments, participants can actually fill in multisensory “gaps” via the heretofore “invisible” interaction / networking. Some of the networking may be energized through proprioceptive and kinesthetic dynamics.

Synesthia was a GII Testbed application demonstrated at ACM / IEEE Supercomputing ’95.