Tracking technology for Virtual Reality (VR) applications typically requires the user to wear head-mounted sensors with transmitters or wires. This paper describes a video-based, realtime, low-latency, high-precision 3D face tracker specifically designed for VR displays that requires no sensors, markers, transmitters, or wires to be worn. One center camera finds the 2D face position using Artificial Neural Networks (NN) and recognizes and tracks upright, tilted, frontal and non-frontal faces within visually cluttered environments. Two more left and right (L/R) cameras obtain the 3D head coordinates using a standard stereopsis technique. This paper presents a novel idea to train the NN to a new face in less than two minutes, and includes background training to avoid recognition of false positives. The system utilizes diffuse infrared (IR) illumination to avoid computing time-consuming image normalization, to reduce illumination variations caused by the physical surroundings, and to prevent visible illumination from interfering with the user’s field of view.

CR Categories: I.5.1 [Computer Methodologies]: Pattern Recognition-Models-Neural nets; I.4.8 [Image Processing and Computer Vision]: Scene Analysis - Tracking; I.3.7 [Computer Graphics]: 3D Graphics and Realism - virtual reality;

Keywords: Head, face, or camera-based tracking; Kohonen neural networks; autostereoscopic 3D display; virtual reality

PDF

Girado, J. I., Peterka, T., Kooima, R. L., Ge, J., Sandin, D. J., Johnson, A. E., Leigh, J., DeFanti, T. A., Real-Time Neural Network-based Face Tracker for VR Displays, Proceedings of the IEEE Virtual Reality Conference 2007(VR ’07), Charlotte, NC, March 10th, 2007.