In game programming, there are many events that can occur. They may be keypresses, mouse movements, and so on. The music playing can also be used to trigger events. Events in the game world can also be triggered by looking at the spectrum of a music file (wav, mp3, ogg, etc).
We used the spectrum analysis functionality of FMOD in our program . However, any sound/music library that has similar capabilities can be used to create a similar effect.
FMOD divides the frequency of the music file into 512 bands. The frequencies are from 0Hz to 22050Hz. The bands are divided evenly into 512 parts, from which we can read the intensity of any specific band. It is similar to the bands that move when a stereo plays music.
With FMOD we can select certain bands to pay attention to. In the image above, the frequency bands that are being singled out are denoted by the red, blue, and green colors. The height of the line is the intensity of that particular band. The intensity in FMOD is a real number from 0 to 1. By polling the intensity of a specific frequency band, the program can have events triggered if the intensity meets certain criteria.
When selecting a frequency to trigger events with, it is usually a good idea to choose a frequency that appears regularly in all the music that will be used. It's a good idea to view the music files with a spectrum analyzer program like the one pictured above to see what bands would be the best to use. Depending on the frequency used to trigger events the effect should work with any music file thrown at it.
Within BreakoutVR we analyze a low frequency band's intensity. The lower frequencies correspond to the bass/kick drum. When the frequency threshold is crossed, an event is triggered. In the case of BreakoutVR the room colors are incremented at random whenever the threshold is met. This gives the a continuously-shifting display of lights at different points of a song.