Audio Analyzing Tool
Late in the program, in our Engine Development class we are given a research assignment of our choice. Many people tend to focus on advanced tech, such as Normal Mapping or Terrain Tessellation, but being a game programmer and a designer I chose to research something that could enhance the immersion of the player. I’ve been fascinated with Audio Integration in games so I decided to research Real-Time Beat Detection.
My goal was to create a system that would allow the designer to integrate audio and gameplay in a way that the gameplay wouldn’t necessarily be restricted by the audio, as is the case with many rhythm-based games.
This tool is a C# music player that uses FMOD for audio playback and data gathering and OpenGL for rendering. The beat detection is very basic and doesn’t work very well, I ran out of time before I could implement a more sophisticated Beat Detection Algorithm.
However, this program contains several impressive features:
- C# audio playback using Managed FMOD
- Fully Multi-threaded Audio Sampling
- C# OpenGL implementation using Tao Framework
My greatest achievement in this project is the multi-threaded system I implemented to keep the program running at a stable frame rate while doing processor heavy operations with the audio data (FFT processing, Beat Detection).
Check out the provided documentation for implementation details.
Thank you so very much for sharing your awesome project! I have been looking for an example like this for quite some time! I play with code & laser projectors from time to time and would like to build a laser visualizer of my own. The AVS Laser winamp plugin is OK, but not what I have in mind. The WMP SDK in c++ is so-so, but I don’t like being stuck in the WMP as a wrapper. This is going to be my new toy!
Hey Mikkojay, thanks for the comment, it really lifted my spirits. Let me know if you need any help exploring my code. I’m glad someone finds my work useful :)
Thanks! Hey, I have a question for ya- do you know if it is possible using the managed fmod library to be able to open a file and do a quick (as possible) scan of the track in order to create an FFT timeline bitmap? You know, like the squiggle lines that you see in just about every audio editor out there? I have seen code that physically opens and parses the contents of a .wav file, but it would be nice if fmod could do the grunt work. If you have any suggestions on that concept, I’d appreciate it!
Thanks, Mike
I can give you an advice, but I’m not sure it is the best way to try to do this, I would search the net to find a better way to do this.
First you need to obtain the output rate of a song (frequency), you can do this by calling the FMOD function “getFrequency” on a channel. This gives you the number of values per second that the song contains.
The “getSpectrum” and “getWaveData” functions get the data of the song, so the way I would go about trying to do what you want to do is to use the function “SetMusicPos” in the FMOD Wrapper to increment a second, and then get the Audio Data for the previous second, and repeat that until the song is over.
Keep in mind, if the frequency of a song is 44100 HZ and you are calling “getSpectrum” every second for 44100 values, it will kill your computer, because running the FFT algorithm on that many values is extremely CPU intensive, also if you want to do that make sure you are loading the song into memory, by calling “PlaySound” instead of “PlayMusic” which streams the song as it plays.
Make sure you check out the FMOD’s documentation on some of those functions. You can’t make an audio based program without having quick access to the docs.
Let me know if you have any other theories and I’ll give my input, unfortunately I can’t spend time trying this out myself because I’m heavy in the development of the final project game mandatory for graduation.
Hey Sam,
I did some digging & searching on the fmod forums and found a great piece of code that reads the sound into a stream, then iterates through it to create an array of samples. I also found a nice snippet of code that draws the waveform for the entire song/array. I am cleaning it up right now, but if you send me an email sometime in the next couple weeks, I can send it to you if you would like.
Good luck with the game project- that sounds intense.
Have a good one, Mike
Hey Sam,
This is really great. I’ve been interested in audio programming for awhile, and this is a great learning tool. The icing on the cake: you’re a fellow Full Sail grad! (congrats on the recent graduation). I’m really enjoying studying your project. Its very interesting.
Thanks for leaving the feedback on my project. If you’re interested in setting up an FMOD audio system this project is a great resource. However if you plan on doing real-time beat detection please don’t stop here. My next step was to implement a Frequency Filtered beat detection algorithm (or Filtering rhythm detection).
The algorithm is outlined in the following GameDev article:
http://www.gamedev.net/reference/programming/features/beatdetection/page2.asp
Check it out if you’re interested. Again, thanks for the feedback :)