Not a new idea here, but it was still fun to make. While in NYC last week, I spoke at the Boards Summit. They asked if Andrew Bell and I would give a talk on Processing. I made the following demo because I wanted an easy way to showcase what Processing can handle in real-time and also wanted a way to segue into showing off a couple audio-visual pieces I made a while back.
I decided to just start with the standard FFT waveform. I figured it was something the audience (of mostly advertising folks) might be familiar with. The point was to show that the raw FFT data can be a bit unruly with tons of seesaw and spikes. First I pointed out the FFT data is heavily biased towards the lower frequencies which presents itself as a sharp falloff as you go from low to high. As a quick fix, I multiplied the FFT data by a section of a sine wave and leveled off the data. This put the majority of the action in the middle of the waveform instead of the left side.
I then applied an averaging pass which smoothed out a lot of the spikes. Next, a decay attack and falloff was added to keep the spikes from popping in and out too frantically. Finally, I threw the whole data set into a 2D ripple algorithm to give the whole thing a bit more of an organic feel (one of these days, Id like to retranslate this back into audio to see what it might sound like).
Suddenly, the once frantic FFT data was turned into smooth rolling peaks that still reflected the quality of the incoming audio. The look was somewhat landscapey so I decided to push that aesthetic by showing the history of the audio (in the video example below, I am keeping track of the last four seconds of audio). I then added in some appropriate textures and got a nice scrolling landscape that responding fine to real-time audio input. It is particularly entertaining to swing the camera around to the back of the passing landscape and turn it into a rudimentary flight simulator over dynamically generated, audio driven landscape.
And yeah, it does look a bit Bryce3D. Next time, maybe I will even throw in some crystal spheres rising out of a mountain lake! Retro-Awesome!!
(My apologies if the previous description is both too simple and too complex. I don’t actually know much about FFT and how it is derived. I know what the data looks like and that is plenty for me to play around with it.)