A few months ago, the Barbarian Group was approached by Erasmus to help with promotions for the Relentless Energy Drink. Relentless, available only in the UK, is a frequent sponsor of music festivals and sports competitions. They asked us to make an audio visualizer that would run in real-time so they could show it at these venues. In addition to the live application, they wanted a content creation version and an iTunes plugin version. The content creation version would allow them to create renders from recorded audio and the iTunes version would allow members of the Relentless Order to experience it on their own computers.

The project went through many different looks. The primary art direction we received was that the end result should be dark and should have some baroque sensibilities. We did our best to hit these goals and we are quite proud of the end result.

Learning Experience
For me, the most rewarding part of this job was that I learned a whole lot. Fellow Barbarian, Andrew Bell, showed me many interesting tricks, not to mention working his ass off to optimize this project so it would run at 30+ fps and also handling the port to C++ for the iTunes visualizer version.

The early versions had the primary tendrils rendered as flat strips that always orient themselves towards the camera so you never (well, almost never) see the strips from the side. This was my way of faking the viewer into believing they were seeing a solid mass. However, this method had its problems. It was impossible to texture map it in an interesting way, and occasionally, the strip would twist a full 360° which added unsightly kinks. Andrew suggested we move to actual tubes instead of strips and I thought he was nuts. The piece was already starting to drag down the framerate and he wanted to add more geometry?? Nuts, I tell ya!!

A couple days later, he showed me the new build. I was floored. Not only did he do what he promised, but he improved the frame-rate in the process AND he added support for shaders. Awesome!

BEFORE:

AFTER:

Suddenly, things got a lot more interesting… more lush. He ended up switching the tendril geometry over to VBOs (vertex buffer objects) which sped things up a great deal. He also pre-cached the texture coordinates because they didn’t actually change once the piece was up and running.

The Birds

The idea of flocking crows or ravens is very much in keeping with the Relentless aesthetic but there was some concern that we were simply repurposing a project we did with Nervo for Fox Movies. We decided to do away with everything we did for Nervo and start from scratch. The original Nervo piece relied on using 160 flat images of a 3D bird in flight. For the Relentless version, we switched over to actual 3D birds (admittedly quite rudimentary). Each bird was made up of 12 polygons. The wings actually flapped and they could be viewed from any angle (whereas the Nervo version worked best when the birds were viewed side-on.

The Background

From the beginning we were instructed to do away with the notion of an infinite black or white space. The entire scene should be filled with content and this proved to be quite the challenge. Originally, we just dropped the whole scene inside a large textured sphere but the end result was lacking. Andrew came up with the idea of using a flat plane as the background but by using some FBO trickery, he was able to make these beautiful flourished textures bleed into view. I still have no idea how he pulled it off but the effect is quite stunning. Later he added a dynamic light source to make the scene feel even more dark and slightly disturbing.

The (faked) Volumetric Lighting

After we settled on the look, we wanted to make it seem like the invisible center of the mass of tendrils was a hidden light source. Earlier versions actually had the tendrils glowing near their base but the look of shadow and silhouette was more intriguing so we killed the glow and added faked volumetric lighting. The light beams themselves behave as if they were magnetic particles. Each one is born, lives for a predetermined amount of time, then fades out and dies. While they are alive, they repulse each other if they get too close. This created a really nice organic movement which made it seem like the movement of the tendrils actually caused the light beams to form or disappear. Combined with the background lighting, the effect became quite engaging.

The Content Creation Tool
Erasmus wanted to be able to create renders from pre-recorded pieces of music so we tweaked the live version to accommodate the loading in of external audio as well as an XML file of adjustable variables. By modifying the XML file, they could run the application and customize their experience. Every graphic element could be toggled individually. So if you want a video with no background for compositing, or perhaps you want to leave the birds out, just tweak the XML and run the application. Additionally, since we were no longer confined to creating an experience that ran live in real-time, you could up the amount of objects as high as your memory will allow. This is how we made the following test render. The bird and particle count was turned up really high, as was the detail on the tendril geometry.

The following video is a test render from The REV content creation tool. Please note this is not an official Relentless video. This is simply a test render we did to make sure the audio responsiveness was as we liked it. The audio is by The Flashbulb (”Six Months Without Light” off the album Kirlian Selections) but be aware that The Flashbulb has no affiliation with Relentless. The chosen audio was simply a test track. I chose The Flashbulb because he is my favorite artist to audio visualize and I love this track.

Relentless, The REV from flight404 on Vimeo.

Or you can view the much higher resolution Quicktime here. Note, it is around 300Mb.

The REV iTunes Version
As I mentioned above, there is a free downloadable version for iTunes. You can find out more at the Relentless Energy site. Have at it!

Not a new idea here, but it was still fun to make. While in NYC last week, I spoke at the Boards Summit. They asked if Andrew Bell and I would give a talk on Processing. I made the following demo because I wanted an easy way to showcase what Processing can handle in real-time and also wanted a way to segue into showing off a couple audio-visual pieces I made a while back.

I decided to just start with the standard FFT waveform. I figured it was something the audience (of mostly advertising folks) might be familiar with. The point was to show that the raw FFT data can be a bit unruly with tons of seesaw and spikes. First I pointed out the FFT data is heavily biased towards the lower frequencies which presents itself as a sharp falloff as you go from low to high. As a quick fix, I multiplied the FFT data by a section of a sine wave and leveled off the data. This put the majority of the action in the middle of the waveform instead of the left side.

I then applied an averaging pass which smoothed out a lot of the spikes. Next, a decay attack and falloff was added to keep the spikes from popping in and out too frantically. Finally, I threw the whole data set into a 2D ripple algorithm to give the whole thing a bit more of an organic feel (one of these days, Id like to retranslate this back into audio to see what it might sound like).

Suddenly, the once frantic FFT data was turned into smooth rolling peaks that still reflected the quality of the incoming audio. The look was somewhat landscapey so I decided to push that aesthetic by showing the history of the audio (in the video example below, I am keeping track of the last four seconds of audio). I then added in some appropriate textures and got a nice scrolling landscape that responding fine to real-time audio input. It is particularly entertaining to swing the camera around to the back of the passing landscape and turn it into a rudimentary flight simulator over dynamically generated, audio driven landscape.

Audio-generated landscape from flight404 on Vimeo. Definitely recommend seeing the HD version on vimeo.com. Audio by Akufen (”Even White Horizons” off the album My Way).

And yeah, it does look a bit Bryce3D. Next time, maybe I will even throw in some crystal spheres rising out of a mountain lake! Retro-Awesome!!

(My apologies if the previous description is both too simple and too complex. I don’t actually know much about FFT and how it is derived. I know what the data looks like and that is plenty for me to play around with it.)

I have always hated New York City. I was never able to see the charm. I have only been there a handful of times and I am always overjoyed when it is time to return to San Francisco. New York City is for the young. New York City is for the patient. I am neither.

However, this most recent trip was different. On this trip, I tried to loosen up and appreciate the pros and do my best to ignore the cons. This was no easy chore but I think I succeeded. I spent time with my beloved East Coast friends. I ate at several fantastic restaurants. I saw some amazing art installations. The weather was perfect. This was the New York City all my friends told me about and I finally got to see it with my own jaded eyes. But of all the experiences I had this last week, there is one that stands out. A single moment that made the rest of the week pale in comparison. It was the moment I realized Paul Smith owed me an apology.

It started simply enough. Lance told me we should swing by the Paul Smith store in SoHo because there was something he wanted me to see. A little background is probably necessary here. Paul Smith is Lance’s favorite fashion designer. He even based one of his blog designs on the iconic Paul Smith stripes. Before I met Lance, I hadn’t even heard of Paul Smith. Lance showed me his genius. A short while later, I bought my first and only suit. It was a Paul Smith and I love it immensely.

We wandered down the hyper crowded cobblestone streets of SoHo-on-a-weekend and made our way to the Paul Smith boutique. We don’t have a Smith store in San Francisco so I was looking forward to seeing his collections in person. Lance led me over to a shelf and pointed at a long sleeved shirt. (just noticed the website refers to it as a sweater… but its pretty much a sweat shirt). It was red with a print of flocking birds in white. “This reminded me of your work.”

It really did. I snapped a picture with my iPhone and put it out of my mind. It was a nice print reminiscent of my work and nothing more. We left empty handed. Back at the hotel room, I pulled out my iPhone and looked at the photo again. It bugged me that it seemed so familiar. Plenty of people have done flocking experiments with code but I have come to find that the personality of the coder is often reflected in the flocking behavior exhibited by the birds. My flocking code differs from Psyop’s. Psyop’s differs from Reynolds’. And so on.

I opened up my laptop and made my way to my Flickr account. I looked over the three images I posted last year of hundreds of birds in flight that I made with Processing. At first, I saw no direct relationship. The Smith design simply looked like another iteration of the work I had done. It nagged at me. It seemed implausible that someone else could have come up with a flocking behavior that had so many aesthetic similarities to mine. Then I saw it. A circle shaped negative space on the edge of a half-circle of flocking birds. This detail was present in both.

“Oh my god oh my god oh my god…”

“What? What is it?”

“I think Paul Smith stole my work!”

I pointed out the detail to Lance who confirmed my suspicion. The more we looked, the more the neighboring details fell into place. Smith’s version was mirrored left to right so I loaded the image in Photoshop and flipped it. “Oh my god! He totally stole my work!” I was dancing around the room. “Paul Smith stole from me!” I will admit it was a strange reaction. I didn’t realize this until later in the day. I was actually thrilled that someone had ripped me off. Someone I liked. That sincerest form of flattery bit doesn’t mean much if the person doing the imitating is making shit work.

“We have to go back!”

“Why?”

“I have to buy it and I want to take better pictures so I can be sure.”

Back at the store, one of the salesmen recognized us from our earlier visit. He asked if he could help us. I pointed at the shirt and said I wanted one. He sized me up as a large and grabbed a couple and led me to the changing room.

“If the arms are too short, we can find you another one.”

I put the shirt on and it fit fine. I couldn’t stop smiling. The whole thing was so damned surreal. Here I am in a Paul Smith changing room trying on a shirt that features a design element stolen from my Flickr site!

“It fits perfectly,” I told the salesman. “It’s like I made it myself,” I joked and smiled at Lance.

“You did make it yourself,” the man replied, oblivious to the inside joke but wanting to play along.

Lance asked the salesman if he knew anything about the print on the shirt. He said something about hobos and the passing of knowledge or something. I was too distracted to pay attention. I said I will take it and he led us to the cashier. $235 later, I was walking out of the store with my very own personalized Paul Smith shirt.

Back at the hotel, I took a couple more pictures with Lance’s camera. I loaded them into Photoshop, flipped them horizontally, and overlaid my image. I had to scale and rotate mine a tiny bit but easily found it to be a pixel perfect match.

In the following two images, cyan represents a perfect match between my content and the shirt content. White represents parts that were cropped out of my original image. The remainder of the red is from content added that isn’t part of my original image. So basically, cyan birds with red glows are stolen.

Amazing, right?! So many circumstances had to line up perfectly for me to even know about this design theft. My boyfriend happens to be a Paul Smith junkie. We both happened to be in New York City around the same time this article was released. Lance and our friend Tara both had to wander into the Paul Smith store. They both needed to be familiar enough with my work to notice the similarity and agree that I should probably see it with my own eyes. All the planets aligned.

Later that evening, Lance, Tara and I went out to dinner at Cookshop. I wore the new Paul Smith shirt. How could I not! At one point during the meal, I went to the mens room. As I stood at the sink, I looked up at the mirror and saw my collaboration with Paul Smith. I couldn’t stop laughing. Processing on a Paul Smith shirt! Casey and Ben are going to love this.

In the spirit of full disclosure, I must admit I am not completely innocent in all this. The birds I used in my coding study were culled from other people’s Flickr photos. I didn’t ask permission. I searched Flickr for images of flocking birds and traced a total of seven small silhouettes which I used in the original flocking study. However, I did not then go on to print my versions on clothing and sell them for hundreds of dollars each.

So what now? I have no idea. I am not angry. I am not feeling vindictive. I am flattered and amused. This isn’t like that Urban Outfitters/Johnny Cupcakes incident. I don’t actually feel wronged. I do feel that some designer for the Paul Smith brand committed an embarrassing act of laziness. This should not be excused and I imagine they will be dealt with accordingly. I don’t get a ton of blog traffic but I would be surprised if this didn’t eventually get back to the Paul Smith organization. And Mr. Smith, if you are reading this, Lance and I have always wanted to spend a week in London. Oh, and I am a 42L.