We just did something we’re pretty pumped about. The December issue of Esquire is the first Augmented Reality issue of a major magazine, and we got to bring it to life. It’s something we started talking to Esquire’s editors about in early spring, and we’ve been quietly making it since this summer, so it’s pretty exciting to finally get to share it with the world.

How does it work? Two steps. First, go buy the latest issue of Esquire. Back? OK now go grab the app we wrote at http://esquire.com/ar (it works on both Mac OS X and Windows, but you do need a webcam).

Now hold the magazine up to your computer while the software is running and you’ll see all kinds of awesomeness based on the page you’ve got up. Hold up the cover and Robert Downey Jr. will leap off the page and start shouting at you. In the fashion story you’ll see Jeremy Renner dressed for all kinds of weather, and you can change the season by rotating the page. You can listen to a track from jazz musician Robert Glasper while you read the article about him, or see actress Gillian Jacobs tell a joke instead of just reading it. And if you ask her politely after midnight, she’ll even tell you a dirty joke.

If you haven’t seen it yet, Augmented Reality is the technology that makes all this possible, and it’s pretty sweet. It is showing up in a lot of different forms, but the general definition is pretty basic – just overlaying information or imagery onto the “real world” – usually images of the real world through a webcam or cell phone camera. In our case, we are using special markers printed on certain pages in the magazine, and our software can detect those markers and figure out in 3D where you’re holding the page and at what angle. Then we can overlay images in exactly the same spot you’re holding the magazine (like we do on the cover, for example) or create “mouseless” user interfaces, saying by letting you change the weather in the fashion spread as you turn the page different directions.

This is all the result of a three-way collaboration between the editorial staff of Esquire, the uber-talented designers at Psyop, and us. It was awesome getting to see first-hand all the exceptional talent that gets poured into an issue of the magazine, and collaborating with the brilliant design minds of Psyop was something we’ve wanted to do for a long time.

The whole process started when Esquire approached TBG looking for an idea that would make their December issue, also known as their Best and Brightest issue, be something really unique. We suggested Augmented Reality as a piece of technology that would fit the bill perfectly. We started to collaborate on what that might look like, considering the possibilities for an AR-ified version of magazine staples like the Funny Joke from a Beautiful Woman as well as articles specific to the issue.

Right around the same time experiments with Flash-based AR started to emerge on the internet. We knew we wanted to do something that would take AR to the next level in terms of visual sophistication, and to do that we’d need to use OpenGL to do real 3D graphics. Enter Cinder, our framework for creative coding in C++. You’ll be hearing more about Cinder in the future, but it was the key piece of technology that helped us pull all this off.

After we nailed down with Esquire the stories we’d do AR for, and the technology we wanted to use to do it, we began the collaboration with our friends at Psyop. In addition to providing their own insights on look and feel, they handled directing the video shoots and then of course the animation for several of the pieces. When it comes to visual creativity they are as good as it gets, and their talents really made this thing look next-level.

So far the press has been pretty excited as well. The Wall Street Journal previewed it a couple of weeks ago, and the editor of Esquire, David Granger was invited to talk about it on the Today Show last Tuesday.

Special thanks goes to our partners at Psyop for their beautiful design work, and to Esquire for having the balls to experiment.

Reading this description really doesn’t do it justice – you should put down the internet and go pick up the latest issue of Esquire to see it for yourself.