A video clip that you can play: Indie rock within the Unity engine

For almost as long as video games have existed, they have had a close relationship with pop music. Back in 1983, Bally-Midway collaborated with Journey to make a game full of licensed songs and digitized faces of the band members (which followed more than a decade of pinball cabinets featuring megaton bands), and that says nothing about sensations from the media like “Pac-Man Fever.”

Meanwhile, interactive musical experiences, somewhat outside the realm of firm “games”, began to emerge in the CD-ROM era. This ranged from simple computer-only content embedded in a normal album data track to multimedia software complete with names like David Bowie and Prince.

Thus, the synergy of games and pop music is replete with several “first”, and this week, a modest music video by an indie band from Texas may not be registered as a big deal. It is not a Doom clone starring Iron Maiden or a hilarious light gun game starring Aerosmith. But this “playable” music video undoubtedly heralds a new era: one in which video game engines and, therefore, the gaming mindset have become totally fundamental in pop culture.

WASD with the beat

“Greatness Waitress” is the first single from Wait Action, Fishboy’s next seventh album. This longtime pop-rock group from Denton, Texas, compares favorably with bands like They Might Be Giants, Weezer and Ben Folds. In his most recent single, nasal vocals wistfully play a meta-narrative about a struggling indie rock band, and the words glide over the heavily percussive piano and the distorted guitar: I usually introduce myself, you should come and take a look / but you can’t take a look, the band is on a break / and the time it took us was specifically taken / waiting … for a great idea.

The single sounds appropriate for a dirty basement or a friend’s backyard, somehow simultaneously loud and intimate, with a lively teenage glee. His music video follows suit, featuring fictional and geriatric members of the band as cartoon characters rendered in 3D (designed by singer-songwriter Eric Michener) on a ruined stage. To get an idea of ​​what the band really looks like, a series of TVs display photos and video clips throughout the song.

It is the band’s first 3D rendered music video, but in the indie-rock style, this is not the result of a Pixar caliber computer rendering each frame at immaculate levels of ray tracing. Instead, the “Greatness Waitress” video was built using the immediate rendering flexibility of the Unity 3D game engine, and its limited geometry means that it will run on almost all gaming-capable PCs. To prove this, the band decided to keep the indie spirit alive by releasing their video as an interactive executable; you can even “play it” in a web browser. This construction removes the intentional cinematography from the YouTube version, allowing viewers to roam the environment with WASD.

Stay back and watch the entire band. Stay uncomfortably close to the vocalist. Or sift through the video’s geometry, cut out polygons and find Easter eggs.

God of rock + Mr Dunk

In an email interview with Ars Technica, Eric Michener of Fishboy said he had already applied the skills of his daily job as a freelance video editor to low-budget music videos. “I work a lot on After Effects, but somehow it never occurred to me to use a game engine that way,” he says.

This idea came about thanks to the encouragement of the director, artist and animator Dann Beeson, who connected with Michener via Instagram as a Fishboy fan. The duo came together for a number of things – they shared their love for the original Monkey’s Planet films, along with the singer’s experience with multimedia album designs (particularly the Fishboy albums that came with Michener’s own complete comic strips).

“I didn’t know he was a game developer,” says Michener. “I just saw cool 3D models that were their own works of art.”

In fact, Beeson has some serious cuts to his resume: more recently, he worked as the only 3D artist and animator for the beautiful NBA Jam homage Gentlemen Dunk, built alongside Andy Hull’s Spelunky programming fame. When Beeson and Michener started talking about a possible collaboration (which Beeson admits is a ploy to listen to a Fishboy album early), Beeson already had a workflow in mind: translating Michener’s 2D art into animated 3D characters; model, texturize and manipulate the “set” in Maya and Blender; and using Unity to compile assets.

“I’ve been playing games for a good part of a decade and never really thought about merging the two disciplines” of music and games, adds Beeson. But the process of applying a game engine to a video clip was a revelation, he says, especially compared to trying to do animation projects entirely on your own. “Rendering just one second of animation can take hours,” he says. “If you need a photo edit, this is your whole night.”

Meanwhile, “Greatness Waitress” worked as a project on a humble scale, requiring “about one night” to build looped animations for each modeled character. “The lip synchronization was done in a strange way,” says Beeson. “I found a way to draw motion, or puppet, on Blender. I ran the music and scaled a circle up and down to make it look like a mouth. It looked a lot better than it should.” It only took about 2 minutes and 30 seconds – “exactly how long the song is,” he notes. After framing the virtual setting for an intentionally filmed video, Beeson and Michener gave the assets a second pass for more interactive fun – including teasing about the “rock opera” story of the full album.

“Increasingly common”

Michener is careful when answering technical questions about the video and wonders out loud how many other video production projects have relied on popular, easy-to-use game engines. (If you are not familiar with the concept, Ars Technica previously addressed the cutting edge use of Jon Favreau’s Unreal Engine in film sets and TV series.) But for him, this lack of technical knowledge is part of the point.

“I love that you can look at this video as if it were a virtual show,” said Michener (not to mention how few of them we have enjoyed in the past 12 months). “I know this has been a little tricky lately, but probably not on a small scale like that for a small indie band like Fishboy.” In fact: only Michener and Beeson did any work on the video, with the singer praising Beeson’s ability to “spin with my ideas”.

“I saw a few other short films and demos made in Unity and Unreal, but my prediction is that this will become more and more common,” adds Beeson.

Thanks to its intentional simplicity, “Greatness Waitress” is unlikely to win the traditional “music video” awards. But how many music videos can you imagine that allow you to take control, live inside a miniature concert and see it from any perspective you want? At the moment, the response is limited; even 360-degree options and immersive VR for shows and videos tend to place viewers in specific seats, unlike Fishboy inviting viewers to hunt for secrets (and cut geometry along the way). But cheap access to Unity and Unreal will almost certainly change that reality, as more artists and musicians invent clever ways to replicate the concert experience in the real world – and as a harbinger of the upcoming interactive music fun, the This project’s enchanting accessibility is in fact its “Greatness”.

List image by Eric Michener / Dann Beeson