Re-writing the rule book on Spider-Man: Into the Spider-Verse
How Sony Pictures Imageworks created the film’s signature graphic look almost entirely in comp
Spider-Man: Into the Spider-Verse wowed the world at the 2019 Academy Awards, picking up the gong for Best Animated Feature.
Much has been written about the unique look of the film, which evokes the feel of a comic book (right down to the Ben-Day dots).
We spoke to the team at Sony Pictures Imageworks (SPI) about how they customized Nuke, Katana, and Mari to re-configure their traditional pipeline, breaking the mould and creating a one-of-a-kind masterpiece in the process.
In the second article of this two-part series, we explore how the studio harnessed the power of Nuke.
Re-writing the rule book
Spider-Man: Into the Spider-Verse is by far the most comp-heavy animated feature SPI has ever done.
The signature Spider-Verse graphic look was created almost entirely in comp, which meant every shot had an extensive amount of work done in Nuke.
The comic-inspired look of the film was a complicated process. It required a completely new toolset, so the team did a lot of work developing and refining the process into a series of granular tools and templates.
Crucially, these had to be artist-friendly enough to be utilized on every shot in the movie — as Geeta Basantani, Lead Compositor on the project, explains:
“We created over 25 individual tools for the compositors, and probably just as many templates. The most widely used were the Hatcher and Thresher tools, which are used to create the most recognizable printed comic look of the film, in the halftone dots and hatch lines.
"But we also had tools for offset colors, the accent ink lines, outlining, smearing, brushing, streaking… the list goes on! Pretty much every part of our traditional pipeline was reworked in some way to achieve the unique look of Spider-Verse.
"Because so much of the look of the film was found in comp, we gave the artists many toys to play with, and they were always finding new, exciting things to do with them.
"For instance, ChromaShifter was a tool originally created for handling the color offsetting effect we used for depth of field, but we found that by swapping motion vectors for depth and pushing it to 11, we got that really unique motion-trail look on things.
"The final look of the film really comes down to the creativity of the artists working with all of these tools.”
Custom tools to empower artists
Much like the original comics, Spider-Man: Into the Spider-Verse is the work of true artistry.
The extensive range of new tools developed on this project were created with one aim in mind: to empower the artists to be as creative as possible. Armed with this arsenal of bespoke tools, the team painstakingly hand-crafting the unique look on each and every shot.
Lead Compositor Marco Recuay explains that the customizable nature of Nuke made it the perfect package for the job, saying: “Developing tools in Nuke has always been extremely easy — from simple artist-created gizmos, to elaborate Nuke Developer Kit (NDK) nodes, there is support for whatever needs we have.
"On Spider-Verse, we took advantage of mixed toolsets. Some of the more complicated tools were written in a very granular way in BlinkScript or C, and then wrapped up into multiple artist-facing gizmos or templates, to present them with the specific controls needed for the task they want to accomplish.
"This was useful for things like the multiple looks on the show. In Gwen’s world, for instance, things are very streaky and painterly, so while the underlying tools were the same as Miles’ world, we had gizmos and templates that presented and used them differently.
"In other areas, we would take a complicated setup created by our lookdev artists and boil them down into a gizmo, just exposing the controls the artists would need to work with those looks across a sequence.
"This was a really important process for our show, as we had to make the tools accessible not only to dedicated compositors, but also lighter-compositors who aren’t always as comfortable working with complicated setups.”
Creating a truly handcrafted look
As Geeta explains, these tools were integral to the entire VFX production: “Pretty much the entire movie was done with the artist dialing shots on an individual basis, using Nuke with the tools we developed.
"Like for hatching and halftones, using the Thresher and Hatcher tool to dial them thick and thin based on the shading from lighting. The artist had freedom to dial the size, angles and space between the screentones — also dial depth of field using our tool sets.
"For instance, in the train station, the flares had halftones which scaled with light and changed the appearance of them based on how the realistic flares react. Halftone dots were sprinkled to represent the glows.”
Marco goes on to elaborate on the process, saying: “The Spider-Verse look is a fairly complicated process to pull off, and required a lot of artistic work to dial things just the way the directors wanted them.
"For example, the halftone dots you see in rim lights on a character. In some instances, you want those to track along with the character, such as when Miles is swinging by on a web, otherwise he’ll appear to be swimming through a pattern.
"Other times, you want them to be a little looser, so they feel like lights and not a texture. Being able to adapt the tools to whatever the shot needs meant we had to give the artists a lot of control, and train them on different ways to use the tools.
"So, while there was a lot of new things to learn, the benefit was that we were able to create a truly handcrafted look on every frame of the film.”
Using BlinkScript to prototype tools
The team used BlinkScript extensively to experiment with the different tools they might need to achieve the desired look, as Marco explains: “We used BlinkScript as a way to quickly prototype tools in Nuke.
"The ability to essentially code directly to the viewer is great, and allowed us to create more complicated results quickly.
"In our workflow we would then convert the BlinkScript to a compiled node that would be used as part of the artist-facing toolsets used across the teams.”
Check out the first article in our two-part series to find out more about the look development, lighting, and texturing process in Spider-Man: Into the Spider-Verse.
