Katana's power helps deliver The Amazing Spider-Man

Director Marc Webb brings the comic sensibility akin to his previous hit indie breakout comedy (500) Days of Summer and combines it into an action packed classic Hollywood tale of teen angst turned superhero. Visually this film takes a lot of what was great about the previous films and adds a level of detail, playfulness and realism that really sets it aside as something totally new. 

Marc Webb is no stranger to visual storytelling. While his feature film showreel is singular, he is in fact a seasoned music video director with more than 70 videos under his belt. The overall Visual Effects Supervisor on the film was Sony Pictures Imageworks' (SPI) Jerome Chen whose previous credits include Godzilla and Contact. Interestingly this was also SPI’s first native stereo feature film and was very much designed to push and play with the limitations of the stereo format.

In total the film features a whopping 1,639 visual effects shots of which SPI undertook nearly 700. Stand out VFX talking points of this version are the scale of the setting (an entirely CGI generated New York City), as well as the creation of both the CG Spider-Man character and his nemesis, The Lizard.

Katana is one of the key tools used on the project and dealt with all of SPI’s look development and lighting needs. Katana is about allowing artists to manage the consistency of lighting and increasing complexity of their shots. It’s also designed to enable creative freedom. As films continue to push the boundaries there is a real demand for tools that work smoothly for high-end production use, enabling artists to achieve that “never-before-seen” look and achieve high end results. This is really Katana's remit.

Nick Loy was one of the first artists to be crewed to The Amazing Spider-Man, and worked on the show and its various incarnations over the past two and a half years. Nick was the look development lead on the film and had heavy involvement with the look development of The Lizard character.

Orde Stevanoski was the show compositing lead and sequence lead for a team of 25 artists on The Amazing Spider-Man. Under the close watch of Nick Loy, the team was responsible for the look development of the main villain in the film, The Lizard. As the show compositing lead, Orde was also responsible for the compositing pipeline in addition to developing Katana to Nuke integration tools and supporting all of the compositing artists that worked on the show.

Katana-sense

Orde describes Katana as much more than a lighting tool and is a big advocate for its benefits. “With its non-linear, non-destructive design, Katana comes as close as possible to a thought process,” he says. “We all know that as artists we crave intuitive tools that don't stand in our way and help us visualize and execute the ideas we have. Katana allows people with various skill sets and understandings of the inner workings to be proficient in performing their daily duties of lighting CG objects.”

One of the misconceptions around Katana is that it requires a steep learning curve, but Orde is quick to lay that to rest. “A novice in Katana can and usually does start lighting on their first day of use. It provides an automated way of pulling in all the bits needed, while allowing a seasoned artist to easily ‘pop open the hood’ for deeper exploration and customization - either through a node network via its built in python module or C++ plugins for heavy lifting. The scripting module allows full access to the tiniest bits of Katana's inner workings through its API, while the attribute script node can handle any scripting and automation that is dependent on the incoming scene data. This allows the python proficient artists to automate their workflows as much as possible, so they can create intuitive and Katana like user interfaces for their pipelines with macros and live groups.”

6th Avenue heartache

The Imageworks’ team created a fully digital version of New York City’s 6th Avenue and the surrounding several blocks. This required hundreds of unique buildings to be modelled, textured and lit.

The 6th Avenue shots contained an intense number of buildings each sometimes containing hundreds or thousands of sub-components,” explains Orde. “Some scenes were literally dealing with several million unique shapes. Katana allowed us to handle these monumental scenes with grace and once a reasonable divide-and-conquer strategy was in place for rendering, allowed us to plow through hundreds of complex shots in a timely manner.”

He continues: “In Nuke you could click on a building in a final render and inspect information about how that render was created—even loading the shapes directly into Nuke for post treatment. Each render of New York City had a supporting database written out via attribute scripts running at render time. Supporting Nuke gizmos were also developed to allow artists to use a pixel picker. This meant that the data could be easily read inside Nuke to quickly identify buildings, their scenegraph locations inside Katana, or even load the geometry inside Nuke for post treatment.”

Organisation is such a key part of working with Katana. In an effort to support rapid look development for their assets, The Amazing Spider-Man team at SPI created a standard material library with pre-dialed materials for character and environment look development. Loy explains: “Our look development template included a simple turntable macro, standard bake section, and several render options. To better hone our materials, we evaluated our assets in a number of different lighting environments. We created scalable Image Based lighting rigs, wrapped them up into live groups, and exposed some user parameters to easily adjust the rigs as needed per asset. We also took advantage of Katana's advanced shading support to help develop complex surface materials.”

All of the CG nighttime New York City shots had thousands of lights. The lookdev department on the show did a lot of work to really simplify and cut down the setup time for the shot lighting. Loy continues: “We did a lot of the lighting in lookdev, essentially including the majority of our lights into the Katana Look Files. So car headlights, street lamps, streetlights, architectural building lighting, miscellaneous prop lights and so on, all came in for free when an lighting artist started a shot. An artist could kick off a first pass of the environment with the lighting 80% of the way there within half an hour or so. The Katana Look Files lighting was a big win for us.”

A coat fit for a lizard

Katana's power goes far beyond just the traditional aspects of look development and lighting. One example of this on The Amazing Spider-Man is The Lizard’s lab coat.

Orde explains: “For the purpose of creating The Lizard’s lab coat that gets shredded by bullets we had the animation team export bullet target locations that were read by python scripts inside Katana. This allowed us to drive multiple aspects of the material properties and feed those directly to the shaders. When a bullet hit the lab coat, the scripts would activate certain properties on the surface and then pass the bullet hole location in UV space to the shader in order to create an actual texture hole at that location on the cloth surface.”

“Furthermore, the dynamic nature of fabric that gets ripped on multiple occasions required correct motion vector information assigned to the new vertices. Since Maya's cloth solver didn't handle this aspect correctly, attribute scripts inside Katana were written to calculate correct motion blur for the dynamic geometry.”

The team also developed a Katana plugin for surface curvature estimation that was used to analyse the organic surfaces and write specific attributes that were passed to the shaders to drive certain properties of the surfaces. “These attributes allowed us, for example, to straighten cloth displacement wrinkles when the surfaces were stretching or folding in the skin simulation,” notes Orde.

Under the skin

Many of the departments at SPI have the ability to output additional data with the assets they produce. For example, the skin had properties beyond surface point positions, like stretch and compress deviations from the base mesh. The peeling lizard skin also had information that defined various aspects like the peeling edge, and the inner or outer sides. Even the bullet hits from animation passed their timing and position data.

All this data gets stored with the geometry in arbitrary channels of data along with position, reference position and vectors. Katana can then process this data, modify it on the fly and feed it to the renderer and the shaders. Orde comments: “This ability was used to assign different shader properties to the peeling skin as it was changing from clean skin to a dried husk for example, or dynamically placing textured holes for the bullet hits on the Lizard’s lab coat fabric. It was especially useful for dynamic geometry like the above examples since handling such properties with animated textures would be cumbersome. That’s where Katana is so powerful.”

A better world for artists

Orde likes that the leading industry packages for lighting and compositing are being developed by Foundry under the same roof. “I think this really helps with integration and allows the two packages to grow together and benefit from not having to invent everything multiple times.”

Orde welcomes the wider adoption of Katana. In recent months we have seen the software picked up by Mikros Image in Paris, Fido in Stockholm, New Breed in Montreal and Tippett Studios in Emeryville, CA. “Katana will make the entire industry a better place for the artists to focus on their craft instead of relearning the basic tools over and over,” he concludes.