Seeing Double with Weta Digital’s Gemini Man
Creating digital humans with Nuke and Katana
As we’ve touched on before, de-aging is becoming more widely used within the VFX industry. With more than five films using the technique last year alone, one that stood out amongst the many was Weta Digital’s Gemini Man.
Their goal was to “finally cross the uncanny valley and bring to screen a totally emotive digital human”, Guy Williams, VFX Supervisor for Gemini Man, tells us. While we have seen digital reincarnations in films before, Junior—Will Smith’s digital counterpart—is one of a kind, as up until now no studio has attempted to create a wholly digital human cast in a leading role.
We caught up with Scott Chambers, Compositing Supervisor, Alessandro Saponi, CG Supervisor, and Guy Williams at Weta Digital, to explore how they created a living, breathing and acting digital human in the form of Junior.
Creating a digital Will
The team at Weta Digital knew creating a realistic digital human was the key to the film being a success—Junior would be centre stage, with nothing to hide behind. So they started with the tech they already had and used it to create their first version of digital Will, saving scrutiny for later. From this, they could start to figure out what didn’t look or feel right, decide on the best way to fix these different aspects, before implementing them into Junior.
“Some of the things we tackled were a new system for creating and simulating the pores on a character’s face,” Guy tells us. “A new representation of the human eye, how it sits in the eye socket, proper representation of skin as a volume of light absorbing pigments, instead of a surface with simple textures and light scattering, and full integration of spectral light, surface and volume math in our system.”
They repeated this countless times, until Junior finally started to take form.
While the project was already unique due to the aforementioned digital human, it wasn’t the only obstacle the team had to crack. Gemini Man was also shot at a high frame rate, which proved to be impactful in various ways.
It had more obvious impacts like an increase in the render time by 40x, meaning the team had to find efficient and creative ways of getting approval without blowing their entire budget on a render wall. There were also more subtle impacts, like the team having to send thousands of frames for review which on a normal show may take five minutes, but on Gemini Man sometimes took almost half a day.
“Then, of course, there was paint and rotoscoping. We knew they would take a hit but it shocked us how much more time the high frame rate (and extra resolution and native stereo) added to the time to prepare the plates for compositing. In the end, it came down to very talented people leaning into the challenge and getting it done,” Guy Williams tells us.
Thankfully, the team at Weta Digital had Nuke on side to help with all their compositing needs which proved to be a saving grace, especially on such a complex and unique project such as Gemini Man.
“Nuke is the first port of call for cutting edge work,” Scott Chambers explains. “We were not only working in 4k, 120 fps but also in stereo which made for high precision compositing. Nuke rose to the challenge by being memory efficient and scalable—at all resolutions and frame rates.”
The team found that a large part of the compositing work for Gemini Man consisted of manipulating, repairing and often creating backgrounds from scratch. This was largely due to the use of stand-in performers, head rigs and, on occasion, bluescreens.
“The projection suite of tools in Nuke was invaluable to help us manage this work—not only in compositing but also our sister departments, Digital Matte Painting and Paint and Rotoscoping,” Chambers continues. “The ability to jump in and out of 2D textures, to camera projections while at the same time keeping quality and detail was a breeze for us with Nuke.”
Nuke also played a significant role in creating Junior, the team leveraging its toolset to assist in their pursuit of digital human excellence. Chambers tells us: “Nuke was integral to the fine detail work we had to complete. From subtle black level adjustments, rotoscoping and paint live manipulations in the comp—Nuke was able to scale effortlessly between brute force broad strokes and dainty, delicate refinement.”
Lighting the scene
However, it wasn’t just Foundry’s Nuke that helped the team create Gemini Man. Katana also stepped up to the mark, playing a fundamental role in the more complex lighting set-ups.
“With Katana, we were able to block the lighting across the shot very quickly in a coherent way, just as a DP on set would have done,” Alessandro Saponi tells us. “We were able to go back on individual shots and refine the lighting and the shading in a non-destructive way. This helped us to not only work on the shots but also to keep an eye on the progression of the entire sequence where Junior had a very complex range of emotions . . . The constant refining of the lighting and shading, via the exposed palette control, gave us the possibility to refine the entire sequence in the most minute details, producing an excellent result.”
What’s more, Katana allowed for the very complex set-ups to be condensed in a single node, with the right parameters exposed. This enables the team at Weta Digital to work in a more user-friendly way, alongside giving the Senior Technical Directors more flexibility to create tools themselves.
“We built the skin and blood compression and procedural wrinkles set up as part of the lighting template. The tools were able to check for production data in the database and turn themselves on or off, plus they could set any parameters in accordance with the production tags. This helped not only the TDs and the production team, but it also aided the render budget management.”
“With Katana, we were able to build a solid template for the lighters, either on a Show or Sequence level,” Saponi continues. “With that, the artists were able to be productive and were able to present the shots in dailies very quickly. This also allowed leads to intervene by adding render quality adjustments as required without having to interrupt the lighter’s work.”
Reaching the uncanny
All in all, making Junior took over two years. The team worked tirelessly to ensure Gemini Man was a success in their efforts to effectively cross the uncanny valley, using Nuke and Katana to aid in their conquest.
“Nuke was our sole package used for all of the compositing work on the show. We’ve used Nuke for some time at Weta Digital and have grown to rely on it for all of our compositing needs. It ties in well with our pipeline,” Guy Williams tells us.
“Same goes for Katana, which was used extensively on the show. All scene construction and lighting was done in Katana and Mari was used to paint the (very) high resolution texture maps for the young Will Smith.”
“We would struggle to do the show without these packages. I can’t even think of a stronger candidate to replace Nuke for compositing. It is just made right to allow us to build our pipeline tightly into it,” Guy continues. ”As for Katana, its flexible nature allowed us to leverage the success of one shot across an entire sequence easily. That allows us to spend more time making it look good and less trying to get it through the system.”
Be like Guy: free up time and unlock creativity today
Start your 90-day free Katana trial
Discover Nuke: the post-production powerhouse
