Finding freedom through deep compositing techniques
Many of the best practices and industry standards of today’s creative landscape have resulted from a willingness of VFX Studios and partners to share knowledge and foster technical collaboration in response to the rigorous challenges of modern productions. Deep compositing is one of the more groundbreaking time-saving techniques to take root across the film industry over the last decade as a result of collaboration by industry leaders and innovators including Weta, Animal Logic, ILM, Peregrine Labs and Foundry, whose introduction of deep compositing tools within Nuke made these powerful techniques available in a commercial software package.
With its increased accessibility in recent years, deep compositing is also now finding its way into more mid-sized studios seeking greater efficiency and freedom to make creative changes later in the production cycle. In the never-ending quest to cast off limitations, growing adoption of deep compositing is giving studios unparalleled flexibility to weave inventive artistry throughout their work across more areas of the production process.
Why deep compositing is changing the game
In contrast to traditional “flat” compositing techniques that provide less wiggle room for making late-stage changes without requiring tedious re-rendering, deep compositing uses a different data-rich format that gives studios more power and control over depth decisions later in the compositing cycle. With deep data, every pixel contains multiple samples at different depths, with each sample holding color data, camera-relative depth, and opacity information on a per-pixel level.
It's really useful for shots where there's lots of changes in depth or semi-transparent elements with overlapping depths that you don't want to decide up front in lighting which order they're going to be in,
explains Foundry CTO Jon Wadelton.
For example, if you had a complex scene with a full volume of smoke where you’d want to put a CG character inside that smoke, you’d normally use holdout mattes to create the desired visual effect. This would require rendering the smoke and element together, so that a portion of the smoke is both in front of and behind the character you’re rendering. While this works, it can be incredibly limiting as you are committing to placement of elements and lighting without seeing the elements in the context of the entire shot.
What deep compositing allows, is a special format you can render out of lighting that lets you combine those elements at comp time,
says Wadelton. So you can render the smoke without worrying about any elements that might be inserted into it, you just insert them in the comp and it just magically works.
Deep composting lets you manipulate the placement of the elements later on, including changing the depth, without having to re-render. This has tremendous practical use beyond volumetrics, too, as it’s useful in nearly any situation where you have many overlapping elements at varying Z depths that you want to combine in comp. Achieving this without deep compositing requires hold-outs that need to be merged in the correct order and can’t be changed without re-rendering.
The tradeoff is deep composite data is much heavier than traditional composites, but studios are finding the greater flexibility and control is well worth the challenges of handling the extra data. Weta Digital has since shifted to using deep compositing for everything, and other large studios are beginning to follow suit.
Big benefits drive growing adoption
In a deadline-intensive production cycle, getting things done faster, easier, and with less hassle is always helpful, but the benefits of deep compositing extend beyond productivity, too.
Deep compositing also gives compositors more creative choices,
says Christy Anzelmo, Senior Commercial Product Manager at Foundry. Essentially, deep compositing lets you delay more decisions, providing compositors with more options to achieve a seamless, natural looking composite and influence the look of their shots.
Bigger choices are being made at the point of comp far more frequently than ever before, particularly as modern VFX productions now lean on compositing as part of the creative process from the very beginning, rather than as a step at the end of the pipeline. The flexibility deep compositing affords is an enticing draw for more studios, now that the technology and processing power has caught up to make handling the extra data a far smoother affair.
I am becoming a bigger and bigger fan of deep compositing,
says Atomic Fiction VFX Supervisor Seth Hill, who used deep compositing extensively to deliver dynamic layered cityscapes for the sci-fi hit Blade Runner 2049. Being able to build a whole atmosphere in Nuke with layers of actual volumetric elements that are thrown in there in deep and layered all the way back just brings this reality, this edge, this life to it that you don’t get otherwise.
Weta Digital’s use of deep compositing on Rise of the Planet of the Apes also showcases how the technique can be used for far more than just fog and atmosphere. In one notable scene, a swarm of apes are rampaging across a bridge towards the camera. Deep composting allowed the Weta team to swap out characters, make depth adjustments, and manage the large number of layered assets weaving in and out of view throughout the intense sequence.
In developing the stunning opening title sequence on Deadpool, Blur Studio also put deep compositing to work to gain full flexibility for adding and remove elements without having to re-render every time. Adding assets was also really simple - we saved a lot of time since we never had to deal with holdouts,
says Blur CG Supervisor Sebastien Chort. We were able to just render each asset under the correct lighting condition and drop it into the comp.
Deep compositing adoption has been growing rapidly since the technology was integrated in Nuke, and the number of companies using it has roughly doubled in the last few years. That’s expecting to continue to rise going forward, now that processing power can handle deep data more easily, giving mid-sized operations the opportunity to tap into its limitless potential.
As we see the use of deep compositing growing across the industry, we are also looking at ways we can improve deep compositing within Nuke, both making it faster to work with deep data and giving artists more tools in their arsenal,
says Christy Anzelmo.
