One billion assets: How Pixar’s Lightspeed team tackled Coco’s complexity

Since its breakthrough work on Toy Story more than 22 years ago, Disney’s Pixar has been a mainstay and leading light in the world of CGI. Producing blockbuster after blockbuster, the California-based studio has consistently redefined and pushed the boundaries of computer animation. One such success was Coco, Pixar’s 2017 tale of a boy’s journey into the fictional Land of the Dead. The film won the Academy Award for Best Animated Feature, and Best Animated Film at the BAFTAs.

Foundry spoke to Renee Tam, Lightspeed Lead at Pixar Animation Studios, about Coco’s complexity — and how her team overcame these challenges to deliver a critically acclaimed animated movie.

Tell us about Pixar’s Lightspeed team — what’s its role and how long have you been involved?

Renee Tam (RT): “I’ve been at Pixar for around 12 years, and have been on the Lightspeed team since we created Brave (released in 2012). Quite a lot of people, even internally, don’t really know what we do — but we’re basically a global team solving technical problems regarding lighting.

“Sometimes we’ll be researching and developing new tools, or dealing with the performance of the show — usually focusing on the end of the pipeline, but also helping upstream as much as we can, for efficiency and optimisation.”

Why was Coco such a complex project?

RT: “We had a tonne of lights — the most we’ve ever had, in fact. There were 8.7 million lights, 27 million objects and 200 million shading points, with 1 billion file assets.

“One of my team’s key responsibilities is to look at how we make all of this work for the artists on these projects — we want them to be able to focus on the art, not the technicalities.

“Coco was especially complex as the movie has two completely separate worlds —t he Land of the Living and the Land of the Dead. Usually, you have one giant setting for the whole movie; one set of characters. It makes optimisation more straightforward.

“But, with Coco, there was two of everything. Miguel, the main character, even had three iterations — one living, one dead, and one in transition. And we had things like the Marigold Bridge that were intricate and needed a lot of attention to detail.

“There were a lot of new things we’d never seen or done before — the towers in the Land of the Dead, for example. The ones you see in the background are completely modelled, not just painted. You can literally fly the camera around the scene and zoom in and out and see all the intricate detail. Technically, it was very ambitious.”

How did that impact Lightspeed’s work?

RT: “One of the biggest challenges was making the whole thing processable through our current CPU and memory. When we first started rendering, we just let the computer run to see how long it would take, and it was taking 1,000 hours per frame for the towers. That’s something like 3,000 years of computer cycles to finish the render, so 70 years to render the movie.

“We managed to fix a few bugs and get it down to 450 hours per frame, which was a big milestone. We also figured out a smart way to split the foreground and the background, so memory could be lowered.

“Then we decided to bake the lighting for some of the towers in the background. But you can only bake from one fixed camera point of view, so you lose parallax as the scene moves. That was a problem because there are a lot of scenic shots in the movie and your eyes are smart, so you could tell if they were baked.

“Instead, I decided I wanted to do a 360-degree bake. Each tower was baked every 20 degrees on latitude and longitude — there are these trains flying around the towers, and you’ll see if you pay close attention that the train is following the track and the track is baked into the tower.

“In the end we got the render time down to 55 hours per frame with quite a few more bug fixes. We had a similar issue with Hector’s skeleton — the shading started out at 125 hours per frame, but we did a lot of work on optimisation and baking out what we could, and got that down to 30 hours.”

What were the most important lessons you learned throughout the process?

RT: “Always look to a simpler workflow. Baking is something we’re always pushing not to do — we want to push the hardware and software to its limit — but at a point you have to think backwards, to see the past experiences we can learn from. It can be a hard battle to fight, but there are lots of tricks from the past we can still use today.

“It’s also important to have a better sense of what’s coming down the line — on Coco, we initially didn’t think the skeleton shading and the Marigold Bridge were going to be too difficult. I’m trying to do more communicating to other departments upstream.

“Teamwork, too, is vital — and something I have had to learn. The larger the team, the harder it is to manage. It’s my job to make sure everyone has interesting work to do, and to balance that with some of the less exciting jobs is hard.

“But, despite all the complexities, Coco was one of the easiest, most enjoyable movies I’ve worked on — because of the team dynamic. From the Supe Tech, the DP, all department supes, the lighting leads, and Lightspeed team has been fantastic. I'm very lucky that I get to work with the people I do.”