Bridging the uncanny valley: what it really takes to make a deepfake

It is no secret that Artificial Intelligence (AI) is changing the world as we know it. From robots to self-driving cars, it has firmly cemented itself as a fundamental part of technological development. 

The most recent trend sweeping the AI scene is the creation of digital humans, something we touch on here. In fact, it seems we have become obsessed with crafting the most lifelike virtual beings possible in a bid to bridge the uncanny valley. This brings about a series of moral and ethical questions due to the increase in digital fakery surfacing, which is having a tangible impact on the VFX industry and its future. 

Better known as deepfakes, these falsified videos and images depict people saying or doing things that they didn’t do. From humiliating famous people to imitating politicians, deepfakes are becoming increasingly prevalent on social media.

This is largely due to how accessible and inexpensive this technology is becoming. Apps like FakeApp allows users to create rather convincing deepfakes in a few simple steps, manipulating them in any way they see fit. Even Adobe released research into VoCo, software that lets you seamlessly alter what someone is saying.

While these deepfakes aren’t as lifelike as those seen in films, perhaps they don’t need to be. With them being largely distributed online and in social media circles, these videos only need to be convincing enough to make people question what’s real and what isn’t—planting an unshakeable seed of doubt in their minds.

Plus, as the technology advances, these deepfakes are set to get more realistic and further push the edges of the uncanny valley—making it more difficult to decipher real from fiction.

It’s like looking in a mirror 

In a world of increasingly spectacular blockbuster films, we, as an audience, expect to be amazed by fantastical elements that clearly transcend the realms of reality. In fact, we almost demand to be deceived, delighting in the new worlds that we’re immersed in through a cinema screen. This expectation is what digital humans are derived from, alongside Hollywood’s quest to create the perfect human, offering the next phase of trickery in film. 

But conquering the uncanny valley has never been easy, and creating digital humans is very much the same. It is a long and complex process, and there are mounting concerns that the rise of inexpensive deepfake technology undermines this process. It is not as simple as clicking a button. Instead, it takes a lot of dedication to make these virtual beings believable as real people.

For Ang Lee’s Gemini Man, it took Weta Digital two years to perfect a digital reincarnation of 23-year old Will Smith, or “Junior” as he was aptly named. Whilst this isn’t the first time a younger version of a star had graced our cinema screens, with Star Wars even bringing back Peter Cushing from the dead, Junior is one of the first digital humans to lead his own film—a new rarity in the world of Hollywood. 

It took hundreds of artists, each studying the intricate details of Smith’s face to finally bring Junior to life. The process involved poring over reference shots from his early film career, as well as creating a database of Smith’s different facial expressions and movements. Every detail was scanned, recorded and saved, from his teeth to his skin and pores—no part was left untouched.

A vital part of this was perfecting the eyes, as they are what give digital humans life. They’re the windows to the soul, after all—and artists have to spend hours studying and perfecting every minor detail of an eye, something the team behind Junior painstakingly endured. The slightest misdirection could mean the difference between a realistic digital human and an empty vessel with no-one home. 

Of course, this isn’t the only factor in bringing a digital human to life. It’s more a symphony of small things all working harmoniously together to create a lifelike virtual being. They possess a fine amount of intricacy in their creation, which takes craftsmanship and technical skill that amateur deepfakes can’t necessarily compete with. They don’t have the capacity or technology advanced enough to achieve this.

Seeing double

However, there is a vital difference between digital humans seen on the big screen like Junior, and online deepfakes: their purpose. 

In films like Gemini Man, Smith’s virtual self is a storytelling device; in fact, he is the story. If Junior had looked fake or out of place in any way, it would have drawn viewers away from the story and break the immersion of the film. 

This is especially the case since the human eye has a unique ability to know when something doesn’t quite look right. It can be as simple as a wrong move of the head or a subtle glance; we know that what we are seeing is wrong—even on a subconscious level—despite not being sure why. 

This is often the trap that amateur deepfakes fall into: they fail to bridge the gap between real and unreal—something artists spend years perfecting.

Nonetheless, the quality of these deepfakes doesn’t matter as much as it does in films. To start, they are not being hosted in high-definition on a large cinema screen, but more importantly, the audience may simply not be as invested—due to lack of knowledge or perhaps lack of caring. 

Even if people are aware that something might seem wrong about a video, they may still choose to believe what they’re seeing is real. This can be due to political agenda, uncertainty, or even because they dislike those in the video. 

This in itself opens up a whole host of issues.

Take the recent videos of Donald Trump and Barack Obama which have been circulating online as an example. While these are somewhat comedic in content, people can—and have—used this technology in a more incriminating fashion and to manipulate common sentiment. This is why there has been much concern with the forthcoming 2020 election and the implication it has for fraud. 

It’s Uncanny

Despite the recent hype, creating believable digital faces is hard. The blood, sweat, and tears poured into Gemini Man’s Junior by Weta Digital’s swathes of artists is testament to that. As time marches on and tech continues to advance, bridging the uncanny valley to make man and machine near indistinguishable remains firmly in the realm of skilled, experienced artists—often for the big screen, where digital fakery is designed to delight, not provoke.