[Nvidia created a virtual version of its CEO to give part of a keynote speech, as reported in this PC Magazine story. Despite the after-the-fact critique of the realism of the illusion, it’s notable that many people either initially believed it or thought the doppelganger gave the entire speech when it was just a short segment (a VICE story helpfully says “if you jump to this part of the presentation you can see [Jensen] Huang magically disappear and his kitchen explode into multiple different 3D models). And even the PC Magazine story notes that “Given the ongoing advancements in technology, it may only be a matter of time before a company CEO’s speech can be convincingly deepfaked, for better or worse.“ –Matthew]
[Image: Source: VRScout]
No, Nvidia Didn’t Fool Everyone With a Computer-Generated CEO
On Wednesday, Nvidia touted how it had used a virtual version of CEO Jensen Huang during April’s GTC keynote. However, the CG-Huang only appears in a short scene, not for the entire presentation.
By Michael Kan
12 Aug 2021
Wait, did Nvidia actually use a computer-generated version of the company’s CEO to make an entire keynote address?
Uh, no. But a company blog post and the ensuing chatter on social media may have led you to mistakenly believe that. For a moment, we certainly did.
On Wednesday, Nvidia revealed that it used a CG-version of CEO Jensen Huang for his April GTC keynote address. A video from the GPU maker also documented the behind-the-scenes process to create the virtual Huang, which involved rebuilding his home kitchen in a 3D environment, scanning his face and body, and even hiring a motion-capture actor.
The news naturally made us wonder if the Jensen on display during the GTC keynote — which lasts over an hour—was an elaborate fabrication this whole time. Others immediately concluded the same, and believed Jensen’s talk during GTC was deepfaked or computer-generated.
However, Nvidia told PCMag the CG version of Huang actually appears for less than a minute during the video presentation.
“Huang was only virtual when he revealed DGX, and, briefly, during the transition to that moment; in all from 1:02:29 to 1:02:56 in the video,” a company spokesperson said in an email. (Nvidia has since updated the original blog post to indicate the virtual Huang only appears for a short moment.)
The CG creation is also easy to spot as fake, at least to us. During the keynote, the virtual Huang appears when his surrounding kitchen is transformed into a Star Trek Holodeck-like room. The virtual Huang then begins to talk, all the while making hand gestures.
However, the fake Huang’s facial movements and mannerisms seem slightly off and artificial. Nvidia also decided to show the CG Huang from a distance, rather than close-up, likely to try and hide the imperfections. So no, we doubt anyone was fooled by the CG creation when it first appeared in April.
Still, it’s noteworthy the lengths Nvidia went to engineer the synthetic Huang. The company took thousands of photos of the real Huang, which were used to create a CG model of himself. In addition, Nvidia was able to map and animate real photos of Huang’s face onto the CG model to make the appearance even more lifelike.
To help create accurate mouth and skin movements, Nvidia also came up with a technology that can use an audio clip of a person to drive the facial performance.
Given the ongoing advancements in technology, it may only be a matter of time before a company CEO’s speech can be convincingly deepfaked, for better or worse.
Nvidia’s spokesperson added: “The real breakthrough was that the transition—from a real person speaking in a real kitchen to a digital one—was the seamless transition, one made possible by animation driven not just by rendering, but by the speaker’s voice.”