[Press coverage of a video demonstration of Meta’s latest avatar technology seems to be universally positive. In the video, podcaster and MIT researcher Lex Fridman interviews Mark Zuckerberg using the technology, as reported in the story below from Futurism. Fridman introduces the hour-long video in a post to his 3 million X (Twitter) followers this way:
“This was one of the most incredible experiences of my life. It really felt like we were talking in-person, but we were miles apart. It’s hard to put into words how awesome this was for someone like me who values the intimacy of in-person conversation. It gave me a glimpse of an exciting future with many new possibilities and fascinating questions about the nature of reality and human connection.”
BGR adds these observations:
“The reaction from Fridman [during the interview] was evident. The host was quite overwhelmed with how realistic the experience felt. While it can be tough to discern exactly what the experience is like when looking at a 2D screen, Fridman talked about how incredible the experience felt by simply looking at Zuckerberg and, despite them being far away from each other, feeling his presence right in front of him. He also noted how precise the expressiveness came across, enabling him to read Zuckerberg’s body language. He actually said, ‘I’m already forgetting that you’re not real.’”
And this is from Mixed:
“’This is the future,’ the MIT researcher says more than once in this conversation. … ‘It’s super interesting that this intimacy of conversation can be achieved remotely. … It just feels like an incredible transition to something else, a different kind of communication. It breaks down all the barriers, like geographic physical barriers.’”
–Matthew]
Mark Zuckerberg Shows Off Wildly Improved Metaverse with Lifelike Avatars
“It Just Feels Like We’re in The Same Room.”
September 28, 2023
By Frank Landymore
Heads Up
Mark Zuckerberg’s cartoonish metaverse that we all so relentlessly bullied just may be shaping up to be something truly impressive.
On Thursday, podcaster Lex Fridman released what he calls the “first interview in the Metaverse,” where he and the Meta CEO have a conversation in VR, using their astoundingly lifelike avatars.
Gone are the legless, dumb-looking Mii ripoffs. Here, while Fridman and Zuckerbeg sit in different rooms in different parts of the country wearing Quest Pro headsets, their in-Metaverse avatars, each a 3D portrait from the shoulders and up set against a black background, seamlessly chat back and forth while looking alarmingly like their real-life counterparts.
“It just feels like we’re in the same room,” Fridman said in the podcast, his avatar faithfully expressing a near deadpan. “This is really the most incredible thing I’ve ever seen.”
These photorealistic clones, known as Codec Avatars, have been a years-long endeavor of Zuckerberg’s.
They’re created through extensive scans of a user’s face, which are used to form a computer model, Zuckerberg said, packaged as a codec. The headset then detects the user’s facial expressions and maps them to the 3D avatar in real-time.
From there, “it can basically send an encoded version of what you’re supposed to look like over the wire,” Zuckerberg said, which he claims is more bandwidth efficient than transmitting video.
That all obviously puts more load on the hardware doing the rendering, but Zuckerberg says that the already available Quest Pro headset they’re using here can drive the technology.
Work In Progress
Credit where credit’s due: this could be a gamechanger if the final product lives up to what’s displayed here in everyday use.
But before you get your hopes up, note that this was not a live demonstration — and if we’re being mean — is showcased using two people who are notably not very expressive.
It’s also worth noting that the avatars displayed here were created using a state-of-the-art, hours-long scan comprising hundreds of cameras, something your average consumer isn’t going to have access to.
Instead, Zuckerberg says the plan for the future is being able to do a “very quick scan” using your smartphone.
“You just take your phone, wave it in front of your face for a couple of minutes, say a few sentences, make a bunch of expressions… and then produce something that’s of the quality of what we have right now,” he told Fridman.
We imagine that an avatar made using a smartphone scan will be a far cry from what’s shown here. Still, we have to admit, this is the most impressive Zuckerberg’s troubled metaverse project has ever looked.