[Here’s a good example of the evolution of presence-evoking technology and (though it’s not discussed in this story from idealog), the obvious ethical issues it raises. The idealog story includes more pictures and two videos; MIT Technology Review’s coverage includes a 1:42 minute video of Nadia; and more information is available from the Soul Machines website. For a related story that raises some of the ethical issues, see “Can Alexa lie?” from Shelly Palmer’s blog. –Matthew]
Soul Machines unveils its first emotionally intelligent, lifelike avatar
By Elly Strang
17 Feb 2017
A Kiwi company at the forefront of humanising AI technology has revealed its first virtual assistant called ‘Nadia’, which has been voiced by actress Cate Blanchett.
Soul Machines is an Auckland-based company that develops intelligent, emotionally responsive avatars that improve the user experience on artificial intelligence platforms.
After receiving a $7.5 million investment from Hong Kong-based artificial intelligence and virtual reality investor, Horizon Ventures, the company formally launched in November last year.
Its technology is based on ‘Baby X’, a creation by Mark Sagar and his engineering research team at the Auckland Bioengineering Institute at the University of Auckland. Sagar is the CEO of Soul Machines.
Baby X is an emotionally intelligent virtual agent – and a digital simulation of an infant – which reacts to the person it’s communicating with via facial expressions and words in real time.
The technology is incredibly realistic, albeit quite disconcerting to some – as demonstrated by one comment on the Baby X Vimeo: “I think that as soon as the baby turned into a floating brain is when I realised that night-terrors would be in my near future.”
“Just about everybody who sees it is absolutely amazed that it’s actually a digital creation and not a video image of a baby,” Soul Machines chief business officer Greg Cross says.
Now, after a process of R&D of about five years, Nadia is the first commercial project to be launched with Baby X’s technology.
It was developed for the NDIS (National Disability Insurance Scheme) in Australia using IBM Watson’s artificial intelligence technology as a cognitive back-end and FaceMe, an Auckland-based real-time video communication company.
Nadia can speak, write and chat online, and was created to help disabled people that traditionally struggle with technology interfaces have better accessibility to the company’s services.
The project took 12 months from start to finish, with people with disabilities involved throughout the entire process.
Soul Machine’s avatars, such as Nadia, are created using incredibly detailed physiological models, with every bone structure and muscle moving exactly as it would on a human.
The avatars even have their own version of a central nervous system that imitates the capabilities of the human brain to make it more realistic.
“The core theory behind our technology is our faces are the mirror image of our brain. You can’t create a realistic face without creating models of the human brain as well,” Cross says.
“Our avatars also have a breathing model because when we speak as human beings, we have to manage our sentences based on when we have to take a breath, so our tone of voice or expression on our face may change slightly.”
The way Nadia works is when NDIS customers go on the company’s website, they can choose to interact directly with her.
The webcam on their computer will act as “the eyes” of Nadia, while the microphone will act as “the ears”, Cross says, resulting in a human-like conversation with an online avatar.
“The unique parts of the avatar technology that we’re responsible for is the ability for the avatar to be emotionally responsive,” he says.
“If Nadia detects that you’re upset about something, she may respond instantaneously to be more empathetic, for example.
“This is the way in which we’re utilising the human face as a next-generation leisure experience between humans and machines.”
Nadia is designed to resemble a real person in looks, while her voice may be quite familiar to some movie buffs.
The NDIS brought on Australian actress Cate Blanchett to be the human voice behind the avatar.
Blanchett was recorded saying thousands of sentences containing various phonemes, which can then be repurposed and made into new content for the avatar.
Outside of Nadia, Cross says Soul Machines is working on other projects across New Zealand, the US and Europe, but companies are tight lipped on releasing any information about them.
“Because it’s at the leading edge of digital innovation, companies we work with are prepared to talk in their own time and in their own way,” he says.
While there’s other artificial intelligence avatars out on the market, Cross says the level of detail and research put into Soul Machine’s creations is putting it at the forefront of the market.
“In terms of avatars that have this level of emotional intelligence and emotional responsiveness, Mark’s research [which informs Soul Machine’s designs] is very, very special.”