[Reminiscent of a well-acted scene from Star Trek: The Next Generation in which the android Data is given a ‘gift’ of laughter (see Laughing Squid for details and the video clip), researchers in Japan are training a robot to recognize and share laughter during conversations with humans. The details are in this story from The Guardian, with supplements from coverage in CNET. For an interview with Divesh Lala, a member of the research team, and a 3:16 minute video (also on YouTube) see a story in the Hindustan Times. –Matthew]
[Image: Erica being trained on how to have a sense of humour. Credit: Inoue et al.]
Scientists try to teach robot to laugh at the right time
Research team hopes system could improve natural conversations between humans and AI systems
By Hannah Devlin, Science correspondent
September 15, 2022
Laughter comes in many forms, from a polite chuckle to a contagious howl of mirth. Scientists are now developing an AI system that aims to recreate these nuances of humour by laughing in the right way at the right time.
The team behind the laughing robot, which is called Erica, say that the system could improve natural conversations between people and AI systems.
“We think that one of the important functions of conversational AI is empathy,” said Dr Koji Inoue, of Kyoto University, the lead author of the research, published in Frontiers in Robotics and AI. “So we decided that one way a robot can empathise with users is to share their laughter.”
Inoue and his colleagues have set out to teach their AI system the art of conversational laughter. They gathered training data from more than 80 speed-dating dialogues between male university students and the robot, who was initially teleoperated by four female amateur actors.
—
[Additional details from CNET:]
To gather training data on the frequency and types of shared laughter, the team tapped Erica, an advanced humanoid robot designed by Japanese scientists Hiroshi Ishiguro and Kohei Ogawa, as a platform for studying human-robot interaction. Erica can understand natural language, has a synthesized human voice and can blink and move her eyes when listening to humans go on about their people problems.
The researchers recorded dialogue between male Kyoto University students who took turns chatting face-to-face with Erica as amateur actresses in another room teleoperated the bot via microphone. The scientists chose that setup knowing there’d naturally be differences between how humans talk with each other and how they talk with robots, even those controlled by another human.
“We wanted, as much as possible, to have the laughter model trained under the similar conditions to a real human-robot interaction,” Kyoto University researcher Divesh Lala, another co-author of the study, told me.
—
The dialogue data was annotated for solo laughs, social laughs (where humour isn’t involved, such as in polite or embarrassed laughter) and laughter of mirth. This data was then used to train a machine learning system to decide whether to laugh, and to choose the appropriate type.
It might feel socially awkward to mimic a small chuckle, but empathetic to join in with a hearty laugh. Based on the audio files, the algorithm learned the basic characteristics of social laughs, which tend to be more subdued, and mirthful laughs, with the aim of mirroring these in appropriate situations.
“Our biggest challenge in this work was identifying the actual cases of shared laughter, which isn’t easy because as you know, most laughter is actually not shared at all,” said Inoue. “We had to carefully categorise exactly which laughs we could use for our analysis and not just assume that any laugh can be responded to.”
The team tested out Erica’s “sense of humour” by creating four short dialogues for it to share with a person, integrating the new shared-laughter algorithm into existing conversation software. These were compared to scenarios where Erica didn’t laugh at all or emitted a social laugh every time she detected laughter.
The clips were played to 130 volunteers who rated the shared-laughter algorithm most favourably for empathy, naturalness, human-likeness and understanding.
The team said laughter could help create robots with their own distinct character. “We think that they can show this through their conversational behaviours, such as laughing, eye gaze, gestures and speaking style,” said Inoue, although he added that it could take more than 20 years before it would be possible to have a “casual chat with a robot like we would with a friend.”
—
[Additional details from CNET:]
The Kyoto University researchers have already programmed their shared laughter system into robots besides Erica, though they say the humanoid howls could still be more natural sounding. Indeed, even as robots becoming increasingly lifelike, sometimes unsettlingly so, roboticists concede that infusing them with their own distinct humanlike traits poses challenges that go beyond coding. […]
Erica, needless to say, isn’t ready for the stand-up circuit yet. But it’s intriguing to think there may soon come a day when it truly feels like she gets your jokes.”
—
Prof Sandra Wachter, of the Oxford Internet Institute at the University of Oxford, said: “One of the things I’d keep in mind is that a robot or algorithm will never be able to understand you. It doesn’t know you, it doesn’t understand you and doesn’t understand the meaning of laughter.
“They’re not sentient, but they might get very good at making you believe they understand what’s going on,” she added.