[From Disvcover’s Inkfish blog]
Illusion Makes People Speak with the Voice of Their Avatar
By Elizabeth Preston | November 26, 2014
Think you’re in control of your own body? A simple virtual-reality session could not only make you feel like an avatar’s body is your own, but make you speak more like the digital character.
First there was the rubber-hand illusion, a classic experiment that showed syncing up someone’s touch perceptions with what they see happening to a rubber hand can make them feel strongly that the fake body part is their own. Since its publication in 1998, this study has inspired all kinds of spinoffs. Some involve entire “rubber” bodies—researchers have swapped people into the bodies of giants and Barbies, and used virtual reality to make them feel child-sized. By playing the sound of a hammer hitting marble while they lightly tapped subjects’ own hands, scientists have even made people feel like their hands are made of stone.
Mel Slater, a researcher at the University of Barcelona, has been studying these illusions at his computer-science-meets-neuroscience lab. This time you might say he was interested in a rubber voice. The rubber hand and body illusions can blur people’s notions of where their bodies start and end. If people saw themselves in a virtual avatar’s body while they heard and felt that avatar speaking, would they start to change their own voices?
Slater and his colleague Domna Banakou put 44 subjects into motion-capture catsuits and virtual-reality headsets. In their virtual environments, people looked around a room, gazed down at their bodies, and studied themselves in a mirror.
For half the subjects, the avatar’s motions perfectly matched their own. When a subject raised her hand in front of her face or hokey-pokeyed her foot, the avatar did the same thing. The other half of the avatars moved around by themselves. Subjects still saw through the avatar’s eyes, but they weren’t in control of it.
As expected, syncing up the avatar’s movements with someone’s own body created a powerful illusion. Subjects tended to agree that the avatar’s body felt like their own, whether they were looking down at their hands and feet or waving at themselves in a mirror. When the avatar moved on its own, the illusion didn’t work.
As people watched, their avatars also moved their lips and spoke a series of words. In a movie of the experiment, this is pretty creepy. A blank-eyed digital character opens her mouth and utters a string of words as if it’s a coded message from the afterlife: “Casa. Mes. Sopa. Vela.” (“House. Month. Soup. Candle.”) People whose movements hadn’t been synced with the avatar’s said the voice seemed to come from somewhere in the room. But the synced-up people felt like they were speaking the avatar’s words themselves.
For the voices of the avatars, researchers had recorded a male and female speaker whose voices were higher than the average Spaniards’. Then they’d tweaked the frequency of the recordings to make their voices 10 percent higher still.
Both before and after the virtual-reality session, experimental subjects read aloud the same list of words that the avatar spoke. Eerily, people whose avatars had matched their movements spoke in a higher voice afterward.
For good measure, the researchers had subjects wear a small device on their throats that vibrated when their avatars spoke. This would give them an extra tactile impression that they were speaking too. The vibrating device contributed somewhat to the illusion, Slater and Banakou found, but what subjects saw was more important than what they felt.
Slater thinks the illusion says surprising things about our feelings of agency, or control over own own bodies.
“According to theories of agency, people infer authorship of actions based on their prior intention to carry out the act,” he says. If we meant to do something, and then it happened, we must have done it. In the experiment, people saw their avatars speak without intending to speak themselves. So in theory, they shouldn’t have felt any agency. Yet they were easily swayed by the illusion and reported that the body and words felt like their own.
Slater’s guess is that after seeing the first few words spoken by the avatar (“Casa. Mes. Sopa”), the subject’s brain concluded that the person was speaking. If you’re looking in a mirror and seeing a person whose movements match your own, the simplest conclusion is that it’s your own reflection. Then the brain prepared the subject’s real body to keep speaking in that voice. “Hence when the person later spoke, they spoke with the computed higher frequency,” Slater says.
For now, this is only a hypothesis. The lab’s next experiments may help answer the question of what makes us feel agency over our bodies. If not, at least they’ll provide more fodder for your next existential crisis (or ghost story).