[A group of medical researchers has proposed using a new combination of presence-evoking technologies to provide personalized emotional support and distraction to reduce the levels of pain experienced by patients. This story from the Association of Health Care Journalists provides some of the details and includes links to more information. –Matthew]

[Image: Source: Getty via MobileHealthNews]

AI-cloned voices could be added to virtual reality to help patients in pain

By Karen Blum
February 14, 2025

When you’re in physical pain, listening to loved ones and looking at relaxing scenes may help comfort you and take your mind off of your ailments. That’s the idea behind a proposed study described in a recent issue of the Journal of Medical Extended Reality.

In their commentary, researchers at the Icahn School of Medicine at Mount Sinai and several other academic medical centers suggested combining a 3-D generated virtual reality landscape with artificial intelligence-cloned voices of a patient’s living or deceased loved ones to help reduce pain. The goal would be to provide personalized, emotionally supportive treatment to target psychological and emotional factors in chronic pain.

Virtual reality (VR) is a computer-generated simulation of a three-dimensional image or environment. People can interact with these images using electronic equipment such as a headset or gloves outfitted with sensors. VR has been used in other settings to try to reduce pain, offering snowy winter scenes for patients suffering from burns, or a VR headset plus a “breathing amplifier” to facilitate deep breathing exercises for patients with chronic lower back pain.

Here, the authors propose adding AI voice cloning technology to the VR environment. Commercially available voice cloning programs can replicate words and capture a person’s unique pitch, tone and rhythm of speech.

“Incorporating cloned voices from significant individuals, whether living or deceased, creates a more personalized and human-like audio experience, potentially influencing oxytocin pathways to alleviate pain and anxiety … in chronic pain patients,” the authors said.

How it could help

Oxytocin is a hormone that plays several roles in the body, including enhancing mood. Evidence indicates that a familiar voice, particularly from a loved one, can activate regions of the brain associated with social safety and emotional regulation, they noted. Voice cloning is available in over 20 languages.

Their vision is to use the AI-cloned voice as a narrator to help guide patients through a VR environment. For example, one of their prototypes allowed a user to generate a relaxing island setting with elements like flowers, trees and animals.

Of course, incorporating cloned voices from living and deceased individuals “raises substantial ethical considerations,” the authors noted. Therefore, while the technology “presents new opportunities in health care, especially for pain and anxiety management, it is essential to prioritize ethical frameworks and identity rights to ensure the protection of all involved.” The work would need to include “explicit consent protocols,” they said, but it’s still an interesting concept.

Next steps to move the technology ahead would include conducting a feasibility study to evaluate a personalized AI/VR pain management device, including surveying health care providers, patients and caregivers to assess their interest in the technology.

More uses of family voices for healing

It’s not the only use of voice technology to help patients. VoiceLove, a company founded by two physicians at Weill Cornell Medicine, is a HIPAA-secure voice application designed to help families maintain communication with those in intensive care units.

The technology, designed in 2020 during the first wave of the COVID-19 pandemic, turns a cell phone into a walkie-talkie and allows families to leave voice messages for their family member that can be played. The idea is not only to help families stay connected but to help reduce a condition called ICU delirium — confusion or trouble paying attention experienced commonly among patients with longer ICU stays.

Using voices to calm patients has been used before technology advanced. For example, researchers at Northwestern Medicine and the Hines VA Hospital found that playing recordings of family members telling familiar stories helped speed recovery for some patients in a coma. Coma patients who heard these stories four times a day for six weeks, via recordings played over headphones, recovered consciousness significantly faster and had improved recovery compared with others who did not hear such stories. Results were published in the journal Neurorehabilitation and Neural Repair.

Resources

• Researchers aim to study AI-generated voice cloning, VR to decrease pain – MobiHealthNews article.
• Artificial Intelligence Enhanced Virtual Reality: A Personalized, Multilingual Approach to Pain Management – commentary from the Journal of Medical Extended Reality.
• Healing ICU Delirium with VoiceLove: The Power of Connection – Becker’s Health IT article.
• App-Based Communication Tool Receives NIH Funding – Weill Cornell Medicine article.
• How to cover virtual reality applications in health care – AHCJ blog post from 2021.
• International Virtual Reality Healthcare Association — a resource to find out about VR conferences, current research and experts.