[From MIT’s Technology Review]
The Avatar Will See You Now
Medical centers are testing new, friendly ways to reduce the need for office visits by extending their reach into patients’ homes.
By Jessica Leber on June 10, 2013
Most patients who enter the gym of the San Mateo Medical Center in California are there to work with physical therapists. But a few who had knee replacements are being coached by a digital avatar instead.
The avatar, Molly, interviews them in Spanish or English about the levels of pain they feel as a video guides them through exercises, while the 3-D cameras of a Kinect device measure their movements. Because it’s a pilot project, Paul Carlisle, the director of rehabilitation services, looks on. But the ultimate goal is for the routine to be done from a patient’s home.
“It would change our whole model,” says Carlisle, who is running the trial as the public hospital looks for creative ways to extend the reach of its overtaxed budget and staff. “We don’t want to replace therapists. But in some ways, it does replace the need to have them there all the time.”
Receiving remote medical care is becoming more common as technologies improve and health records get digitized. Sense.ly, the California startup running the trial, is one of more than 500 companies using health-care tools from Nuance, a company that develops speech-recognition and virtual-assistant software. “Our goal is basically to capture the patient’s state of mind and body,” says Ivana Schnur, cofounder of Sense.ly and a clinical psychologist who has spent years developing virtual-reality tools in medicine and mental health.
Using Sense.ly’s platform, patients can communicate their condition to an emotionally reactive avatar through their phone, desktop, or TV. The avatar asks the patient simple questions, and if programmed by a doctor, it can answer questions too—such as what a diabetes patient with high blood-sugar readings should eat that day. The software also collects data from other medical devices that a patient uses, such as a glucose meter, and can capture gestures with a Kinect. The reports sent to the doctor include red-flag notifications to act on right away; charts, graphs, and analytics tracing the patient’s progress over time; and a transcript of the voice interaction.
“A physician’s time is always limited,” says Benjamin Kanter, chief medical information officer at Palomar Health in San Diego. “For a long time, we’ve had the challenge of just getting information into the system. Now the system is starting to actually help me.”
Schnur says one real advance is the avatar itself, which is important in helping both patients and doctors to trust the interactions. Molly, still a work in progress, can modulate her tone of voice and facial expressions. Schnur says that sometimes patients are more willing to share sensitive information with a nonjudgmental avatar than with a doctor.
Patients in San Mateo seem to like the interaction, Carlisle says, and he does too: “I’ve gotten used to the avatar. I look forward to seeing it when it comes online.”
The Sense.ly software, currently in beta, is also being tested at an addiction and detox clinic in California, doing patient intake and assessment in a crowded waiting room. Schnur hopes the system will eventually be used for even more complex tasks. The company, a product of the French telecommunication company Orange’s Silicon Valley incubator program, is working to include additional features, such as the ability to interpret and respond to a patient’s facial expressions.
Of course, doctors see some risks in such approaches, especially if the software makes an error or misinterprets an interaction. Kanter points out that although electronic systems often reduce errors, any errors that occur can propagate more quickly than those made only on paper.
Carlisle, who will enroll 50 to 60 patients by the time the study is done, is looking forward to getting more data. Over time, he hopes, not only will he improve the care of individual patients in their home environments, but what he learns from the data will improve therapy for everyone.