UCI team adds AI to telepresence robot for homebound children for more natural experience

[As described in this story from UCI News, a team from the University of California, Irvine is working to improve the experience of telepresence robot users by incorporating AI to make the robot more autonomous; just as humans need not focus on every part of a complex but familiar task, the robot allows them to automate tasks. See the original story for a second image and a 3:53 minute demonstration video. –Matthew]

[Image: CARL SR, a robot being optimized for classroom use, leads lab members (from left) Hirak Kashyap, Tiffany Hwu, Giselle Tian, Jeffrey Krichmar and Lara Mirzakhanian in a display of school spirit. Credit: Luis Fonseca / UCI School of Social Sciences]

Building a better bot

An interdisciplinary UCI team is improving assistive robotic technology for homebound children, limited-mobility adults

By Heather Ashbach, UCI
June 24, 2019

If you’re in Social & Behavioral Sciences Gateway this summer, you may (or may not) be surprised to find a robot riding the elevator solo, delivering documents to the fifth floor and winding its way through one of many hallways. The intriguing automaton is CARL SR – short for Cognitive Anteater Robotics Laboratory Support Robot – a state-of-the-art robot on lease to UCI from Toyota Motor Corp. to be optimized for a specific task: helping homebound children and adults with limited mobility navigate everyday experiences, such as attending school and doing household chores.

“Children who can’t go to school due to either long-term illness or limited mobility have begun to rely on telepresence robots that livestream video of the remote student and the classroom, giving the student the social interaction that they miss when they must do their work from home,” says Jeffrey Krichmar, UCI cognitive scientist and director of the campus’s Cognitive Anteater Robotics Laboratory. “But current assistive robotic technology has its limits. It can’t move on its own or reach out and touch objects, and it requires constant Wi-Fi connectivity. All this reduces a child’s ability to naturally engage with a teacher and fellow classmates.”

Assistive technology for adults similarly leaves a lot of room for improvement, he notes, and applications developed through work with homebound children can extend to limited-mobility adults and elders.

CARL SR may very well be the solution. The project began in 2017 with a chance meeting between Krichmar and Veronica Newhart, a postdoctoral fellow in UCI’s National Institutes of Health-funded Institute for Clinical and Translational Science. She studies the impact of telepresence robots on home- and hospital-bound children but wasn’t satisfied with the existing technology.

“After working on this research for four years, I realized that it wasn’t enough to explore robot-mediated experiences for homebound children. We also needed to contribute to the design and production of improved technologies,” says Newhart, who earned a Ph.D. in education at UCI in 2018. “Currently available off-the-shelf robots were designed for adults in medical or corporate settings. We needed robots designed for children to engage in their real-world settings – school, community, extracurricular – on a daily basis.”

She approached Krichmar – who, with his lab, was already developing autonomous robots to meet this demand – about utilizing data collected from her user-centered studies to improve robot design, and an all-star research team was born.

Over the past year, Krichmar and his lab members have been adapting a robot designed by Toyota for household use to operate in a classroom setting. It has four onboard cameras, including one with high enough resolution to livestream a teacher’s whiteboard notes to a student thousands of miles away. The robot also has a movable arm with a grasping “hand” that can be raised when a homebound child has a question or wishes to contribute to a class discussion, reducing the need for verbal or other interruptions that may be necessary with current telepresence models. The same technology could help limited-mobility adults and elders perform common household tasks.

But what really makes CARL SR stand out from the crowd of assistive robotic devices is the very thing for which the UCI CARL team is internationally renowned: artificial intelligence modeled on the mammalian brain. Since joining the UCI faculty in 2008, Krichmar has received more than $4 million in public and private funding for his work in neurorobotics, which involves programming robots with real-life neural data to approximate thinking, moving beings.

“We design robots whose behaviors are guided by large-scale simulations of the brain,” he says. “And because these simulated nervous systems are embodied in a robot, they provide a powerful tool for studying brain function and developing smarter robots.”

Supported by over $300,000 from Toyota Motor North America, Krichmar tapped into the expertise of Tiffany Hwu, a UCI graduate student in cognitive sciences who received her Ph.D. this month. She enhanced CARL SR’s operating system with software that mimics the human brain’s neural networks – particularly those pertaining to contextual memory. The robot is now able to “remember” how to behave differently in a classroom versus a cafeteria or break room and which items are commonly found in each setting.

“The software is based on the idea that context is important,” Hwu says. To help, the team transformed the lab into a break room and a classroom. CARL SR, through trial and error, began to “learn” where it should go, if asked, to locate things that belong in a classroom – such as books and backpacks – or a break room – such as food and cups – and retrieve them.

Another lab researcher, Hirak Kashyap, a UCI graduate student in computer science, developed a software model based on the human visual system that allows the robot to predict where an object in motion will go.

“It’s similar to when a car goes behind a tree,” he explains. “We know it’ll come out on the other side, so we look there.” His model imbues CARL SR with the same predictive behavior. When applied, this will enable a telepresence robot to turn its head to follow a teacher or friend who may be moving, letting the remote operator spend more time engaging with peers than running the robot.

And thanks to undergraduate researchers Lara Mirzakhanian, who’s majoring in cognitive sciences, and Giselle Tian, a psychology major, CARL SR can be moved from point A to point B at the click of a mouse. This ease of navigation – coupled with the robot’s movable arm, grasping hand, and obstacle avoidance and contextual recognition features ­– means that homebound students can participate in the classroom in a more active and seamless way than permitted by other assistive devices on the market.

Recently, the team demonstrated that CARL SR could travel from Krichmar’s robotics lab on the second floor of Social & Behavioral Sciences Gateway to the elevator, ride up to the fifth floor and navigate to the social sciences dean’s suite – all with only a few clicks from the remote user.

Mirzakhanian and Tian were also part of an undergrad group that, based on their work with CARL SR, fabricated an inexpensive robot alternative from basic parts, kits and 3-D-printed components – suggesting that more inclusive, affordable access to telepresence robots may be well within reach. They, along with group members Mikayla Minton, a mechanical engineering major, and Steven Seader, a computer science major, presented the prototype in May at UCI’s annual Undergraduate Research Opportunity Program symposium.

“This is an ambitious project that addresses an important need,” Krichmar says of CARL SR. “And none of it would be possible without a strong interdisciplinary effort to accomplish our goals.”

Four members of the Cognitive Anteater Robotics Laboratory team will travel to Sydney next month to show off CARL SR’s capabilities in the annual RoboCup@Home Challenge. Follow the competition online at https://www.robocup.org/domains/3.

This entry was posted in Presence in the News. Bookmark the permalink. Trackbacks are closed, but you can post a comment.

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


  • Find Researchers

    Use the links below to find researchers listed alphabetically by the first letter of their last name.

    A | B | C | D | E | F| G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z