[This first-person report on a new, life-size, interactive, intelligent assistant who appears in augmented reality and apparently evokes intense social presence experiences, is from Next Reality. The original story includes several images; a 1:43 minute video is available from Upload VR via YouTube; and a consideration of some of the practical and ethical implications of the new technology can be found in coverage from Daily Journal Online. –Matthew]
I Met Magic Leap’s AI Assistant Mica & Saw the Future of Augmented Reality
By Adario Strange
October 12, 2018
Unlike VR, when you’re talking about augmented reality, describing what an experience is like can be incredibly difficult — primarily because the experiences are even more contextual than relatively static virtual worlds that don’t involve real-world settings.
In AR, everything is about how “you” see things interacting with your real environment. Such is the case with what I’m calling the most important demonstration of Magic Leap technology to date in the form of an AI assistant called Mica.
The experience was previewed on stage during Wednesday’s keynote event at the L.E.A.P. conference in Los Angeles by Magic Leap‘s John Monos, vice president of human-centered AR and dDNA, and Andrew Rabinovich, the company’s head of AI.
Together, the team described a world in which a Magic Leap user will be able to interact with intelligent assistants in the form of fully realized augmented reality humans that can recognize your position in a room, as well as items in that room. Having mapped the area and your position within it, the AI assistant will then interact with you to help you do any number of things.
For example, as detailed in the presentation, the AI assistant might scan the Magic Leap One wearer’s eyes to detect his mood and then suggest an appropriate song to play through the home’s music system. Similarly, the AI assistant may access the Magic Leap One user’s preferences to adjust things such as the level of light in a room at a certain time of day.
We’re already becoming accustomed to such interactions on the audio plane via digital assistants like Amazon’s Alexa, Google Home, and, to a lesser extent, Apple’s Siri. But what Magic Leap is describing is putting an even more robust and responsive version of such a digital assistant in the form of a human that inhabits the same space as you, thus taking the assistant metaphor to its highest level. It sounds and looks a bit like science fiction, but it’s not.
What Magic Leap is describing is so close to reality, the company now feels comfortable enough to offer demonstrations of a rudimentary version of the dynamic at work with the Magic Leap One in conjunction with its intelligent assistant Mica.
The result is a stunning experience that takes AR into brand new and exciting territory.
I met Mica for the first time earlier this week. And if you get a chance to meet her, she will fundamentally change how your view the Magic Leap One and augmented reality in general.
When Magic Leap’s team brought me into an empty room hidden deep in the bowels of an LA event center, I didn’t know what to expect. The space was designed to look like a normal room, complete with a table, two chairs, and other furniture situated around the table. Nothing looked particularly futuristic or tech-enabled, so I wasn’t expecting much. Wow, was I wrong.
Upon donning the Magic Leap One, I’m greeted by a virtual woman (Mica) sitting at the very real wooden table. Then, Mica, with an inviting smile, gestures for me to join her and sit in the chair opposite her. I oblige, and then a very weird interaction begins — she starts smiling at me, seemingly looking for a reaction.
I’ll admit, I deliberately avoided smiling (though it was really hard, Mica seems so nice) and kept a poker face in an attempt to see if I could somehow throw the experience off by not doing the expected, that is, returning the smile.
Undaunted, Mica continued to look into my eyes and go through a series of “emotions” that, surprisingly, made me feel a bit guilty about being so stoic.
It’s at this point that I should mention that she doesn’t speak yet, so all of our interactions were conducted in silence, and instead of using words, she communicated using gestures, eye moments, and various body language. At first, I thought this might be a limitation, but in retrospect, I think this served to make the experience even more impactful.
That would have been enough to mildly impress me, but what came next was the kicker. She then pointed to a real wooden picture frame on the table, gesturing for me to hang it on a pin on the wall next to us. I did as asked, and… it was the eureka moment. This was a virtual human sitting at a real world table and she just got me to change something in the real world based on her direction.
But then it got better. Once I’d hung the empty frame, Mica got up (she’s about five feet six inches tall) and began writing a message inside the frame, which in context looked about as real as if an actual person had begun writing on the space.
Alas, I don’t remember what the message was (honestly, I was too blown away by what was happening), but I’m assuming it was somewhat profound, as Mica then looked to me in a way that seemed to ask that I consider the meaning of the message. After a few beats, the life-sized, augmented reality human walked out of the room. But she didn’t just disappear into a wall in a flurry of sparkly AR dust. Instead, she walked behind a real wall in the room leading to a hallway. It was a subtle but powerful touch that increased the realism of the entire interaction.
As I said earlier, it’s incredibly difficult to describe just how profound this experience was, but if and when it’s made available to the public, you’ll be doing yourself a grave disservice if you pass the opportunity up.
I’ve been trying to think of a tool or app that would compel me to wear the Magic Leap One for an entire day. And while I’ve had the device for months now, I haven’t been able to think of anything that would get me to wear it beyond one hour spurts of activity. That’s all changed now. Although battery life and the experience itself aren’t quite ready for such rigorous and extended use, I could easily see coming home and slipping on the Magic Leap One for the rest of the night if it meant having access to such a fully realized AI assistant such as Mica.
After meeting Mica, I have no doubt that this is what the virtual assistant future will look like for most people in the very near future. It’s not assured that it will be Magic Leap that delivers it, but whichever company does, I think it’s safe to say that Magic Leap was first to show us that future in this particular way, and it’s incredible.