[This would seem like an ideal application area for presence; the first story below, from ZDNet, describes the new Mondly language learning VR app, and the second, from The Next Web, provides a first-person user’s evaluation. For more information see the company’s press release via PR Newswire. –Matthew]
Mondly launches virtual reality for learning languages, powered by chatbots
Practice talking in 28 different languages with virtual characters, which teach language fundamentals and help build confidence, with the Mondly VR app
By Eileen Brown
February 28, 2017
Romania-based ATi Studios, the creator of the Mondly language learning platform, has released a virtual reality app for language education.
The VR app uses Mondly’s conversational voice chatbot, released by the company in August 2016, which has self-learning technology.
It combines voice chatbot technology with speech recognition in virtual reality to create a new way for users to learn a new language.
The Learn Languages VR by Mondly app allows people to experience lifelike conversations with virtual characters.
The company created the VR application based on what users need to effectively learn a new language: instant feedback and an “immersive, real-world experience to build their confidence.”
The technology uses speech recognition that understands phrases and words in 28 different languages. It has learned millions of new phrases and responses since its release, due to its self-learning technology.
Its language-learning platform has been downloaded over 15 million times. The VR application intends to realize virtual reality’s promise of immersive and automated educational experiences.
It gives instant feedback on pronunciation and suggestions to add to a learner’s vocabulary, and it offers ‘surprises’ to turn learning a language into a fun experience.
The app aims to bridge the technologies behind chatbots, speech recognition, and VR with a new automatic voice detection system so that the speech interaction feels natural in a VR environment.
The system calibrates to the room’s background noise and ignores unintentional sounds.
The first version of the app offers three scenes across 28 languages, such as a restaurant scenario where users can practice ordering food and drinks in a small bistro.
Other scenarios include the Eurotrain, where users can have a conversation and make new friends on a train in Europe, or they can check into a boutique hotel and secure a room with a view.
Alexandru Iliescu, the CEO and co-founder of ATi Studios, said: “Because VR is so immersive, we quickly realised that traditional speech interaction models that require a tap or a voice command to enable speech recognition would kill the flow of the experience.”
He added: “So we developed our own automatic voice detection system. The result is amazing, the conversations with the virtual characters happen as naturally as real-life conversations – they just flow.”
—–
[From The Next Web, where the original includes more images…]
Mondly’s VR language-learning app is the closest thing to actual immersion
by Bryan Clark
February 15, 2017
Experts agree that language learning is best accomplished with full immersion. Most of us, unfortunately, can’t pack up and head to Tokyo for a year to learn Japanese.
Mondly, a language-learning app on iOS and Android, thinks VR could help bridge the gap. And today it’s launching an app for Gear VR owners that, in my experience, is the closest thing you can get to actual immersion without putting pants on.
First, some background. After my first year in Mexico, I was struggling to pick up the language. I used Mondly, and others, to improve my vocabulary and help me to speak more conversationally. Neither worked. Aside from the canned phrases offered on Duolingo, Mondly, and others, I really couldn’t advance the dialogue into something resembling conversation.
My problem wasn’t vocabulary — which these apps are wonderful at building — it was recall and recognition.
Recognition came with time. It’s simply a matter of understanding that native speakers don’t sound like the voice actors on your app of choice, and altering the way you hear patterns of speech. As for recall, there’s really not much you can do to recall words and phrases faster other than to actually use them. Unfortunately, the apps commonly available only train you to use your knowledge in the context of a canned conversation. If your real-life conversation partner deviates from the script, you’re left scrambling to find the right words.
Immersion works. I can attest to that. But what about immersion via virtual reality?
Mondly is a bit gimmicky at first. The lessons are typical of those you’d find on the existing Mondly app, but unlike the iOS app, for example, the background starts to fade and it feels closer to a real conversation. Granted, it’s a cartoon woman on a train asking what my favorite sport is, but the rapid-fire question and answer sequences force your brain into something resembling real immersion.
It’s not perfect.
The microphone is quick to snap you back into reality each time it makes listening mistakes such as hearing “he’s my potato” when you’re attempting to say “he’s my dad” in Spanish. Aside from some temporary frustration, though, it’s about as close [as] we’ll get to real immersion, at least for the time being.
Virtual reality offers real opportunity for language learners, and while Mondly’s app is nice, it’d be hard to imagine a scenario where a major player in the space — looking at you Duolingo, or Rosetta Stone — don’t jump into the fray with something far more elegant.
Still, for a free app it’s definitely a cheaper option than travel.
Mondly’s Gear VR app is available today in the Oculus App Store.