The iPad’s next killer app: Osmo connects iOS kids’ games with the real world

[From Gigaom, where the story includes several more images and a 1:52 minute video]

Osmo: Tangram puzzle game

[Image: Osmo’s camera mirror allows the iPad to recognize objects placed in front of it, extending game play into the real world.]

The iPad’s next killer app: Osmo connects iOS kids’ games with the real world

Summary: Osmo is combining simple but elegant hardware with clever computer vision technology to turn Tangram shapes, letter cards and everyday objects into iPad game controllers.

By Janko Roettgers
May 22, 2014

“Everything we are doing here is magic.” I’m not usually someone who embraces Silicon Valley’s tendency to paint every little product iteration as magical and revolutionary. But when Tangible Play co-founder and CEO Pramod Sharma uttered those words while demonstrating his company’s Osmo gaming device at our office, I couldn’t help but agree, and I couldn’t really stop smiling either.

That’s because Osmo, which debuted on with a self-hosted crowdfunding campaign Thursday, truly is magical, and has the potential to redefine how kids play and learn with mobile devices.

The gist: What Osmo is all about

Osmo is best described as an iPad accessory that brings real-world gameplay to the tablet. The device itself consists of a simple white iPad stand that props up the tablet in portrait mode, and a small red clip that covers up the iPad’s camera. Once in place, an internal mirror redirects the camera view point to an area on the table in front of the iPad that’s roughly the size of an open magazine. It’s that area that Osmo uses for gameplay.

At launch, Osmo comes with two packaged games with extra accessories. One of them is Tangram, the traditional Chinese puzzle game consisting of seven objects that can be combined to create an endless number of shapes. Just like the original, Osmo uses differently-colored wood shapes for its Tangram game. But unlike the original, this one also comes with an iPad app.

That app tasks players with solving ever-more-complicated Tangram puzzles by showing them the shapes on the screen. Players then pick up the physical shapes, and arrange them in front of the iPad, which gently provides feedback, and offers up some clues for harder levels.

The second game, simply called Words, is essentially Osmo’s version of Pictionary. It consists of two sets of letter cards in two different colors, allowing you to play with or against another player as well as in single-player mode, and yet another iPad app. The Words iPad app shows you a picture and tells you how many letters the word you are looking for consists of. To play, you simply start to put letter cards in front of the iPad, and the app automatically starts to count the right and discount the wrong answers.

Words is a simple game, but it shows some of the technology behind Osmo at work. During our demo, I first tried to align all of my letters neatly in front of the iPad, only to be interrupted by Sharma. That’s not how kids play, he told me. Instead, he just dumped a handful of letters on the table, and Osmo quickly recognized all of them, regardless of alignment and orientation. It’s a pretty addictive game, and very impressive technolgy as well.

But the real magic of Osmo revealed itself when Sharma showed me a third game, one that comes without any additional objects. Newton simply works with any white sheet of paper and a pen, and of course yet another iPad app.

The goal of this Osmo game is to redirect balls that are falling on the iPad screen to hit other objects on the screen. Except you don’t do the redirecting on the screen, but on your piece of paper. Draw a line, and the iPad app automatically imports it into the game. Draw another one, and you might have a funnel for those falling balls. Now add a physical object — a key, a pen, a toy, your own fingers, or any object, really — and Osmo automatically registers its outlines and adds them to the game as well. It’s augmented reality meets scribbling, and it’s a whole lot of fun.

The idea: using proven games to connect the iPad to the real world

Sharma told me that one of the inspirations for Osmo was his five year-old daughter, who is “constantly glued” to the iPad. Sharma wanted to help her embrace real-world play, and also give her a chance to be more social and play with others — something the iPad doesn’t exactly lend itself to. At the same time, he recognized that some traditional toys and games have started to lose appeal to a generation that’s growing up with the iPad. “Lego has lost the battle,” he argued, adding that it’s increasingly being replaced by Minecraft.

That’s why Tangible Play decided to marry the two. Sharma’s team purposefully concentrated on existing and proven games for its launch: “They are good for kids, we know that.” Of course, you could also play Tangram without the iPad. But Osmo is trying to make games like it more interesting, he said, adding: “It’s not if you can play it. It’s how long you can play it.”

Tangible Play has been testing Osmo in a number of schools over the last couple of months, and Sharma said that watching kids play with it and getting feedback from teachers helped the company shape its vision. “It has given meaning to what we are doing,” he added.

The challenge: kids don’t play under ideal conditions

The other inspiration for Osmo was Sharma’s previous job. Before launching the company, he worked for seven years at Google, where some of his duties included working on Google’s book scanning project. Back then, he started thinking about what else you could scan, and how you bring real objects into the virtual space.

Turns out, making this work in the real world isn’t actually all that easy. Osmo has to work under variable lighting conditions, on different kinds of tables and with objects that align in countless ways. Conventional image processing doesn’t work for this unless you force kids to always align their letter cards (the way I did the first time I tried it), require play on white backgrounds and impose all other kinds of unrealistic constraints.

Cornell University Professor Serge Belongie, who is a computer vision expert and who helped Tangible Play tackle some of these issues, described Osmo’s approach this way:

“We make use of statistical machine learning methods together with advanced image feature extraction techniques to accommodate real world variations in the patterns we need to detect, ranging from letters on cards to shapes of Tangram pieces. In this respect, the approaches we use are data-driven, taking into account real world noise, rather than based on ideal templates of the patterns of interest.”

The future: where Osmo goes from here

Two things struck me about Osmo when Sharma showed me the product. First, it’s incredibly well-designed, from the stand itself to the packaging to the delightfully minimal games that don’t come with a whole lot of directions, but instead encourage kids to explore and figure out how to play on their own.

Sharma credits his co-founder Jérôme Scholler, who previously worked as an engineer for LucasArts, with all of these design decisions, and explains that the goal was to build a kids’ game with the look and feel of an Apple product. The company succeeded with this, and I can see Osmo popping up in Apple stores once it’s commercially available later this year.

Osmo also has great potential for future games or even experiences that go beyond traditional gaming. Sharma told me that his team has “at least 50” ideas for additional games that could make use of the Osmo hardware. After playing with it for a few minutes, I couldn’t help but think about other apps as well. For example, what if Osmo allowed kids to arrange music by manipulating real-world objects? Or what if it transformed kids’ watercolor drawings into digital animations, without the constraints of touch screens that make us jump through hoops to recreate things that really work better on paper?

Sharma said that he doesn’t quite know yet how Tangible Play is going to make money. Maybe it will be hardware sales, maybe paid apps, maybe a mix of both. For now, Osmo is available for pre-order through a self-hosted crowdfinancing campaign, which will give early adopters a chance to buy a device for $50, as opposed to the $100 Tangible Play plans to sell it for when it retails later this year. The company has a very modest campaign goal of $50,000, which it will surely (and justifiedly) surpass in a matter of days, if not hours.

I know I’m putting in my order, and I can’t wait until I get one shipped in a few weeks.


Leave a Reply

Your email address will not be published. Required fields are marked *

ISPR Presence News

Search ISPR Presence News: