[From MIT’s Technology Review, where the story includes an audio version]
[Image: Flutter lets you play or pause music in iTunes or Spotify by holding up an open palm.]
Hold Your Hand Up to Play Some Music
Via an ordinary webcam, a startup puts simple gestural control in the palm of your hand.
Tuesday, April 3, 2012
By Rachel Metz
When Navneet Dalal listens to music on his computer, he doesn’t want any keyboard taps or mouse-button clicks to interrupt the experience. He’d rather just hold up an open palm to play or pause the tunes.
Dalal is cofounder of Flutter, a startup that offers simple software that lets users control ordinary computer apps with such simple gestures. Flutter emerges as gesture controls are becoming increasingly popular on consumer devices: Microsoft’s Kinect motion sensor brought it to the Xbox, and the company now sells a Kinect for Windows setup that encourages users to build applications using the technology. Samsung, meanwhile, has added it to some of its TVs.
Unlike existing gesture-control software that is only available on certain devices or that requires special hardware, Flutter harnesses a regular webcam. Anyone can download the free software, which was released in a public “alpha” testing phase last month (it’s currently available just for Macs). Flutter users can control the play and pause functions on iTunes and Spotify by holding up a flat, open palm.
If Dalal and his cofounder, Mehul Nariyawala, have their way, this will just be the beginning. With billions of laptops, smart phones, and tablets already featuring built-in cameras, and more on the way, Flutter envisions a not-so-distant future where you’ll be able to use gestures to control all sorts of applications and games. So far, the idea is catching on. Over 20,000 people have downloaded the software since it was released in March.
Flutter doesn’t use your computer’s webcam to look for movements; instead, it determines if an open palm is present or not. The program’s algorithms process the image information and determine what action to take. Again, those actions are currently limited to pausing or playing music or videos in iTunes and music in Spotify, but Flutter plans to enable more gestures and the ability to control more video content.
The company thinks gesture controls should be simple, and they want the act of controlling an application or game to feel natural. “We treat this webcam as the eye of the machine. And our thought process is that machines and humans should interact the way two humans with eyes should interact,” Nariyawala says.
While the Kinect requires time to calibrate when you start it up and is sensitive to sunlight, Flutter doesn’t need calibration time, Dalal says, and will work in any lighting conditions except complete darkness. Its software will also ignore background movement, such as someone walking behind you, he says.
But as the “alpha” designation suggests, it’s still quite finicky. I tried out Flutter on a MacBook Pro and found that while it could definitely play and pause music on command, it often interpreted my hand moving away from the screen postgesture as an additional gesture, and would turn the music back on after I had paused it (or the reverse). Nariyawala says they’ve solved many issues with the software, but are still working on eliminating these false positives.
Flutter may have to win over some skeptics, too. Sidhant Gupta, a graduate student and researcher at the University of Washington who builds and designs interaction and sensing systems, says the software doesn’t look much different from other efforts he’s seen from Toshiba and the Israeli startup eyeSight. But he does think it might work as part of a more powerful gesture-recognition system if combined with a Kinect-like 3-D depth sensor and sonic sensor.
Dalal and Nariyawala have plenty of additions in the works. In the coming months, they intend to make it possible for Flutter to control Netflix and YouTube videos and roll out additional gesture controls such as skipping forward or back and turning volume up or down.
They showed me a demo of a two-player video game they built. Players use their hands to intercept blue shooting stars that fly across the screen, blowing them up in a shower of red and yellow. The game is simple, but it required no more startup time than the average computer application and seemed to accurately interpret its creators’ gestures.
In the future, the company envisions charging for some advanced features. Flutter may try to make money by letting app developers use its software, with Flutter taking a cut of the resulting revenue. The company might also license its software to device makers so it can be included on new gadgets. For now, though, Dalal and Nariyawala are offering the software for free as they work out the kinks.
“We want it to be as smooth and seamless as possible,” Dalal says.