[If you’re a member of the ISPR Presence Community Facebook group you may have seen the impressive 48 second YouTube video that demonstrates the clever use of mixed reality described in this story from Fast Company. For more information see Greg Madison’s Twitter post. –Matthew]
In the home of the future, every surface is an interface
A UX magician maps a VR world over his apartment to create an entirely new interactive environment. Could your home be next?
By Mark Wilson
January 22, 2020
Virtual reality is still in its infancy. But thanks to Greg Madison, who is both a magician (literally!) and an interaction designer specializing in spatial computing at the video game engine company Unity Labs, we’re getting a taste of how burgeoning VR technologies might add up to something new. After spending 22 hours in 3D modeling software, he used the Oculus Quest to turn his apartment into a fully tangible, digital copy.
The demo would’ve been impossible until last year, when Oculus VR released the Quest headset. It’s the first VR product that’s both jaw-droppingly immersive and easy to use. The Quest provides a compelling window into a VR world, but it also provides key creature comforts, like mapping your room’s layout to provide a safe play space and even seeing your hands so you don’t need to use controllers. Most of all, the Quest can see the real world and digital world at the same time, which means the boundaries between the two get murky, fast.
This effect is what Unity calls an “extended reality” or Microsoft might call a “mixed reality.” Basically, it’s an environment that’s part real, part not.
As you can see in the video [in the original story or via YouTube], Madison built a system where he could sit at his real desk, or his real piano, or his real couch, but through his headset, he could see their digital shadows, and a whole secondary layer of screens and interfaces. It means he could literally walk through, and touch, a piece of software, because it’s built 1:1 atop his room.
Madison turns his desk into a giant touchscreen that puts an iPad to shame. He gives his keyboard digital sheet music that floats where a paper score would be. His armchair has a controller to create a virtual, floating screen. Oh, and it appears he can pull up video on a model of his actual TV, too. Perhaps that sounds cheesy. Why put video on a TV when it could float anywhere? But it looks perfectly normal in practice because it matches our existing expectation, that video plays on a device in our environment. And it’s a good example of how, while all these use cases might not blow you away Minority Report-style, they signal a more realistic future for VR where real and virtual objects coexist mindlessly.
“I wanted it to feel as comfortable as it feels to be at home, not in a disconnected VR space,” says Madison. “We are not Tom Cruise! Standing upright and making big gestures are the opposite of our everyday interaction habits.”
Madison didn’t build the demo for his day job at Unity. It was a side project, unaffiliated with the company. But he released it, he says, to serve as an “idea whisperer” and push the medium forward.
“[Soon] every single app will be redesigned for alternative realities,” says Madison. Imagine Uber repurposing the street signs on our block to hail rides, or Amazon displaying virtual clothing in our real closets. “We can guess that in the near future physical screens will begin to disappear and be replaced by virtual ones. Whether it happens in less than five years, or in 10 years…the best way to achieve that is to use tangible reality as a lattice.”
Everything Madison has described and demonstrated is possible today, but for now, that possibility requires being a graphic and interaction design expert who is capable of rebuilding his own living spaces as 1:1 digital shadows. The challenge now is not simply exploring these foundational technologies as they work in a vacuum, but making them scale to a shared global world. Though, until that day comes, I’d happily hire a service to give my office a VR makeover.