[This blog post from the Huffington Post argues that the transformative power of VR (and thus implicitly presence) is not in visual displays like Oculus but in “VR for the hands.” –Matthew]
What Nobody Will Tell You About Virtual Reality
Steven Schkolne, Caltech VR PhD, internationally renown artist, and 3dSunshine founder
02/24/2016
There’s a lot of hype about virtual reality these days, and most of it is wrong. Science fiction films and books have created a hunger for Oculus-style, head-mounted displays (HMDs) and “immersive” experiences, but does wearing a pair of goggles really have more power to transport us? Based on my 10 years of experience in the field I don’t think so.
The VR revolution will not be experiential, but rather enabling. VR is a feast for our hands, not our eyes. Curious? Read on.
The flagship of today’s VR craze is the Oculus Rift, which is a stereoscopic, wide field of view, head-tracking device that blocks out the world just like the movies promise. In our collective cultural consciousness, VR is the headset. Sensational as the effects of HMDs are, it turns out that the additions they lend to the viewing experience don’t actually add more information than a human can gather from looking at a more traditional image. If Bruce Willis is taking down a helicopter with a motorcycle on your TV, he’s still doing that in the Oculus as well. It’s the event that’s exciting, not how you’re seeing it.
I hate to break it to everyone, but great societal upheavals — like the Internet and smartphones — were generated by advances in our ability to act on and process the world. VR headsets show us what we can already see in a photograph but more sensationally. This is not enough to change society. VR for the eyes, while certainly entertaining, is unlikely to create new industries.
So, when you read headlines that The Future of Travel Has Arrived: Virtual-Reality Beach Vacations, Google Cardboard Saves Baby’s Life, Father Witnesses Son’s ‘Miracle’ Birth In Virtual Reality, realize these authors are making a cognitive error. They believe a VR headset is necessary to see a 3D world in a computer. Somehow, these authors forget that people have been living inside 3D worlds since the days of Doom (1993) and Super Mario 64 (1996), with nothing more than a monitor and a mouse (or joystick) to look around. The brain is where reality resides, and the experience a father has watching his child’s birth over Skype is just as profound. We can already experience intense psychological immersion in both 2D and 3D worlds. Just ask the 100+ million players of World of Warcraft or anyone who has been dumped on FaceTime.
Nevertheless, financial influencers like Goldman Sachs have made bold predictions about VR/AR. Will it actually net $80 billion in the next decade? I’d wager not. VR for the eyes, while sensational, offers no new information to the viewer. VR is more aptly comparable to 3D televisions than smartphones.
VR for the hands, on the contrary, empowers humans like never before. If you want a magic thread to follow to find VR gold, just follow the hands.
While I was doing my PhD at Caltech in the early part of the millennium I laid the foundations for what would later become Tilt Brush and Quill. What I found, is that far more important than the ability to see 3D space, was the ability to manipulate it with your hands in real time. My program, Surface Drawing, allowed a profound human experience: for the first time people were able to create meaningful 3d creations without learning highly technical software like Maya. In researching my new project, 3dSunshine, our team has discovered that users don’t even need expensive displays, just the motion controllers.
The mouse was the first VR input device most of us touched. Unlike arrow keys, with the mouse you can move up, right, down, wherever you want with rapidity. The mouse brought our hands into the 2D reality of my computer screen, and it absolutely changed the way we use computers.
There’s a new kind of VR input device that does what the mouse does, but for 3D worlds. It’s called the 1:1 (“one to one”) motion controller. Hold one in your hand, and whichever ways your hand moves, it moves that way in the 3D world. This technology is dead-simple to use, and — most importantly — enables us to do things that we can’t with a mouse or touchscreen. Like the Nintendo Wii controller, perfected.
This new generation of 1:1 motion controllers, pioneered by the Playstation Move and Razer Hydra, is being brought into maturity by devices like the Oculus Touch and Vive controllers. It is with these new input devices that we see a dimensional difference in what we can do. Mice track our hands with 2 dimensions (x, y). Next generation motion controllers measure not 3, but actually up to 12 dimensions at once (each hand has 3 positional and 3 orientational dimensions). Channeled appropriately, this massively higher dimensional input stream allows us to specify much more simultaneous information to a computer.
We live in a visual culture, and often forget that our hands got us here. We use our voices for social communication. Everything else we do — whether lifting, typing, or suturing — we do with our hands. Hands are how we work. VR for the hands has the potential to change how we all work in virtual space.
If you want a bold prediction for VR, I offer you this: VR for the eyes will fail. People will try the headsets, love them, but neglect to use them again week after week (unless perhaps they are extremely low-cost in terms of both money and convenience). VR for the hands will become a mainstay of how we manipulate digital spatial data, as soon as the mainstream gets a taste for how empowering this technology truly is.
To accept this pill, you have to set your expectations for VR slightly below the rabid hype you see in today’s media. This is not a new internet. VR for the eyes may supplement the entertainment industry, but it simply cannot change the way we interact with the digital world the way VR for the hands will. As developers begin to take advantage of the 1:1 motion controller, we will see changes that science fiction has yet to predict. Sight is our primary sense, but it’s time we all started paying a little more attention to our hands. It’s how we get things done.