Experience presence as you produce music and ride a hoverboard in VR

[Two more contexts for presence experiences – music production and hoverboard riding – are described in this post from The Creators Blog, which features more images and three videos. –Matthew]

Soundscape VR screenshot

Now You Can Make Beats in Virtual Reality

By DJ Pangburn — Dec 8 2015

In virtual reality, there is a great emphasis on visuals and haptic feedback. But apart from some established musicians like Jack White and The Who dabbling in VR, or the multisensory experience of Soundself, there’s been precious little done as far as exploiting the possibilities of sound in virtual reality.

Amsterdam-based creative coder and motion graphics designer Sander Sneek hopes to change this with Soundscape, a virtual reality music production environment. Soundscape is not a full-scale DAW (digital audio workstation), but something more like a virtual soft synth. Wearing Samsung Gear VR, users encounter a futuristic world where a monolithic synthesizer lies underneath a large cathedral-like structure, while in the distance there appears to be a moon.

Sneek created Soundscape for Oculus ‘s Mobile Virtual Reality Jam, which was created to find new content for their partnership with Samsung for Gear VR. Soundscape took the Silver medal in the Experiences category.

On huge screens, users can add musical notes, change effects and influence the characteristics of one of three instruments. Users can either do this solo or by jamming along with two other randomly-selected people. A seated experience, users input sounds via gaze tracking and clicking on the tracker on the side of the Gear VR.

“In virtual reality, I am trying to push the boundaries with current devices available like Kinect, Leap Motion or Wii Balance Board to get the most immersive experience and the best sense of presence as possible,” Sneek tells The Creators Project. “As a musician, music should always play an important role, since in VR audio makes 50% of the experience. Music and specifically creating music via software has always been a major interest of mine.”

Sneek created the visuals for Soundscape in Unity 3D. The audio is generated with drumkit samples and real-time synthesized sounds, which Sneek created from scratch. He also wrote a custom library for the Gear VR experience.

Soundscape features a Tonematrix, inspired by Andre Michelle’s VR ToneMatrix experiment that pairs with the Leap Motion haptic device. Sneek’s Tonematrix allows users to add notes, adjust two waveforms for unique synth sounds, and add four effects—reverb, delay, cutoff and a custom effect. The Drumcomputer allows users to add and trigger drum samples and add reverb, delay and cutoff. A Tone RPG allows users to create “deep, continuous synths” by adjusting waveforms and effects.

Sneek added visualizers for the sound—birds in the air that grow in number depending on amount of notes created in the Tonematrix. He also programmed global lighting that reacts to the Drumcomputer’s drum samples, and the Tone RPG’s real-time generated waveforms appear on the synths’ monolithic screens.

In future versions of Soundscape, Sneek hopes to incorporate a sampler, extra Drumcomputer samples, and Volume and BPM controls. He’s also hoping to collaborate with the Amsterdam Symphony Orchestra using Soundscape.

Apart from this work, Sneek also recently released Hoverboard VR, a virtual reality experience that incorporates the Wii Balance Board (BB). Inspired by both Tron: Legacy and Back to the Future, Hoverboard VR works with an Oculus Rift and Xbox Kinect.

“You move in virtual space via your weight distribution on the Wii BB,” Sneek says. “When you lean to the left or right you will go forward or backward (depending if you are regular or goofy). While moving forward you will automatically follow the road, which makes it far more easier to steer but can cause some motion sickness. When you lean forward or backward you will strafe to the left or right on the highway.”

Sneek is also currently planning a collaborative music application based on the new Oculus Touch and HTC Vive controllers, which will include binaural audio. “I believe people will make music in virtual reality together,” he says. “People love to make stuff, like in Minecraft or with LEGO bricks.”

Click here to see more of Sander Sneek’s work.


Leave a Reply

Your email address will not be published. Required fields are marked *

ISPR Presence News

Search ISPR Presence News: