[The company Zylia has developed a new set of immersive audio technologies that expand the potential for evoking spatial presence. The original version of this VRScout story, which was “created in partnership” with Zylia, includes four different images and a third video (“Navigable 360 music streaming experience with the Poznan Philharmonic Orchestra”; also available via YouTube). An August 2020 story from Audio Media International adds this detail:
“’Applications of 6DoF audio and our technology are wide. Cinematic trailers for VR projects, naturally sounded audio for games, live performances recording, artistic audio content for domes with multi-loudspeakers installations, or virtual presentation of unique places are just a few examples,’ says Tomasz Żernicki, CEO of Zylia. ‘As a listener you can freely move throughout the recorded scene going closer, further, up, or down from the sound source. Moreover, spatial resolution is so high that you have a feeling of being there.’”
–Matthew]
[Image: Title screenshot from Zylia video “The Walk Through The Music.” Source: YouTube.]
Zylia’s 3D Audio Technology Could Be The Start Of Virtual Concert Halls
A collaboration with the Poznań Philharmonic Orchestra signals a new era in live-recorded audio for virtual reality creators.
By Kyle Melnick
December 14, 2020
The immersive audio experts over at Zylia are all about pushing the boundaries of 3D audio recording, whether it be for academic research, art, games, music, sports, VR, and everything in between. Now the company is upping the ante even further with 6 Degrees of Freedom Navigable Audio capable of generating 3D sound for “virtual concert halls,” an immersive auditory experience unlike anything available at the moment.
ZYLIA 6 Degrees of Freedom Navigable Audio, as it’s referred to by the company, allows operators to record musical performances, Q&A’s, panels, and other live audio by capturing all audio within the spherical sound field of each and every microphone and recreated in VR. What does this mean exactly? Put simply, Zylia’s 6DoF Audio solution allows you to create your own audio path by physically moving throughout your real-world environment with a VR headset or a mouse and keyboard if operating on a computer. Each time you change positions in relation to the sound sources, the audio will change in volume and direction, just like it would in real-life. Music too loud? Take a few steps back and enjoy the show from the nose-bleeds. Looking for a more intense experience? Join your favorite artists on-stage and jam shoulder-to-shoulder.
“The mission of our company is to help artists and to bring artists and their audience together. That’s why we developed this navigable audio system in 6 Degrees of Freedom – to connect them in the virtual world,” said Tomasz Żernicki, CEO of Zylia. “When we look into the future, what we see is a completely different world of interaction with creative art. We see streaming platforms enabling people to participate in concerts of any artists they want.”
“Platforms that they can enter from their homes and join the experience with their friends. We see hyper-realistic games, both visually and audibly, allowing players to fully immerse themselves in the plot,” he adds. “Finally, we see virtual reality creations, available for everybody from anywhere in the world and limited only by the imagination of their designers. ZYLIA Navigable Audio is a huge step in that direction, and we are very proud of the impact that we are having on the future of 3D audio technology.”
Earlier this year, Zylia showcased the true power of its 6DoF audio technology in a collaboration with the Poznań Philharmonic Orchestra in Poland in which the company used 30 ZYLIA ZM-1S spherical, ambisonic microphones with synchronization to capture a 360-degree soundscape of a live performance. This third-order Ambisonics microphone features a 19-capsule microphone array with high-end 24-bit recording resolution. That means crisper, cleaner high-fidelity audio recordings over longer distances.
“To record the Poznan Philharmonic Orchestra performance we placed in the concert hall 30 third-order Ambisonics microphones ZYLIA ZM-1S with synchronisation,” explains Żernicki. “We put them on stands and placed them on the stage among musicians. The microphones have been connected with computers through simple USB HUBs. We used 2 laptops for recording and one PC Linux workstation as a backup. These 30 microphones gave us 600 audio channels – 20 for each ZM-1S. As a final result, we gathered 3 hours of recordings and 700 GB of audio data.”
The team then used Zylia software to mirror the physical location of each of the 30 mics within the virtual space, allowing listeners to explore this space to experience the music from multiple angles, just as they would at a live venue. You can check the full performance in the video provided [in the original story or via YouTube]. You can also view a 360-degree demo in VR using a PC VR headset, and check out an exclusive behind-the-scenes making-of here.
Apart from ZYLIA ZM-1S, the Philharmonic project was brought to life using a set of Zylia software, including the ZYLIA 6 Degrees of Freedom Recorder, software which allows for simultaneous audio recording on ZYLIA ZM-1S microphones; the ZYLIA 6DoF Synchronizer, software used to upload audio recordings and synchronize them with one another; the ZYLIA Ambisonics Converter, software which converts ZM-1S multichannel recordings into Higher Order Ambisonics; and the ZYLIA 6DoF HOA Renderer, a plug-in that allows for the processing and rendering of ZYLIA Navigable 3D Audio content.
“The border between the musician and audience was existing for ages,” said Łukasz Borowicz, the Principal Guest Conductor of Poznań Philharmonic Orchestra. “Once you pass this border you are entering a real musical world. This kind of experience is an experience I cannot describe. You have to feel it. In the last months, we faced a problem of making concerts without a public. We’ve been broadcasting the concerts but it produced a lot of problems on how to show the acoustic of the hall.”
“This is something very important in the ZYLIA system – it makes it possible to listen to the orchestra not only from one fixed place. You can go to the balcony or the last row and acoustic changes. I think people are very open to new technologies and notice that the experience of the sound which comes to you directly is one of the best experiences you can have. Technology seeks the possibilities to reproduce this real feeling and I think this technology which was used for our recording shows that ability.”
In addition to the above-mentioned use-cases, ZYLIA’s 6 Degrees of Freedom Navigable Audio solution can also be used to create immersive multi-point 360-degree audio-video experiences. When combined with 360-degree cameras, for example, ZYLIA mics can be used to transport listeners to various parts of a live stage.
Each time they are being switched to a new listening and vantage point they’ll experience the audio from a different perspective, just as they would in real-life. Additionally, they can turn around in each spot to experience adequate immersive effects. This same technology can also be applied to livestreams; as the live operator determines the walk path, a live audience enjoys the 3D binaural audio remotely from their computer or VR headset.
The potential use-cases for such a technology are virtually limitless. Not only could Zylia’s 3D audio technology offer casual listeners a more immersive and engaging way of enjoying their favorite music, but it could also provide musicians and other performers with a more impactful way of connecting with live audiences. With the COVID-19 outbreak forcing the cancellation of nearly all live in-person performances, entertainers have begun turning to popular live streaming solutions to connect with their fans remotely.
VR technology has proven especially useful in this regard, allowing users to immerse themselves in virtual worlds and connect with their favorite artists on a more personal level. ZYLIA 6 Degrees of Freedom Navigable Audio could enhance these experiences even further, offering a next-generation 6DoF audio experience just as if not more engaging than the VR experience itself.
For more information on Zylia’s game-changing 6DoF audio technology, visit zylia.co.