‘Eight’ is a breakthrough for virtual reality in classical music

[This story from The New York Times about a new opera experienced through virtual reality contains some useful insights about how to effectively evoke presence. See the original story for a second image and two videos. –Matthew]

[Image: A still from “Eight,” a so-called mixed reality work by the Dutch composer Michel van der Aa. Credit: doubleA Foundation]

‘Eight’ Is a Breakthrough for Virtual Reality in Classical Music

In his latest work, the composer Michel van der Aa has fully integrated emerging technology with musical and theatrical ideas.

By Joshua Barone
July 5, 2019

AIX-EN-PROVENCE, France — The world of classical music is still figuring out what to do with virtual reality, an emerging technology that can be buggy and burdensome. So far, orchestras and opera houses have come up with little more than 360-degree videos that show concerts from the perspective of the conductor’s podium.

But there is a breakthrough in Michel van der Aa’s “Eight,” a so-called mixed reality work that had its premiere last month in Amsterdam and is currently on view here at the Aix Festival, through July 30, at Château La Coste.

“Eight” is an opera taken in through a virtual-reality headset. In about 15 minutes, with a genre-bending score that verges on pop (the singer-songwriter Kate Miller-Heidke is featured), it tells a poetic story of an old woman looking back on her life.

She beckons you to follow her down a corridor with white walls; once you start walking, you almost never stop, even as the hallway seems to open up into a mountainside and the full cosmos. (In reality, this all takes place in a space no larger than a bedroom.) As viewer, you are very much part of the piece, your hands visible as they reach for a railing and, at one point, lift a red tablecloth to crawl underneath.

There’s nothing gimmicky about “Eight,” in part because the medium is fully integrated with the concept of the piece — a rarity in classical music, where VR is often applied to existing repertoire, modern technology superimposed on Mahler. (“Eight” was built from the ground up in collaboration with the firm The Virtual Dutch Men.)

“I saw quite a few earlier VR projects where usually you would sit on a chair, and you could look around a little,” Mr. van der Aa said in an interview. “It’s cool for a few minutes, but I want more. We need to step in and offer an extra layer to these experiences for them to make sense. Otherwise I’d much rather go to a concert hall.”

Mr. van der Aa’s music has long been tech-forward. His 2002 chamber opera “One” featured the soprano Barbara Hannigan singing alongside virtual versions of herself. “Blank Out,” which is also at the Aix-en-Provence Festival this month, incorporates 3-D film for an effect of blurring reality. He’s at work on a new opera, that, he said, will involve motion-capture technology rendered live.

“We’re surrounded by electronics and multimedia technology,” he said. “So it would feel artificial to not allow that on an opera stage.”

“It’s a reference for an audience now, and it’s an extra storytelling tool,” he added in the interview. Here are edited excerpts from the conversation.

Can you explain how technology heightens an idea like yours for “Eight”?

For me, adding technology is like adding a tuba in an orchestra. It’s a new color; it’s a new possibility. I have an idea, then I think: What do I need to get this across to an audience? Sometimes I need a string quartet. Sometimes I need a 3-D video and electronics. The technology always is in service of the idea.

This piece is short. Did the medium have anything to do with that?

We did a lot of tests with audiences. We found that if it goes over 15 minutes, it has quite an impact on your brain. There’s a limit to what’s still comfortable. After 15 minutes, some people became nauseated. Shorter would have less impact, and longer would be a strain. So this felt like a proper arc.

Is there a tension between the music and how overwhelming VR can be?

It’s so overpowering. It’s such a big part of your intake, it’s hard to get very detailed and complex in the music. Early on, I decided the music needs to be very focused. It’s one of the lines of the storytelling, so I can’t write a 16-layer, dense orchestra piece. I opted for a more direct, indie-pop type.

Since VR is still emerging, what problems did you encounter?

The biggest challenge was the characters. First, we tried to make them as realistic as possible. I found quite early on that that didn’t work at all, because the more perfect you make them, the more uncanny-valley they get, and the more unrealistic they become.

We decided to let go of the realism and show the digital imperfections more. So we created a glitch that moves through the body that almost makes them transparent at different parts. They dissolve into their surroundings. It became also conceptually much more appropriate for the piece.

What other technology do you have your eye on?

I’m thinking a lot about how we could lower the threshold for music theater and opera and present it through a web browser in an interactive way. I took a very small step there with my song cycle “The Book of Sand.” I wonder what we could do if we could present a new art form digitally through the internet that would make sense and wouldn’t try to mimic a live experience. We stream everything these days. There’s something to be said about an art form that’s there all the time and could be accessed through Chrome and not at the Met Opera.

How do you feel about the way technology ages in relation to your art?

The way we’re doing this, the software is completely hardware-independent. So we have a more fluid system. It will look better; we’ll do more upgrades. But we stop at some point. In 2030, everybody will be laughing about this. Then again, for me it’s not about technology. It’s about the story that people experience. If you look at old video artworks, on VHS, they still move me intensely. It’s not about the vehicle, but about the content.

Eight:

Through July 30 at Château La Coste, Le Puy-Sainte-Réparade, France; festival-aix.com.

This entry was posted in Presence in the News. Bookmark the permalink. Trackbacks are closed, but you can post a comment.

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*
*

  • Find Researchers

    Use the links below to find researchers listed alphabetically by the first letter of their last name.

    A | B | C | D | E | F| G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z