[The augmented reality experience “Pilgrim” described in this story from Adweek is a reminder of how powerful audio, including audio without video, can be in evoking presence. See the original story for more pictures, and the Voices of VR podcast for a 26 minute interview with co-creator/director Lauren Hutchinson. –Matthew]

[Image: The Camino de Santiago is now the setting for a new AR audio podcast. Credit: Getty Images.]

This Augmented Reality Audio Podcast Brings the Camino De Santiago to Life Everywhere

Pilgrim was made for Bose Frames

By Marty Swant
April 17, 2019

For centuries, the Camino de Santiago in Spain—also known in English as “The Way of Saint James”—has been a destination for Christian pilgrims from all walks of life. Some come for the scenic hiking through rural villages across France and Spain. Others come for religious reasons as they make their way toward the cathedral of Santiago de Compostela in the coastal Spanish city of Galacia.

Yet, for many, it’s also been a pilgrimage of another type—a path for processing some of life’s biggest confusions and disappointments—while sharing stories with fellow travelers. But what if those of us who can’t make the journey could still walk with the pilgrims in another way—regardless of where in the world we are?

Combining radio-level quality journalism with cutting-edge technology, a small team of developers and creatives—along with the technology and content studio Tomorrow Never Knows—has created an augmented reality audio podcast. The experience, “Pilgrim,” lets listeners anywhere around the world digitally walk alongside hikers while hearing their stories—thanks to augmented reality-enabled sound technology that’s part of a new wave of AR storytelling that goes beyond what the eye can see.

“When you walk with someone for a while, the intimacy with that person builds up,” said Lauren Hutchinson, who created Pilgrim with producers Gabo Arora and Tom Lofthouse, executive producer Nathan Brown, and co-director Saschka Unseld. “And you meet people and then you say goodbye to them and you don’t know if you’ll see them again. It’s a little bit like life.”

A radio journalist with the BBC, Hutchinson first gathered stories for “Pilgrim” while walking part of the path in 2015. The initial idea was to collect them for a radio documentary. Walking with them for varied lengths of time allowed that story to come out.

The experience—which debuted at the Amsterdam International Documentary Film Festival late last year—uses algorithmic storytelling that’s both flexible and adaptive to length depending on each listener’s own location, walking speed and interest. With Unity’s developer software, Pilgrim’s creators were able to include something Hutchinson described as a “latched, unlatched state.” When a listener stands still, they can hear footsteps and voices seem to walk toward them from behind. If the listener wants to hear more, they can begin walking as well—creating the illusion of “latching” a hiker to walk alongside them as the story unfolds. If they don’t want to hear anymore, the listener can stop, allowing the hiker to “unlatch” and continue on with their journey until another hiker nears.

The software also works in another way: the longer a person walks with a pilgrim, the more that pilgrim opens up about their own life.

“It’s this kind of stream of consciousness of what comes into their mind when they’re walking,” she said.

While the stories are powerful on their own without technology, they become visceral when listening to them while wearing a pair of AR-enabled headphones like the Bose Frames sunglasses, which Pilgrim was developed specifically for using a beta version late last year. By using a mobile app to connect the sunglasses with a smartphone’s accelerometer, gyroscope and compass, Pilgrim’s creators digitally mapped out a path for someone to walk while hearing the stories.

The one-hour experience consists of nearly a dozen stories from the trail, which were trimmed from two-hour interviews to five-minute clips. The interviews also weren’t structured, with open-ended questions allowing for pilgrims to process as they walked and talked. One such story: an 18-year-old woman whose mother had passed away, and she was reading her mother’s diaries as she walked. Another features someone who recently found out she would soon become blind.

“It wasn’t obvious to themselves when they were walking,” Hutchinson said of the pilgrims. “I found out when people get to the end, they realize they’re walking for a different reason than when they set out.”

The concept for Pilgrim was inspired by how people often think better when they are walking. And that’s not simply anecdotal. In fact, a 2014 study by Stanford University researchers found that a majority of a study’s participants scored higher on a creativity test while walking, suggesting a mind-body connection to walking and thinking.

“At the end of the day, it’s using our movement as UX design, because that’s what it is,” Unseld said. “When do we register someone as walking? When do we register someone as stopping?”

Pilgrim’s creators are also working on a mobile app version based on the length of time spent walking, which will help infinitely broaden the number of places where the AR podcast can be experienced instead of having it limited to mapped out routes. Hutchinson said those who have already tried it have felt a sense as they’re walking that maybe the strangers in their own town might have their own stories to tell.

“As people were walking along, they were kind of walking by people and they were thinking you have this story, and you have this story,” Hutchinson said. “You’re just not telling me right now.”

I got the chance to test the technology a few months ago using a beta version set along a path around Manhattan’s West Village. And because Bose has not yet released a version with prescription lenses, I had to take off my regular glasses—forcing me to rely even more on what I heard rather than what I saw. And because the speakers in the sunglasses purposefully don’t block out external sounds, they brought stories from trails far away and placed them right on a New York sidewalk.

There are also challenges that differentiate depending on location. For example, pilgrims passing through Spain’s smallest towns won’t have to deal with block-by-block traffic lights like in New York City. That creates a different dilemma for the listener: If someone’s at a crosswalk, do they stop and risk letting the hiker move on? Or do they risk jay-walking in order to continue to keep the story from continuing without them?

“It’s like creating a world where you can flow through it yourself,” Hutchinson said. “But you’re not having to too consciously think: ‘Do I want to go this way or do I want to go that way?’”

But maybe that mirrors our own real lives, and how we sometimes must choose between the stories we want to hear and those we want to ignore.