[This is a long but easy-to-read and extremely interesting Fast Company story by technology writer Mike Elgan about the return of Spiritualism, in which modern presence-evoking technologies are used to bring us closer to people who have died. Many of the examples have been the topic of previous ISPR Presence News posts but Elgan provides a valuable framework for understanding the larger phenomenon. See the original story for four videos. –Matthew]
[Image: Source video: MART PRODUCTION/Pexels]
How holograms, deepfakes, and AR are raising the dead
As tech creates digital replicas of the deceased, a deep human need could be fulfilled—and a long-ago opportunity for scammers could make a comeback.
By Mike Elgan
May 5, 2021
Considering his final frontier, 90-year-old Star Trek star William Shatner recently decided to boldly go into a Los Angeles studio and turn himself into a ghost.
Shatner spent five days recording a StoryFile, a type of interactive video created by a company also called StoryFile. Portions of the recording, which were captured by 3D cameras, will be “tagged” using StoryFile’s proprietary system. Later, Shatner’s ghost will be beamed to his family members, to fans via the internet, and possibly to museums and entertainment venues. People will be able to ask Shatner’s ghost questions. StoryFile’s system will “play” the answers, creating the illusion that William Shatner lives, even long after he passes on.
Welcome to the new spiritualism.
A hundred years ago, the idle rich of Europe and America indulged a fascination with the great beyond. A quasi-religious movement called Spiritualism, which began in the 1830s and rose in popularity during times of great trauma, such as during the U.S.’s Civil War. The movement peaked in the years between 1918 and the early 1920s, when Spiritualist ideas spilled over into mainstream popular culture.
Spiritualists believe in the possibility of communicating with the dead through mediums (people who can allow the dead to speak through them), via seances, “automatic writing” (Ouija-board-style letter picking), and the conjuring of ghosts.
Spiritualists believe that the dead are more evolved than the living and have learned esoteric knowledge in death. They turned to mediums seeking not only communion with deceased loved ones, but also information and guidance.
As World War I was drawing to a close, the pandemic of 1918, the so-called Spanish flu, was just getting started. These twin events drove the Spiritualism craze to new heights. People were grieving and wanted answers, which created a market for Spiritualism. That was exploited by hoaxers to trick people using the technologies of the day, including photography and phonographs.
A basic 1920s-era seance involved several people sitting around a table in a dark room, perhaps holding objects belonging to family members who had recently died. A medium would be tasked with coaxing spirits and ghosts into making their presence known by knocking, raising, or turning the table, or by actually appearing as a ghostly apparition.
Some mediums weren’t frauds, but true believers. For example, Sherlock Holmes’s creator, Arthur Conan Doyle, believed that his own wife, Jean, was a gifted medium.
Interest in Spiritualism led to lucrative scams, which in turn drove interest in Spiritualism, as the gullible saw false but believable “evidence” with their own eyes and ears. Photography, movie projection, sound recording, and other technologies had special power because they were new and unfamiliar to lay audiences at the time. One technology that will come back to haunt us in this very article was the use of gauze coated with reflective or phosphorescent paint which, when waved around by an unseen assistant, made for a convincing ghost.
The rich and famous went nuts for conjuring the dead 100 years ago. And now, they’re at it again.
THE RISE OF CELEBRITY ‘HOLOGRAMS’
Captain Kirk won’t be the first celebrity to get digitally cloned. Not by a long shot.
Musical superstar, sneaker mogul, and former presidential candidate Kanye West gave his future ex-wife Kim Kardashian the birthday gift of a “hologram” of her dead father, Robert Kardashian, who died in 2003. The elaborate and expensive simulation said the kinds of things a ghost would say, such as that he watches over Kim, her siblings, and children, and that sometimes he leaves hints that he’s still around. (He also said the kinds of things that Kanye West would say, like what a towering genius Kanye West is.)
The Kardashian stunt came years after the appearance of California rapper Tupac Shakur at Coachella some 15 years after his death. Tupac blew drug-addled minds at the concert by performing a duet with living rapper Snoop Dogg. The ghost of Tupac was organized by rapper and producer Dr. Dre and largely constructed by the Hollywood digital effects company Digital Domain.
In recent years, many celebrities have been raised from the dead in the form of ghostly “holograms,” including Whitney Houston, Marilyn Monroe, Ronald Reagan, Vince Lombardi, Michael Jackson, Roy Orbison, Frank Zappa, Elvis Presley, Amy Winehouse, Glenn Gould, Maria Callas, Buddy Holly, and Ronnie James Dio.
Everybody calls these artifacts “holograms,” but that’s a misnomer. By definition, a hologram is a 3D projection that can be viewed at different angles. These recreations are 2D projections that require the audience to be sitting in the right location.
Creators of these celebrity “hologram” performances hire an actor or impersonator who looks and moves like the subject of the hologram. Once the footage is captured, the face is replaced by a deepfake video that looks like the subject, and visual effects specialists touch up the combined video to remove some of the glitches and imperfections generated by the current state of the art in deepfake technology.
Some “holograms” use a technique called Pepper’s ghost. Pioneered by English scientist John Henry Pepper, who first demonstrated the technique in 1862, The Pepper’s ghost illusion takes advantage of the fact that light can pass through glass, but also reflect off of it. The idea is that you place glass at a 45-degree angle, then shine the hologram from below. The audience can see through the glass, but they can also see the hologram’s reflection, which appears in the same visual space as what’s behind the glass.
This is the effect that Disney has long used for Disneyland’s Haunted Mansion attraction. Ghosts appear to be dancing in a room. The room is visible through the glass. The “ghosts” are mechanized and dancing below the visitors, but reflected in the angled glass that sits between the guests and the room.
The state of the art in hologram projection uses a product called Holonet from a company called Kaleida. Holonet is an invisible gauze that reflects light more brightly than glass. It’s the technology used in the Kardashian hologram, as well as the Tupac Shakur event at Coachella in 2012. Gauze ghosts haunted seances in the 1920s; now they continue to haunt concert halls in the 2020s.
In the case of Shatner’s “ghost,” StoryFile CEO Heather Smith told me that part of the company’s core mission is to produce faithful, unedited, unmodified video of their subjects—there is no deepfake technology involved, no fake anything. The “AI” used is for retrieval of the right video segments based on a user’s question to the subject in the video. While StoryFile is working hard to make its technology available to everyone using low-cost, low-tech media, the high-end version used in the Shatner recording provides future creators with all they need to make actual 3D holograms.
Another difference between Shatner’s hologram and other recreated celebs is that he consented, whereas most other subjects did not. There are ethical implications to profiting without permission from the likeness of the dead, something that authors like Elaine Kasket, author of All the Ghosts in the Machine, have explored deeply.
So the celebrity “holograms” you’ve heard about aren’t holograms. And they’re also not ghosts.
WHAT IS A GHOST, ANYWAY?
The ancient idea of a ghost is that our physical bodies are meat puppets animated by an invisible, nonphysical “mind” or “soul” or “spirit.” When our bodies die, the idea says, our souls do, or can, persist.
I would argue that a “ghost” is not a soul in the abstract, but the perception or the experience of the presence of one of these “spirits.” It’s not a “ghost” unless someone experiences or perceives it.
And it’s clear why people perceive ghosts. Our brains are hardwired to detect human faces, human bodies, and human voices. The experience of interacting with other people—especially our loved ones—is burned into our memories. Ghost perception is no doubt related to the psychological phenomenon of pareidolia—the tendency to see a bunny rabbit in the clouds, a man on the moon, or the face of Jesus in our toast.
That’s why seances were so convincing. Show a grieving human a glowing wad of gauze, and voilà, grandma has risen!
Ghost sightings require a trigger. The seance hoaxes of the 1920s often involved photography. Fraudulent medium Eva Carrière used fake “ectoplasm” made out of chewed-up paper, and photographs that people believed were ghosts. In one famous photo, Carrière is exposed materializing the “ghost” of King Ferdinand of Bulgaria, which is obviously a cardboard cutout.
We’re on the cusp of a new world of technology that will be capable of inducing not only triggers, but ultra high-resolution, interactive, 3D versions of the dead that could bring back a new age of Spiritualism.
THE TECHNOLOGY BEHIND THE PERFECT GHOST
Apple is reportedly working on two augmented reality products. The first is likely to be a set of goggles that will mostly do VR. The real world and the virtual world will be blended together, then displayed on two very high-resolution screens built into the goggles.
In the second product, the real world will be seen through clear glass, and virtual objects will appear like holograms in the real world. These will be designed for all-day, everyday wear.
Dozens of major companies may eventually make such glasses, but Apple will mainstream the technology. It may only be a matter of time before normal reality is always augmented for many people.
Unlike the Tupac and Kardashian projections, consumer AR glasses will enable simulated 3D holograms—different people viewing the same virtual object will see it from their own perspective.
Deepfake audio is already nearly perfect. And Deepfake video is getting better every year.
AI in general—and generative adversarial network (GAN) technology, the machine learning frameworks behind all deepfakes in particular—combined with AR will result in the ability to reproduce in 3D and in the real world the face, body language, mannerisms, voice, and patterns of speech of any person that can persist after death. They’ll permit the creation of a dead ringer, if you will, of a real person.
These digital simulations will be made interactive, so you can hold conversations with them. The opinions, beliefs, and knowledge of actual humans may also be uploaded into the mix, augmented with real-world knowledge.
HERE COMES THE DIGITAL GHOST INDUSTRIAL COMPLEX
I believe that AI-constructed digital humans experienced through AR glasses will become the primary way the deceased are memorialized and remembered. Shrines will be constructed in location-based mixed reality. At minimum, they’ll feature hologram-like videos of the deceased, standing there waving as ghostly apparitions and interactively conversing with surviving family members.
In fact, this basic concept is already here. A Japanese tombstone-engraving company called Ryoshin Sekizai offers an AR app called Suma Tomb that creates an AR “hologram” of loved ones that can be placed anywhere—a gravesite, in the home, or at the place of death. The ghosts are visible only through a smartphone at present. But as with all smartphone AR, it’s just a precursor to the AR glasses revolution. As with Kim Kardashian’s dad, the AR ghosts tell loved ones that “we’re always watching over you.”
A South Florida company called Artistry in Motion Holographics offers the recording and performance of “holograms” at funerals, enabling people to basically eulogize themselves postmortem.
The latest ghost technology that animates the dead comes from the online genealogy company MyHeritage. Its Deep Nostalgia service uses AI to animate uploaded photos, so that grandma is looking around, blinking, and looking somewhat alive. It’s flawed and creepy, but also moving.
In 2015, a Belarusian tech entrepreneur named Roman Mazurenko was killed in a Moscow car crash. And so his best friend, Eugenia Kuyda, recreated Mazurenko as an AI chatbot. She also created a chatbot product called Replika, which asks users questions in order to learn how to recreate the speech patterns of that user to simulate them in chat.
These and other similar projects and products suggest a potential market for interactive digital versions of the people we have lost. Ghosts, in other words.
A RETURN TO THE SPIRITUALISM OF THE 1920S
It’s reasonable to expect the rise of services that offer Siri-like personal assistants, but represented by digital versions of deceased loved ones. These digital ghosts will serve the same purpose as the seance of the 1920s: to reconnect with the dearly departed, and also get answers and guidance from more evolved beings.
There is also no doubt that AI and AR will be used not only to simulate the dead, but also for hoaxes to convince the gullible that ghosts are real and are speaking to us. Fraudsters and mediums will offer to channel lost loved ones, and will appear to do so through deepfake photography, audio, and video.
Nobody’s talking about it. But I think we’re on the verge of a new era in technology-enabled Spiritualism. Technology will not only raise the dead, but also countless ethical questions about keeping the deceased around as digital ghosts to haunt the living.