[This story from The Creators Project (where it includes more images and the mentioned video) describes a telepresence system that lets users transition easily between first and third person views with the goal – as noted in the Rekimoto Lab headline – of allowing “more effective telepresence collaborations.” –Matthew]

[Image: A “ghost” user experiences the JackIn Space’s immersive remote point of view. Courtesy of Rekimoto Lab.]

Take a Walk in Someone Else’s Shoes with POV-Altering Headgear

Alyssa Buffenstein — March 26 2016

There’s a whole new meaning to the term “ghosting,” thanks to a new project called JackIn Space. A system of headsets allows users to trade points of view, and for a “ghost” user to virtually enter and interact with a real remote space as if they were there.

It was created by Rekimoto Lab, a Tokyo-based Human Augmentation Lab. It appears to be a culmination of a few projects the lab has developed over the years, all relating to their JackIn system.

“JackIn is the vision for augmenting human capability and human existence. It enables immersive connections between humans and other artificial entities, or between humans and other humans. JackOut enables the out-of-body experience. We are investigating Human-Machine JackIn, Human-Human JackIn, and an environment that enables seamless migration between 1st and 3rd person perspectives (JackIn Space),” reads Rekimoto Labs’ JackIn page.

The JackIn Eye, originally presented in 2014, is “a person-to-person communication framework connecting two or more people.” The setup comprises a head-mounted camera that captures first-person views, which are then shared with a second user, the “ghost,” who can virtually and remotely control their viewpoint. According to Rekimoto Labs’ research, “ghosts” can actually understand the spatial situation of the person wearing the JackIn Eye.

Then there’s the JackIn Head, a similar system that also incorporates a “body user” and a “ghost user.” The body user wears headgear equipped with an omnidirectional, motion-stabilizing camera, and the ghost user experiences the body user’s surroundings via a viewing device like an Oculus Rift.

Finally, the JackIn Space builds on all of these technologies to create a multi-user view-sharing experience that also addresses the problem faced by most telepresence systems, which only support first person views. By adding an external camera and depth sensors to the equation, not only can ghost users wearing headsets switch between as many first-person views as are connected to the system, but they can also channel into a third person view of the entire scene, which they view in an immersive “cave” environment. Inside the cave, the ghost can move freely around the remote space.

Watch a [3:41 minute] demonstration video of the JackIn Space [on Vimeo], and try to let the insane nature of the project distract you from how unfortunate the name “JackIn” really is.

Check out more of Rekimoto Lab’s work on their website.