[From Wired’s GadgetLab blog, where the post includes a 1:46 minute video]
[Image: A concept image of the Human Media Lab's TeleHuman pod. Photo: University of Queens]
3-D Video Pod Delivers 360-Degree, Holograph-Like Projections
By Alexandra Chang
May 7, 2012
Imagine walking up to a tall, cylindrical pod, and talking to a life-size, 3-D projection of a faraway friend, family member, or colleague. And not only do you see the person in 3-D, you can walk a full 360 degrees around the pod, and see your conversation partner from every angle — just as if the person was actually standing inside the cylinder.
It may sound like the science fiction of Star Trek or Star Wars, but this is exactly what a team of researchers at Queen’s University have developed in the Human Media Lab. Grandly dubbed TeleHuman, the life-size cylindrical pod allows users to conduct real-time, 3-D and 360-degree video conferences.
It’s not quite a holodeck, but it’s a whole lot closer than today’s 2-D video-chatting options like Skype or FaceTime.
“It’s all about social proximity cues. You can walk up to a person instead of having to phone them or go through an interface,” Roel Vertegaal, director of the Human Media Lab and Professor of human-computer interaction, told Wired. “Instead, there’s just this pod. It might be designated for a lover or a business partner. It’s near the water cooler, and the same thing is in his or her office or home. You walk up, the thing starts to glow and boom — beam me up, Scotty.”
There are two main components to the system: the cylinder itself, which features a special display and contains an internal projection mechanism, and a set of cameras that track and capture your position as you communicate with your conversation partner.
The Human Media Lab team started with a hollow, cylindrical display that’s made of sandblasted acrylic and mounted on top of a wooden base. A stereoscopic DepthQ projector, along with an Nvidia 3D Vision Kit, is located at the bottom of the base, and a convex mirror is installed at the top. Shooting upward, the projector beams video of your conversation partner into the mirror, which then beams the video back onto the display.
With the help of the embedded Kinect technology and distortion correction, TeleHuman projects a holograph-like image of a person onto the interior of its cylindrical display. And the structure is a little more than 6.5 feet tall, making it possible to project most people in life size.
But that’s just the display portion of the system. TeleHuman is two-way, interactive technology, so it also needs to capture image movement as well.
As you walk around the display, talking with your partner, TeleHuman tracks your position and captures your image with 10 different Microsoft Kinect sensors, each containing a built-in camera. Six Kinects line the top of the pod, tracking your position around the cylinder and capturing front-facing images. The other four Kinects are arranged in a square around you, about 8 feet away from the pod. These capture your side and rear views.
“What’s nice about the cylindrical form factor is that it’s essentially a flat display curved in one dimension, which is exactly the dimension you’re walking in,” Vertegaal said. “You always get a good view of the person as you walk 360 degrees around the display.”
While TeleHuman doesn’t use holographic technology, its results still looks like a hologram. To this extent, TeleHuman is a major design win as current holographic systems can’t match the promise of their sci-fi inspirations.
Unlike most holography-like projections, TeleHuman accurately preserves motion parallax — the changing appearance of an object as you move around it. And this is an important development in terms of social interactions. In its studies, the Human Media Lab team discovered that real-world-accurate, 360-degree views are integral to how we share ourselves with each other.
“Communication breaks down even with a subtle little thing,” Vertegaal said. “When you think about preserving human communication, it’s more about what you leave out rather than what you add. With this system, we’re trying to leave out as little as possible.”
The team specifically studied eye gaze, pointing direction and body pose. They made sure that users could experience direct eye contact while talking — a huge improvement over most current video chats, where the camera is placed higher than a person’s face in the video chat interface.
In one experiment, the Human Media Lab asked a yoga instructor to show off various poses to a participant in another room, using both TeleHuman pods and 2-D displays. Participants were asked to replicate the poses, and the yoga instructor then judged how well the participant performed. Results showed that users did much better when they could walk around a display and see a yoga pose in 3-D.
TeleHuman can also accurately depict where people are pointing and gesturing, another benefit for real-world-accurate interactions. “When you turn over to show something or point at something at the screen, people see you turn around, and it’s actually relative to the screen that’s there in the room,” Vertegaal said. “Being able to merge the virtual and the physical is really, really important.”
This has a number of real-world applications. Doctors, for example, can use the device to conduct remote examinations, seeing a patient’s life-size body at different angles and views. The Human Media Lab’s paper on TeleHuman also mentions that the technology would have useful applications in sports instruction — for example, teaching golfers how to improve their swings. Naturally, it could also be used for businesses and classrooms, and even for digital theater performances.
TeleHuman isn’t without its problems, though. The image is a bit too dim and is somewhat low-res, since it’s projected onto such a large display. And in order to see the projection in 3-D, users still need to wear stereoscopic glasses.
“Glasses are a bit of a problem, because you lose eye gaze,” Vertegaal said. “You can either use [TeleHuman] with glasses and get a fully stereoscopic view, or without glasses, but you still get motion parallax. With glasses, the projection totally appears holographic. You get the sense that there’s this person hovering in space.”
Vertegaal says his team hopes to fix these issues and further develop the TeleHuman pod. For instance, they want to have multiple pods video-conferencing simultaneously. Roaming pods, where the devices can move around a space, is another option. Vertegaal also says that he would like to use the Kinect cameras to show the environment that a person is in — currently, the cameras block out background images.
How much would one of these pods cost a consumer? Vertegaal estimates that TeleHuman could be mass produced for around $5,000 per device. A single Kinect sensor costs about $150, and the device uses 10 sensors. So maybe you won’t see the TeleHuman in many offices or homes soon, but hey, we’re just one step closer to making science fiction a reality.