[This story from MIT Technology Review describes the emergence of a new form of presence-evoking photography – it’s early days at this writing but new technologies are lowering the cost and making it easier than ever to create 360° images that can be viewed in VR, and “When you see 360° imagery that truly transports you somewhere else, you want it more and more.” See the original story for 6 videos and 5 more images. –Matthew]
[Image: ALLie Camera. It uses technology originally developed for the surveillance industry and can capture images in low light.]
The 360-Degree Selfie
Inexpensive cameras that make spherical images are opening a new era in photography and changing the way people share stories.
Availability: Now
By Elizabeth Woyke
March/April 2017 issue of MIT Technology Review
Seasonal changes to vegetation fascinate Koen Hufkens. So last fall Hufkens, an ecological researcher at Harvard, devised a system to continuously broadcast images from a Massachusetts forest to a website called VirtualForest.io. And because he used a camera that creates 360° pictures, visitors can do more than just watch the feed; they can use their mouse cursor (on a computer) or finger (on a smartphone or tablet) to pan around the image in a circle or scroll up to view the forest canopy and down to see the ground. If they look at the image through a virtual-reality headset they can rotate the photo by moving their head, intensifying the illusion that they are in the woods.
Hufkens says the project will allow him to document how climate change is affecting leaf development in New England. The total cost? About $550, including $350 for the Ricoh Theta S camera that takes the photos.
We experience the world in 360 degrees, surrounded by sights and sounds. Until recently, there were two main options for shooting photos and video that captured that context: use a rig to position multiple cameras at different angles with overlapping fields of view or pay at least $10,000 for a special camera. The production process was just as cumbersome and generally took multiple days to complete. Once you shot your footage, you had to transfer the images to a computer; wrestle with complex, pricey software to fuse them into a seamless picture; and then convert the file into a format that other people could view easily.
Today, anyone can buy a decent 360° camera for less than $500, record a video within minutes, and upload it to Facebook or YouTube. Much of this amateur 360° content is blurry; some of it captures 360 degrees horizontally but not vertically; and most of it is mundane. (Watching footage of a stranger’s vacation is almost as boring in spherical view as it is in regular mode.) But the best user-generated 360° photos and videos—such as the Virtual Forest—deepen the viewer’s appreciation of a place or an event.
Journalists from the New York Times and Reuters are using $350 Samsung Gear 360 cameras to produce spherical photos and videos that document anything from hurricane damage in Haiti to a refugee camp in Gaza. One New York Times video that depicts people in Niger fleeing the militant group Boko Haram puts you in the center of a crowd receiving food from aid groups. You start by watching a man heaving sacks off a pickup truck and hearing them thud onto the ground. When you turn your head, you see the throngs that have gathered to claim the food and the makeshift carts they will use to transport it. The 360° format is so compelling that it could become a new standard for raw footage of news events—something that Twitter is trying to encourage by enabling live spherical videos in its Periscope app.
Or consider the spherical videos of medical procedures that the Los Angeles startup Giblib makes to teach students about surgery. The company films the operations by attaching a $500 360fly 4K camera, which is the size of a baseball, to surgical lights above the patient. The 360° view enables students to see not just the surgeon and surgical site, but also the way the operating room is organized and how the operating room staff interacts.
Meanwhile, inexpensive 360° cameras such as Kodak’s $450 Pixpro SP360 4K are popping up on basketball backboards, football fields, and hockey nets during practice for professional and collegiate teams. Coaches say the resulting videos help players visualize the action and prepare for games in ways that conventional sideline and end-zone videos can’t.
Component innovations
These applications are feasible because of the smartphone boom and innovations in several technologies that combine images from multiple lenses and sensors. For instance, 360° cameras require more horsepower than regular cameras and generate more heat, but that is handled by the energy-efficient chips that power smartphones. Both the 360fly and the $499 ALLie camera use Qualcomm Snapdragon processors similar to those that run Samsung’s high-end handsets.
Camera companies also benefited in recent years from smartphone vendors’ continuous quest to integrate higher-quality imaging into their gadgets. The competition forced component makers like Sony to shrink image sensors and ensure that they offered both high resolution and good performance in low light. As the huge smartphone market helped bring down component prices, 360°-camera makers found it possible to price their devices accessibly, often at less than $500. “There are sensors that now cost $1 instead of $1,000 because they’re used in smartphones, which have incredible economies of scale,” says Jeffrey Martin, the CEO of a 360°-camera startup called Sphericam. Advances in optics played a part as well. Unlike traditional cameras, which have fairly narrow fields of view, 360° cameras sport exaggerated fish-eye lenses that require special optics to align and focus images across multiple points.
Most 360° cameras lack displays and viewfinders. To compensate, camera makers developed apps that you can download to your phone to compose shots and review the resulting images. The cameras connect to the apps wirelessly, and many of them allow you to upload photos and video directly from your phone to Facebook and YouTube. In turn, those sites have made it possible over the past year for people not just to post recorded 360° content but to live-stream 360° videos as well.
Because creating 360° content requires stitching together multiple images, doing it on the fly for live streaming represents an impressive technical achievement. Computer-vision algorithms have simplified the process so that it can be done on the camera itself, which in turn allows people to live-stream video with minimal delays. (It helps that most consumer-grade cameras have only two lenses and thus one stitch line. Professional versions can have six to 24 lenses.) The ALLie camera supports fast stitching and live-streaming, as do Ricoh’s upcoming Ricoh R development kit camera and Kodak’s Orbit360 4K, which will be available later this year for $500.
Spherical cameras represented 1 percent of worldwide consumer camera shipments in 2016 and are set to reach 4 percent in 2017, according to the research firm Futuresource Consulting. The popularity of these devices will benefit the virtual-reality industry as well as camera makers. You don’t need special VR gear to view spherical videos, but YouTube says many people look at them on smartphones slipped into VR headsets, such as Google’s Cardboard and Daydream devices. And more people experimenting with 360° cameras means more content for other people to watch in VR.
In fact, John Carmack, the chief technology officer of Facebook’s Oculus VR subsidiary, has predicted that people will spend less than 50 percent of their VR time playing games. Instead, they may don VR headsets to do things like virtually attend a wedding.
Once people discover spherical videos, research suggests, they shift their viewing behavior quickly. The company Humaneyes, which is developing an $800 camera that can produce 3-D spherical images, says people need to watch only about 10 hours of 360° content before they instinctively start trying to interact with all videos. When you see 360° imagery that truly transports you somewhere else, you want it more and more.