Bullet Time in Real Time

The Wachowski Brothers popularized the camera array effect known as ‘bullet time’ in their 1999 film The Matrix. Creating the visual effect — where time appears to freeze as the audience’s point of view moves around the subject in real time — has traditionally required an array of expensive, high speed cameras all programmed to capture the scene at exactly the right moment. It’s a complexity that has kept the effect out of the hands of the average videographer… until now.

Jue Wang, a principal scientist from Adobe Research, and students from Columbia University have developed a way to create the bullet time effect with a few friends and their smart phones. “Initially we built a prototype based on the idea of taking panoramas, where each cellphone takes a different view of the picture at the same time,” Jue explains. “The idea was to capture a panorama of a busy scene without any ghosting or odd artifacts. But we realized we could leverage the same technology to achieve a bullet time effect with smart phones.”

The app, called CamSwarm, is a prototype concept that currently works on iOS devices. It works by creating an ad hoc camera array out of multiple, synchronized smart phone cameras. “The most innovative aspect of CamSwarm is the real-time guide that helps people form a good camera array,” he says. “You need each camera to photograph the subject from a specific location. We give you this nice visualization in the interface so you can see real-time where the best angle for each camera should be, and everyone can quickly adjust his or her camera.”

Once the footage is captured, the app sends video data from each phone to a server in the cloud where the different shots are stitched together into a single sequence. It works with as few as two cameras, but Jue notes that they’ve tested it with up to 10. “A larger number of devices will cover a bigger range of view angles, making the effect more impressive,” he explains.

Although the technical capabilities of CamSwarm are uniquely innovative, Jue is most interested in unlocking new social experiences in photography. “The main idea is to transform the photography experience from a solitary task to a social experience,” he says. “To do that, we need a platform that allows the phones to talk to each other and share data with each other.”https://blog.adobe.com/media_f78fb6e8d1c019befd17b4d0ca3eabff728c2aa3.gif

Although CamSwarm is only a prototype technology today, Jue hopes to make it — or something like it — more broadly available in the future, as he continues to explore the social aspects of digital photography and imaging technology.

Adobe is a great place to do that, he notes. “We have the freedom to do advanced research and explore new ideas, but at the same time apply our research to real products. I’m not just interested in publishing papers. I want my work to be truly useful to consumers and customers. That’s what’s really rewarding.”

Check out CamSwarm media coverage on NBC News, PetaPixel, and Daily Mail.