Peek Behind the Sneaks: Your Photography in the Next Dimension

For more than a century, we’ve thought of photography in two dimensions — flat images that visually represent a three-dimensional world on paper or screen. Even in the world of digital imaging, working with three dimensional models and content has been more of a playground for professional game developers or high-end special effects engineers, than for the average designer.

Today, however, digital imaging is gaining an extra dimension. Suddenly, devices that demand 3D content — like 3D printers or virtual reality headsets like the Oculus Rift — are within reach of the average consumer. We can print actual objects instead of ink on paper, and we can experience content in an immersive 360° environment, rather than viewing it on a screen.

Unfortunately, personal content for this new devices is still a rarity, as 3D cameras have yet to go mainstream. But Adobe recently removed this complexity by introducing products such as Fuse CC that enables anyone to easily create highly customizable 3D human characters in minutes without any prior 3D experience.

Taking Fuse CC a step forward, 3D Portrait was recently showcased as part of the Adobe MAX 2015 sneaks presentation. This technology lets anyone with a camera create realistic 3D models from nothing more than a portrait-style photograph.

“With 3D Portrait, you can rapidly [in a matter of minutes] make something that would normally take hours and hours — and trained expertise — to create in a 3D modeling program,” explains Nathan Carr, principal scientist and research manager at Adobe. “Our goal is to make it possible to create personal 3D content with just a few clicks.”

3D Portrait works by combining automatic face recognition technology and a large database of human faces that were captured in full 3D — including a range of expressions, ethnicity, gender and age — to quickly generate an accurate 3D model from a photo. The user simply needs to provide a little bit of input about the location of the hair and body. Behind the scenes, the lighting information of the photograph is analyzed, and then combined with a natural hair simulation as well as the colors and personalized facial characteristics from the photograph into a realistic, 3D-printer-ready model.

“Capturing more of the personal identity of the person in the photo is challenging,” explains Nathan. “We want to get all of the wrinkles, the creases of the eye shape and other local features. To do this, we estimate the lighting behind the scenes. And the shading helps us to understand where the wrinkles and folds are, and gives us more information about the unique geometry that comprises the face. So we’re able to add that back into these template models to get a more accurate reconstruction.”

Today, the technology in 3D Portrait creates more of a bas-relief for printing, rather than a full 360° model — but the potential is there for that and more.

“Any kind of editing applications that use 3D models can benefit,” explains Kalyan Sunkavalli, another Adobe research scientist on the project. “Originally this started as a project to make better content for 3D printing, but the quality of the reconstruction we are getting is so good, that there are other applications for it.”

“One of the things we’ve explored is relighting — adding light sources to a photograph, or moving light sources around and still casting accurate shadows around the features of the face. You can’t do that unless you have 3D geometry.”

Nathan adds, “We’re also working towards being able to do complete 3D portraits, where you can capture yourself, or someone else, from all sides and all angles, and completely reconstruct yourself as a 3D avatar which can go inside the computer, or be 3D printed.”

It’s a compelling vision — 3D content for the masses. Although 3D Portrait is still a research technology, it has the potential to allow anyone to take their personal photography into the next dimension.

This story is part of a series that will give you a closer look at the people and technology that were showcased as part of Adobe Sneaks. Watch other Sneaks and videos here.