Peek Behind the Sneaks: Lose the Photobombers and Get the Perfect Shot with Monument Mode

Ugh! It happened again. You had the perfect picture! But then a group of tourists step into the shot. Or a car drives by. Or your annoying cousin photobombs you. It’s hard to get the perfect shot.

But don’t worry. Ashutosh Sharma, a senior computer scientist at Adobe, is on a mission to make it easy. He’s created an app called Monument Mode that automatically detects and removes unwanted objects or people that happen to wander into your shot.

“I had the idea while I was traveling,” explains Ashutosh, “I like to take pictures of famous locations and viewpoints when I’m on vacation, but I noticed that when you’re at a popular spot, there’s always a lot of people around.”

Ashutosh demonstrated Monument Mode as part of the sneaks presentations at Adobe MAX 2015, and proved that getting a postcard-perfect shot can be as easy as point and shoot. “Everything happens live right on the user’s screen,” says Ashutosh. “You just hold your camera phone up to frame your photo, and any people or objects that are moving around disappear from the scene. When you’re happy with the composition you simply click to take the photo.”

Monument Mode works by looking through frames captured during the live preview of your camera phone — those moments before you’ve clicked the button to take the shot. By analyzing pixel changes in the scene over time, the app can detect what parts of the scene are dominant, and which parts are transient, then remove temporary objects from view. The process takes just a few seconds, to a few minutes, depending on how long the photobombers are in the scene.

It sounds simple, but there’s some serious software engineering happening behind the scenes. “The biggest challenge was trying to do this on a mobile device, and do it live. We need to analyze a frame from live view every few milliseconds, and that requires a lot of computational power. Fortunately, we were able to combine multiple optimizations within the app to make the most of the GPU. That speed is key for the user to get instant feedback on the screen and achieve the result they really want.”

Monument Mode is currently a concept technology and not yet available as a commercial app, but based on the reception it received at MAX, Ashutosh hopes to make it consumer-ready in the future. He sees other possible applications for it as well. “Since it works by analyzing dominant pixel values of a scene, we could potentially use it to remove noise (i.e., excessive graininess) from a photo after it’s been taken in a low-light situation, or perhaps even remove noise generated by the camera sensor itself,” he muses.

To learn more, check out coverage on Monument Mode in Engadget.

This story is part of a series that will give you a closer look at the people and technology that were showcased as part of Adobe Sneaks. Watch other Sneaks and videos here.