Light and Magic

The magic hour — or the time right after sunrise and before sunset — is both boon and bane of cinematographers and photographers seeking perfect quality of light. Depending on the time of day, photographers can capture unique moods with hard or soft light, and explore different hues from red to blue. But these moments are fleeting. The light for that perfect shot may last just a few minutes and artists have been known to wait hours, or even days, to capture the perfect shot.

Imagine the time (and money!) that could be saved if every minute of the day could be magic. Thanks to Sylvain Paris, a principal research scientist with Adobe Research, it’s becoming a real possibility.

Sylvain partnered with researchers at MIT and Adobe to develop software technology capable of analyzing the light qualities in a photograph and applying a real-time simulation of light changes throughout the day.

Shown as part of the Sneaks program at Adobe Max 2014, Sylvain’s Time of Day demo wowed the audience with accurate and stunning light effects as he simulated time changes to a photo — turning day shots into night shots across the elusive magic hour.

Sylvain credits his interest in amateur photography, and the time he spent reading photo blogs, as his inspiration for the demo, “There are common themes that emerge,” he says, “you need the right lighting, you need to be at the right place at the right time, such as right before sunset.”

As Sylvain applied the techniques he was learning to his own photography, he found that he could imagine what his shots would look like if taken at different times of the day, “I realized as I looked at a scene, even if it’s not sunset, I could imagine what it would look like … and if I could do it, I could probably teach an algorithm to do it too.”

Of course, it takes more than inspiration to make an algorithm that works for a practical application. For that, Sylvain credits Adobe’s culture of collaboration.

“There are a lot of product managers who are open minded to receiving ideas outside the product team,” he reveals, “I’m not part of Adobe’s product teams in terms of the organizational structure, but it is very easy to go and knock on their door, and say, ‘Hey I have this new thing and I think it could be useful for you.’… And people always have an open door policy and an open mind to the situation. Sometimes they like it. Sometimes they don’t. Sometimes they have good suggestions on how to make it better or more useful. There’s this very good contact between research and product, and I think that’s a unique strength of Adobe.”

Sylvain says he also benefits from Adobe’s collaboration with research universities like MIT. After earning his Ph.D from the The French Institute for Research in Computer Science and Automation, he was a post-doctoral associate at MIT. The relationships he established there were key to solving some of his biggest technical challenges. “I’m lucky to have very good people to work with me on my research,” he muses.

In fact, it was one of his collaborators at MIT who first had the idea of using time-lapse video as the basis for teaching the algorithm how color changes throughout the day.

“The sky does not change color in the same way as a building, or a forest, or the sea, “ he explains, “The different elements of a scene all have different ways to react to the change of light throughout the day. … To do it precisely enough so that it looks good was really hard.

“It turns out time-lapse videos have so much information that we can extract and analyze and use.”

Sylvain hopes to see his technology made widely available to photographers in the future, and is excited to apply time-lapse video analysis to new problems, noting, “A nice by-product [of our technology] is that we can accurately predict the time of day that a picture was taken, so in a situation where you don’t have the actual time stamp, this could be a useful piece of information used to sort images or organize them across a collection.”