Preview Technology Gives Your Voice the Power of a Creative Director

Creative professionals can spend a lot of time organizing visual assets or trying to find the right photo for a campaign. But what if finding the perfect image were as easy as simply asking for it out loud?

Preview technology from Adobe — called #PhonicFilters — promises to make searching image libraries as easy as talking into your mic. It’s like having your own personal curatorial librarian. This technology and others were presented at Adobe Summit as part of our Sneaks session, where our incredible Adobe engineers take the stage to share short demos of potential future tech at Adobe.

“Most people just search for images using keywords in a text-based query,” says Brett Butterfield, director of software development for Sensei agents at Adobe. “We actually have some amazing search capabilities built into products like Adobe Stock and Lightroom that allow you to search by categories like depth of field, or placement of subject in a frame, but you have to know how to navigate all the menus to use them. We wanted to see if we could make those capabilities accessible with an interface that was more like a natural conversation.”

During his Sneak presentation of #PhonicFilters, Brett showed how a natural language interface can be paired with Adobe Sensei — our artificial intelligence (AI) and machine-learning technology — to categorize or find images in powerful new ways.

“Show me images organized by category,” Brett requested during the demo, and instantly his entire photo library was grouped into browsable folders labeled by categories like art, business, nature, politics, sports, and more. It’s possible because Adobe Sensei analyzes each photo as it is added to the library, automatically generating a diverse set of metadata and set of associations for each photo.

Brett onstage at Adobe Summit, presenting the sneak technology #PhonicFilters.

“It’s not just keywords about the subject of the photos like dog or sunset. We can leverage machine learning to uncover broader concepts like color palette, depth of field, color vibrancy, and even broader concepts like authenticity,” Brett said.

The result is a powerful new way to explore and riff off of hundreds of millions of images in a service like Adobe Stock. With three simple voice commands and a click, he was able to search through more than 60 million images to find several that he liked. “Find pictures of wine,” he said. “Add people toasting,” he said, and then clicked on a promising photo. “Find similar images,” he added, and was presented with dozens of on-target options.

“With just about a month of initial development, we were able to create something pretty magical,” Brett said. How does it work? The Sneak uses Rasa, an open source chatbot, on the back end that interprets natural voice commands and generates JSON to add to, or delete triggers from, the search. “Then, we’re able to use AI and Machine learning on the back end to understand intent and apply those filters automatically,” he said.

The result is a natural voice interface that allows the user to start off with a broad, base query, and then build upon it or narrow the scope. “The VP in charge of Adobe Sensei liked the demo so much that he changed my job,” Brett said with a laugh. “I was always focused on finding innovative uses for Sensei, but now I get to be much more focused on finding solutions for voice-controlled applications and creative agents.”

https://www.youtube.com/embed/d_I27Wx-CsI