Peek Behind the Sneaks: Like Shazam for Type

You can stream your TV, swipe right for a date or tap an app to catch a ride. So why are designers still selecting typefaces like it’s 1985? That’s the question that inspired Hailin Jin, a principal scientist in Adobe Research, to create DeepFont — a new technology that can automatically recognize typefaces from a photo as well as suggest similar type families.

“My research background is in image recognition and artificial intelligence techniques like machine learning and deep learning,” Hailin explains. “I wondered — can we apply the power of machine learning with the font manual to make a useful tool for designers?”

The art and science of typography has been important to Adobe since our earliest days. We’re a prolific digital type foundry as well as the developers of PostScript technology and key contributors to OpenType. That’s why we’re focused on solving a paradoxical problem — the more the type universe expands, the more difficult it can be for designers to navigate.

Traditionally, designers have had to use a font manual or scroll through long pull-down menus that list hundreds (or even thousands) of typefaces in alphabetical order to find the right typeface for their project. It’s a surprisingly tedious and inefficient way to select a critical element of design. After all, the right typeface is crucial for bringing the message of a design to life — it’s emotional impact, it’s memorability and usability can all be enhanced or undermined by the quality of type.

So, it would be a real breakthrough if a designer could find inspiration from the typefaces that surround them in daily life. Unfortunately, it’s surprisingly difficult to identify a typeface by sight alone and searching for a match in a type manual can be like looking for a needle in a haystack. That’s where DeepFont comes in. As Hailin demonstrated at Adobe MAX 2015, it can be as simple as point and click.

“A designer can be out in the world, and be inspired by a typeface they see in a poster or billboard,” Hailin elaborates. “With the DeepFont app they can take a picture of it, select the area of type they are interested in, and the app will immediately identify the typeface.” It’s kind of like Shazam, but for type. The same technology can be utilized inside Photoshop as well, through a simple “identify type” command.

DeepFont’s capabilities are made possible by utilizing deep learning. “We used rendered images of 5,000 different typefaces to train the system — about 1,000 images for each typeface, or about 5,000,000 images total. With deep learning we don’t have to manually design the algorithm for font recognition, the algorithm will learn the feature based on a multi-layer interpretation of the data we provide it. It not only allows us to identify the correct typeface, but it allows us to discover new relationships, we might not have thought about beforehand,” Hailin adds.

Hailin demonstrates use of the DeepFont app to select and recognize type from within an image.

DeepFont can do more than correctly identify a typeface. Similar to a feature that shipped with Photoshop in November, it can also suggest similar typefaces installed on the user’s system, allowing the designer greater range of experimentation or economy. Although a designer might not have an exact match of the typeface installed on their system, they may have one that is very similar, or they may wish to select a variation that makes a better match with their creative vision.

The DeepFont app is currently a concept technology and not available to the general public, but Hailin hopes to continue to scale it, improve it and eventually get it into the hands of designers. “Right now it works with about 7,500 typefaces,” he explains. “We want to add even more type libraries into the mix, and we want to expand the type-similarity feature beyond system fonts to include Typekit in the Creative Cloud as well. We see great opportunities in the future to help designers in their daily use of fonts.”

To learn more, check out coverage on DeepFont in Fortune.

This story is part of a series that will give you a closer look at the people and technology that were showcased as part of Adobe Sneaks. Watch other Sneaks and videos here.