The Science Behind Adobe Scan

Business cards are still important currency when we work together, but who wants to be weighed down by paper business cards that will be stuffed into a pocket or lost in the back of a drawer? The latest innovation introduced in Adobe Scan now quickly and easily turns physical business cards into digital contacts on your smartphone. This seems like a simple capability, but there’s a lot more going on underneath the hood. From a machine learning standpoint, making mobile apps like Adobe Scan can actually be pretty complex.

Engineers at our global headquarters in San Jose and offices in Noida, India have been using Adobe Sensei, our artificial intelligence and machine learning platform, to solve a lot of the complicated machine learning challenges that come with Adobe Scan. Functions like recognizing what kind of document is being scanned and determining whether the info in a business card is a name or phone number require high levels of machine learning sophistication that continue to challenge and excite our team.

Are you a business card or a receipt?

Scanning a business card seems simple. Point, click, scan. But how does Adobe Scan know that what you scanned is a business card, rather than a receipt, a contract, or any other kind of document? This is where Adobe Sensei comes in. A sophisticated algorithm around document classification is running in the background in Adobe Scan. We feed the algorithm different examples of documents so that it can learn to identify the kinds of documents being scanned—in this case, a business card.

But there’s more Adobe Sensei beyond recognizing the type of document you are scanning. By using Adobe’s advanced image processing techniques, powered by Adobe Sensei, Adobe Scan can make the digital text on a business card extractable, reusable, and searchable in a secure, reliable PDF. It even automatically removes unwanted objects from your business card scan, like that thumb or finger you’re using to hold the card. This is actually a pretty big challenge because business cards are not made in the same way. They all have different colors, different fonts, and are even scanned with different backgrounds.

What’s in a name? A heuristic approach

We’ve come a long way in incorporating Adobe Sensei into Adobe Scan. One example is recognizing the different fields of a business card like names, phone numbers, addresses, and emails.

We’re using the science of heuristics or identifying patterns for each of the fields in a business card. An email address, for example, will likely include “.com,” “.edu,” or “.gov.” By incorporating heuristic principles into Adobe Scan, the app can recognize with high confidence that “joe.smith@adobe.com” is an email address. Our team continues to work on new Adobe Sensei models in Adobe Scan that will quickly recognize more fields like company names and addresses with even higher accuracy.

For customers, by customers

No matter how sophisticated the AI and machine learning is, it won’t matter if it isn’t easy for our customers to use. Our partners in product marketing continuously look for both internal and external feedback to improve the usability of Adobe Scan. In some cases, Adobe employees are the best beta testers! We installed beta builds of Adobe Scan in our colleagues’ devices who work in areas like business development and sales management. They certainly amass a lot of business cards over the course of the year. We asked them to share feedback like what were the most important elements of the business card to capture? Was the process from scan to contacts quick, intuitive, and seamless? We asked even the most basic questions like is this feature particularly useful in your day-to-day work?

Where we go from here

The most exciting things about infusing Adobe Sensei into Adobe Scan are that the possibilities are endless, so the hard work continues. The San Jose and Noida engineering teams continue to investigate capabilities like being able to run OCR within a specific device rather than the cloud. Running OCR directly on a device means a more real-time experience converting physical text to digital text. And what if, after you scan a document, Adobe Scan recognizes the document as a form and then asks you whether you want to fill it out, making suggestions for you?

As you can see based on the machine learning work already done with Adobe Scan, the engineering teams behind it thrive on what’s possible and what the next frontiers for mobile scanning look like. For now, see the latest Adobe Scan in action and try it for yourself.