From the ACR Team: Masking Reimagined
Today, we’re excited to announce a completely redesigned and reimagined way to make selective adjustments in Adobe Camera Raw (also known as ACR, the raw photo processing tool found within Photoshop), Lightroom, and Lightroom Classic that we now collectively refer to as masking. These new masking tools will become available on October 26th, and we wanted to share a bit of the process that went into creating these powerful new features ahead of the release.
The new Masks panel, shown here in Lightroom Classic, with a number of mask groups, each made up of multiple masking tools.
Selectively adjusting certain areas of a photo differently than other areas is a technique that’s nearly as old as photography itself. With black and white negative-based photography, the most common techniques of dodging and burning enable photographers to lighten (dodge) or darken (burn) specific areas of a photo, with the goal of adjusting the balance of tones to support the intent of the image maker. Ansel Adams may have summarized it best in his quote, “Dodging and burning are steps to take care of mistakes God made in establishing tonal relationships”, and his finished prints were realized only through extensive dodging and burning.
Early versions of Photoshop introduced tools like the dodge and burn tools, as well as selections, masking, layers, and layer masks which enabled photographers to make selective adjustments in the digital darkroom. Lightroom 2, released in 2008, introduced the brush, linear gradient, and radial gradient tools enabling direct selective adjustments in a non-destructive, photography-specific environment. The new masking functionality represents the biggest change to providing control over selectively enhancing photos since the release of Lightroom 2.
The Adobe Research team (our internal skunkworks group that is tasked with dreaming up the impossible and then making it a reality) has been working on creating new approaches to making selections for some time and had recently partnered with the Photoshop team to release some of their AI-powered selection tools, including Select Subject and the Sky Replacement tool. These tools were incredibly well-received, and made us think — how could we bring them into ACR and Lightroom? The existing image processing engine used by ACR and Lightroom wasn’t compatible with the new AI-powered tools — we were going to need to make some really big changes to that image processing engine. With this big foundational work came the opportunity to reassess the fundamentals of how selections are made within Lightroom, which meant we were going to want to talk to our customers.
The new Select Subject tool automatically creates a precise mask of the salient subject with a single click, and works on people, animals, and inanimate objects.
One of the things I love most about working on the ACR and Lightroom team is that we build our products with our customers, not for our customers. We’ve always prided ourselves on our connection to the photography community, and to build the new masking tools, we pulled out all of the stops. In fact, we started working on these features over a year ago, partnering with our amazing user research, design, and marketing teams to figure out exactly what was missing and how we could solve for those needs. We ended up surveying tens of thousands of customers, across ACR, Lightroom, and Lightroom Classic, ranging from hobbyists to professionals to get an understanding of the biggest pain points with making selections, as well as what they loved about the existing tools, and what they wish they could achieve with their photos. Our Design Research team then talked directly to over a hundred customers over the course of a year and a half to watch how selective adjustments were being done in the current tools and eventually to start testing prototypes and refine the experience until we had a balance of power, precision, and familiarity, something that new customers could become successful with quickly, but that didn’t hamper creativity and control, all while ensuring that existing customers would feel at home with the tools.
New improved Masking Engine
The first opportunity we identified was how we could take advantage of all the new and powerful AI-based, machine learning-powered masking tools coming out of Adobe Research. These tools first made an appearance in Photoshop in the Select Subject as well as Sky Replacement tools, making it possible to identify the subject in a photo (regardless if it’s a person, animal, or inanimate object) or the sky, resulting in a precise selection. The trouble was, there was no way to integrate these new models into the existing selection tools in ACR, Lightroom, and Lightroom Classic.
Select Sky creates a precise mask, even around foliage and uneven edges around the sky, making creating complex selections incredibly fast.
This required the team to go back to the drawing board and re-imagine how the new masking engine could incorporate these AI-based tools across both mobile and desktop devices. We knew it would be essential that photographers have access to our best new tools anywhere, and on any device.
Previously, the only types of selections that were supported were vector based, meaning that any adjustment made via brush or gradient were recorded as mathematical expressions. This meant that small amounts of data could be used to represent even complex masks on very high-resolution images. Vector-based masks are useful for avoiding creating large files and catalogs, which would eat up valuable hard disk space and make synchronization slower. The new AI-based masks, however, require bitmap, or image-based, support. In essence, the new AI-based masks create a grayscale image, where lighter values represent varying levels of selection and pure black represents no selection being made to that area. A rather common example of these grayscale bitmap-based masks can be found in Photoshop’s layer masks. With the new masking functionality, we needed to make it possible for both vector-based as well as bitmap-based masks to live together in harmony. The brush, gradients, and range masks continue to be vector-based (to limit space used when making masks) while the AI-powered tools such as select subject and select sky use these bitmap-based masks. We’ve also got some fun new tech brewing that we’re really excited about that will take further advantage of bitmap-based masks which we’ll be releasing sometime next year.
A reimagined Masking experience
The second major opportunity was to improve the overall interactive experience. With the major architectural work being done to the masking engine, we decided that there was an excellent opportunity to rethink and greatly improve the experience and capabilities in every flavor of ACR, Lightroom, and Lightroom Classic, across desktops, mobile devices, tablets, and the web.
Over the course of a year and a half, the Design Research team brought together user experience designers, engineers, and a wide range of customers to build and test prototypes that drove the process of creating the new masking experience Through working with our customers, we identified four major areas of opportunity for the new masking features:
- More control & flexibility
- Improved workflow & organization of selections
- Consistency across all devices
- Better in-app support to maximize the potential of the tools
More control & flexibility
The most important job of any selective tool, obviously, is making it possible to select specific objects or areas, and only those objects or areas. We needed to find a new paradigm that would not only enable these fancy new AI-powered tools (as well as future tools that are currently in the works) but also make it possible to refine the selections, and make selections that current (and possibly even future) AI-powered tools could not.
To do this, we created mask groups, which make it possible to combine any mask tool with any other mask tool. This means any selection tool can be mixed and matched with any other tool, including the brush, gradient, and luminance and color range tools and AI-powered tools like select subject and select sky.
Add (or subtract) any masking tool to any other masking tool.
We also realized that just adding masking elements together wouldn’t be enough, so we also created the ability to subtract any mask from any other. The ability to subtract an AI-powered mask from another masking tool results in some amazingly powerful selections, and is one of my favorite ways of using these tools.
Next, we made sure any selection could be inverted. For example, everything but the subject can be selected by inverting select subject. Or, select just the ground by inverting select sky. And for an even more fun technique, if there’s a subject in the scene, invert select sky and then subtract select subject from the group to result in the ground being selected without the subject or the sky.
By first using Select Sky, then inverting the sky selection, and then subtracting using Select Subject, a single masking group can be made to select only the ground.
Finally, we heard from a number of our customers that they wanted more powerful range masks, including the ability to target the entire photograph. So we made the range masks work globally (they can still be applied within a gradient, just as before, using the add and subtract controls) and brought them to Lightroom desktop and mobile. Additionally, we added more control over the luminance range’s falloff.
The Luminance Range Mask tool now provides a falloff control, enabling control over how quickly the selection transitions from the selected tonal range to the unselected range.
Improved workflow & organization of selections
With all this new power, we realized it could pretty quickly get overwhelming and hard to keep track of all of the masks. We created a new masking panel to help organize everything. On desktop, the masking panel can be placed anywhere, docked into the edit panel, or minimized showing just a preview of the mask.
Another major request was the ability to name each mask, making it easier to keep track of each mask group and what it’s doing.
Lastly, we brought in a number of the overlay visualizations from Photoshop, so that the mask could be previewed a number of different ways, including the default color overlay, color overlay on black and white, image on black, image on white, and more.
Consistency across all devices
A major topic we heard from our customers was that they wanted a consistent experience no matter where they used the tools. Many customers use a combination of devices and applications and want to make sure that they can start and refine edits with the same tools and capabilities, regardless of where they are. To support this goal, we made sure that every single tool and capability works on any device and in all apps. The AI-powered tools work on mobile devices equally as well as on desktop. The powerful range masks, previously only found in ACR and Lightroom Classic, are also now available in Lightroom as well as in Lightroom mobile. And all of the powerful capabilities to modify and refine masks are equally available throughout the ACR, Lightroom, and Lightroom Classic ecosystem.
Color Range Masks (in addition to Luminance Range Masks) are now available in Lightroom on both desktop and mobile devices.
Better in-app support to maximize the potential of the tools
Something that came up time and time again during our research sessions was the need for a different type of help. Something that wasn’t constantly in your face but also wasn’t full of arcane symbols and shortcuts, expecting you to ‘just get it.’ Found in every client is help built directly into the masking experience. In Lightroom, there’s also a new interactive tutorial that walks you through the various tools, step-by-step.
What’s next?
Both the Adobe Research and Design Research teams have a bunch of more tricks up their sleeves. From new and more powerful AI-powered tools to further refinements and improvements to the interactive experience, keep your eyes open for even more announcements in the near future — we’re just getting warmed up.