Understanding what AI is using to get results drives smarter, more efficient testing and personalization.
With AI powering your personalization efforts, you can easily optimize your campaigns to send the most effective offers to the right people. But duplicating the lift you see in KPIs on an auto-personalized campaign may not be quite so easy. While AI did improve your campaign success and your customers’ experiences, you don’t know if your customers were segmented by age, location, browser type, or something completely different. However, if you could figure that out, then you could find more customers with that same profile.
The “black box” issue of AI — not knowing why the algorithm makes the decisions it makes — represents a significant challenge to AI and machine learning in general. To help overcome this issue and open the door wide for using the AI-driven personalization capabilities, a team of Adobe data scientists set out to deliver those key insights.
The result is a patent-pending technique named MAGIX: Model Agnostic Globally Interpretable Explanations. MAGIX finds rules that define automated segments that explain the patterns the model uses across the board. Think of MAGIX as an interpretation layer that sits on top of the machine-learning model that actually determines how visitors are allocated.
How MAGIX reveals AI decisioning
MAGIX uses a genetic algorithm to help you understand the decisions made by the algorithms used for personalization. For input, MAGIX relies on the visitor profiles that the personalization algorithms score to determine which piece of content in the activity is best to deliver each of your visitors. For example, the AI machine-learning model determines Sarah’s and Jim’s scores for each offer in an automated personalization activity, and determines that Sarah is most likely to convert for offer A, and Jim is most likely to convert for offer B.
At a high level, MAGIX takes each visitor profile and:
- Generates condition sets — the building blocks of rules for the visitor that reflect what was important about them that resulted in their seeing the specific content they saw. For example, one rule (segment) could be, “If 35 and from Boston, show offer A, but if 52 and from San Diego, show offer B.”
- Uses a genetic algorithm to uncover the automated segments that let it merge all the condition sets across all visitors.
- De-duplicates the list of automated segments it has identified, ranks them, and sorts them by relevance.
Here’s a deeper dive into the details of what happens at each step:
Step 1: Generating condition sets
When MAGIX generates the condition sets, it’s questioning why the machine-learning model showed a given visitor a specific offer or experience — or why the model showed Sarah offer A. MAGIX creates these condition sets by asking what would happen if each visitor attribute in the condition were turned off — would the model still show that visitor the same offer, or would it select a different one? This helps MAGIX determine if the feature is predictive of showing the offer for that visitor.
Let’s say Sarah’s visitor profile has just three attributes associated with it: gender, geolocation (city), and browser. MAGIX does the following:
- Turns off the age feature that shows that Sarah’s age is 35, and asks the model if it would still show her offer A. The model says, “No.”
- Turns off the geolocation feature that says her location is Boston, and asks the model if it would still show her offer A. The model says, “No.”
- Turns off the browser type feature that says her browser type is Chrome, and asks the model if it would still show her offer A. The model says, “Yes.”
If you had additional attributes for your visitors, MAGIX would do the above step for all of them. In the simplified visitor profile example, you see that the rule that defines why Sarah gets offer A is that she is 35 and from Boston — her age and geolocation matter. Her browser type does not appear to influence the model’s decision to show her offer A.
For each visitor, MAGIX generates the set of conditions, or rules, that explains why each received the offer they received. Of course, if you have 10,000 visitors, you’ll have 10,000 condition sets, some of which may be duplicates. Other condition sets may be so granular that they consider 10, 20, 50, or more visitor attributes when deciding to deliver a specific offer. There’s no way to practically apply this information more broadly, and for a human to be able to interpret this depth of information — it’s information overload. That’s where the next steps help.
Adobe Target uses AI and machine learning powered by Adobe Sensei to determine automatically who should get which offer. It personalizes the experience and optimizes your strategy. With new Personalization Insights reports from Adobe Target Premium, you can understand the features that Adobe Sensei uses to determine the segments.
Step 2: Merging condition sets
In this step, MAGIX merges your condition sets (and there may be as many condition sets as you have visitors) to create segments that a human can interpret. Remember, a segment is just a set of conditions like, “35 and from Boston.”
At this stage, two competing goals are considered to determine how “good” a segment is. On one hand, it’s important to create segments that represent as closely as possible how the model actually allocated visitors to different offers. On the other hand, it’s also ideal to build out shorter rules that cover a larger number of visitors in the activity to make them interpretable and actionable_._
If a segment has the highest precision possible, every visitor in it will see the offer that the model thinks they should see every time. Coverage is the size of that segment. You don’t want each visitor to have their own unique rule for interpretation — that’s too detailed, and you might even have a segment for each visitor. But you also don’t want a single rule that covers all visitors — that’s too general, and the segment size would be the entire visitor population.
When developing MAGIX, it’s essential to create segments that are optimized for both goals — that’s why, ultimately, a genetic algorithm was used. This algorithm gets its name because it takes a “survival of the fittest” approach — in this case, identifying the “fittest” visitor segments.
The genetic algorithm in MAGIX takes the population of individuals, which are the condition sets for each visitor. Once assembled, a “fitness function,” or “F-score,” is applied — this score was developed to evaluate if a condition set is fit or not fit. By design, the F-score maintains a balance between these competing goals.
From there, crossbreed or “cross over” these condition sets with each other, taking the various attributes used to define one condition set and trading it for the attributes defined in conditions of another set, and vice versa. This gives rise to a new generation of condition sets.
Also, just as would happen with humans, expect a mutation here and there — maybe switch out one attribute in a condition set for a random one, then start the process again for a specified number of generations.
