Reimagining the Customer Profile — Lessons Learned from Customer Implementations
Adobe Experience Platform’s Unified Profile brings together customer data from disparate sources and weaves it together into a robust and true 360-degree view of the customer. Adobe Experience Platform’s new and innovative approach requires enterprises to think beyond the traditional relational database to reimagine the notion of what a customer profile can be and all the possibilities that it has for delivering real-time customer experiences.
Image source: Adobe Stock.
Adobe Experience Platform’s Unified Profile brings together customer data from disparate sources and weaves them together into a robust and true 360-degree view of the customer. Adobe Experience Platform’s new and innovative approach requires enterprises to think beyond the traditional relational database to reimagine the notion of what a customer profile can be and all the possibilities that it has for delivering real-time customer experiences.
Unified Profile in Adobe Experience Platform enables enterprises to unify their customer data across many disparate sources, all of which potentially have different identities. Unified Profile offers a whole new approach to developing customer profiles by recognizing that each and all the identities that relate to a single customer’s profile cannot be known based on theory only, and can change over time. This key insight is captured in our design of Unified Profile as a set of profile fragments associated with distinct identities.
Recognizing each identity as a profile fragment provides several advantages, such as the ability to perform a dynamic runtime profile merge with no loss of data, the ability to evolve individual identity relationships, and more simplified data governance. However, this approach has also led to a couple of interesting questions from Adobe Experience Platform customers when they compare the data they upload into the system with the number of unified profiles in their profile store:
- Why is the number of profiles less than the number of records we imported?
- Why is the number of profiles growing when we haven’t added any new data?
Implicit in both of these questions is the fact that customers expect a one-to-one result, that the number of profiles they get when they bring their data into Unified Profile should equal the number of records imported. And when that doesn’t happen, customers naturally assume that an error has occurred somewhere in the process and often spend a great deal of effort to find its source.
Contrary to the perfectly logical expectation of a one-to-one match, these types of inconsistencies are, in fact, normal when working with Unified Profile. They are rarely the result of a problem with the process. Rather, they reflect the power of Unified Profile to match the identities in disparate data sets and stitch them together to create a more accurate and robust customer profile.
This post will describe the reasons for differences between the number of records uploaded into the Unified Profile and the number of profiles returned. Our goal is to provide a fuller understanding of how Unified Profile works and to save our customers significant time investigating false or perceived errors.
Understanding the Unified Profile data model is key to understanding your results
Unified Profile is built on Adobe Experience Platform’s Experience Data Model (XDM), which provides the necessary structure to ingest data from multiple sources, including demographic data, CRM data, online clickstream data, offline and online transaction records, computed attributes, and more.
The way Unified Profile works to stitch identities together from all the different profile fragments ingested into the system has implications for how we interpret results, and it requires a new way of looking at those results. This is because the number of profiles is usually different from the number of records uploaded into the system. Understanding how the data model for Unified Profile works is key to interpreting results.
The Unified Profile data model has the ability to ingest and stitch together two fundamentally different types of data (as shown in Figure 1):
- Time-independent data: Profile data that has time-dependent attribute values, and such a value is considered the latest value.
- Time series data: Consumer behavioral data presented as a set of time series events, where each event has a time-ordered set of values.
Figure 1: The two main types of data in the Unified Profile data model.
A single customer can have many different and disjointed identities, such as an email address, a CRM ID, a phone number, and so on. Each profile data set might contain one or more of these and other identities. The relationship between any or all of these identities may not be known theoretically, and that relationship can change based on new data.
Unified Profile data is organized by identity, much like a document-based data model in which objects are stored as documents containing both the data and its description. This is in contrast to a relational database model, which stores data in a table with columns named based on the type of data they contain.
In Unified Profile, profile data associated with a given identity is placed in a container, much like a document, which we call a “profile fragment.” Naturally, each identity can have only one profile fragment associated with it. All the profile fragments exist in the system separately, which allows them to be stitched together in any number of ways to create a unified profile. This structure is also how Unified Profile can be continually updated and enriched with new data. As new data streams into the system, new profile fragments are added, which can be stitched into the existing Unified Profile to make it more accurate and robust. An anonymous profile is a logical collection of one or more profile fragments, one for each identity. The diagram below illustrates how profile fragments can be stitched together into a unified profile (Figure 2).
Figure 2: Stitching together disparate types of identities, each of which create fragments of the customer profile into a single Unified Profile.
This approach allows identity relationships to evolve independent of profiles. The holistic view of a unified profile is created by using a lazy merge (run-time merge) of the profile fragments. This approach makes it easy to handle privacy regulations when dealing with a combination of related personally identifiable information (PII) and non-PII identities.
Now, let’s take a closer look at how this approach explains the apparent inconsistencies we see in our Unified Profile results.
Scenario 1: The number of profiles in Unified Profile is less than the number of records uploaded
The Unified Profile system is designed to ferret out common identities among different data sets and use them to create the linkages necessary to build the Unified Profile. Therefore, the total records ingested are not additive and will never total the number of resulting profiles.
For example, if there are two separate records in your CRM data that share a common identity, Unified Profile will recognize those as the same identity and link them together to create a single profile. Once they are linked, the resulting number of profiles is reduced by one. Linkages will also occur between different types of data sets. Behavioral data will often contain many of the same types of values that exist in the CRM data set. For example, you might have a login ID, which is the same as the CRM ID, along with an email address. When the email address from the CRM starts showing up in the behavioral data, Unified Profile will recognize those records as the same and link them together, again reducing the number of resulting profiles.
Another common issue can arise when you have two or more different teams loading data into Unified Profile. For example, one team may be responsible for loading the CRM data, while a completely different team is loading data from other sources such as Adobe Analytics. And, the two teams may or may not be coordinating with each other.
Within Adobe Experience Platform, segmentation happens in a single place. So, when one team performs segmentation on the data, if they are unaware of data uploaded by another team, the number of profiles returned will be larger than what they expected based on the number of records they uploaded. In cases like this, it is important to remember that the system allows the data ingestion to come from many different sources.
Scenario 2: The number of profiles in Unified Profile is growing on its own, without additional uploads
The number of profiles can also increase. Remember that each data set ingested resides independently in the Unified Profile system as a collection of profile fragments, which allows identity relationships to evolve independently of profiles. The moment you build an identity graph, connections start happening. As time goes on, new data is continually streaming into the system, and new data means new linkages.
So, even if you are not uploading new data, the probabilistic algorithms used in the identity graph calculations are going to change how the graphs are displayed based on the streaming data. The periodic recomputation of the identity graph is going to split identity graphs, connect identity graphs, and change how they behave. Therefore, even a slight change in the data can provide a different segmentation result.
Real-time data ingestion and real-time identity graph
Historically, data was data. It existed in a traditional database, and when you ran the query you would get your results. Now, with Adobe Experience Platform, we have created a system with multiple moving parts, including identity graphs and any number of data sets that are used to build them. In order to get the same results for our segmentation every time we run it, the sum total of the records ingested and the resulting identity graphs have to remain static. This cannot happen in Adobe Experience Platform’s Unified Profile system because it was designed to stitch together data from multiple sources, including streaming data, to create more fluid profiles that reflect our ever-evolving knowledge of the customer. This requires that we think differently about our expectations regarding the results of our segmentation.
Consider a scenario in which two of those 20 people are sharing the same computer. One of them goes into a browser to check his Yahoo email. Later, the other person goes into the browser to check her Gmail. From a browser perspective, these people look like the same person. So, when that data is ingested into the system, they will be linked as one person because the ECID is the same, resulting in 19 profiles instead of the 20 you might expect.
These situations are naturally occurring phenomena. Families share computers, and there are shared computers in libraries and on college campuses. Situations such as the one described above naturally create additional linkages in the data that are troublesome. Some of these issues can be figured out if you are aware of the possibility ahead of time given the particular data set you’re using. However, it is also not at all uncommon for a person to have two or more email addresses, which makes it difficult to determine in many cases — whether the difference in the number of profiles ingested reflects two different people with their own email addresses or one person with two email addresses.
It’s time to reimagine our notion of customer profile
When issues like those described above occur, there is always a desire for a detailed explanation for why our results differ from our expectations. Determining the specific reason for them requires a significant amount of forensic investigation. Problems like these can keep both our customers and Adobe Experience Platform’s support teams spinning their wheels trying to find an answer because the normal way to solve them is iterative, time-consuming, and, we would suggest, unnecessary.
The unique and powerful capabilities of Adobe Experience Platform and its Unified Profile require that we begin to think differently about our data and how it can now be used to build a holistic understanding of the customer. Consumer behaviors change over time, and with its ability to incorporate both static and streaming data into profiles, as records start flowing in, new data is going to continually create new linkages and disconnect others. This will cause conditions to fluctuate, and, in our segmentation queries, the counts will change constantly.
When you understand all the possible reasons why these differences occur, you can be confident that they are just the nature of working with Unified Profile. Adobe Experience Platform and Unified Profile offer capabilities that reflect today’s data landscape and require us to reimagine our notions of the customer profile from one that is relatively static to one that is fluid. Unified Profile empowers enterprises to recognize the ever-evolving nature of customer profiles, allowing them to deliver richer and more relevant customer experiences in real time.
