Attribution Modeling: It’s Like Ultimate Frisbee

Ultimate Frisbee is one of the favorite lunch-break past times of many Adobe Consultants. If you’re not familiar with the game, it’s actually pretty simple. The object of the game is to score points by throwing a Frisbee downfield to a player in the opposing end zone, while any player in possession of the Frisbee is not allowed to run or move. The team with the most points wins. While we were playing during a lunch break a few days ago, one of my fellow consultants pointed out to me how attribution modeling can be applied to this game the same way we would apply it to digital marketing campaigns. Let me elaborate:

Think of a digital marketing campaign as a player on an ultimate Frisbee team. Each player is working together to move the Frisbee (or site visitor) to the end goal. Oftentimes it takes multiple players (or multiple campaign touch points) before the conversion succeeds, and each contributing player has an impact on the end conversion.

So the $100 question is, when your team scores a point, who gets the credit?

Last touch attribution would give all the credit to the player who caught the Frisbee in the end zone. First touch would give all the credit to the player who made the initial kick-off pass. These are the most common attribution models in web analytics, but neither model incorporates the performance of all the players who helped move the Frisbee downfield!

Perhaps a linear model would be better since it would give credit to all the players who helped move the Frisbee, but it doesn’t give a whole lot of insight into which player has the best throwing ability, or which player is the best catcher, or the fastest runner. Perhaps there are even combinations of players that seem to work very well together, but are not as effective separately.

In my opinion, a good model would give the most credit to the players who participate in scoring runs most often. In other words, when my amazing teammate Jessica Olsen touches the Frisbee, we seem to score more often than when she does not. As a result, I want a model that is sure to give Jessica the credit she deserves. With that in mind, let me explain how we would determine the best campaign performers using statistical modeling techniques, and most importantly, how you can use this for your web analytics data.

First, let’s make the big assumption that digital marketing campaigns are causing people to convert (hopefully your marketing campaigns aren’t turning people away from your products!). With that assumption in place, it’s ok to say that the campaigns that are most correlated with conversion are the most successful.

Second, we need to look at the experience of each visitor over their lifetime and capture all of the campaign touch points they experienced before converting within a look-back timeframe. In order to do this, Adobe Consulting services has constructed a specialized data pull algorithm that summarizes all users’ behavior over any given date range. Essentially, the output dataset consists of a single row for every user that purchased and each campaign they touched on the path to their conversion. With this dataset, the statistical modeling can begin.

One simple and effective model is to create a correlation matrix. This is essentially a grid that shows how related a row and column item are, or how they tend to move together. It’s also important to note that differences in aggregate numbers won’t affect the correlation score, or in other words, if you have a bad player on your team that hogs the Frisbee, they won’t get any extra credit.

In this example, we’re interested in the two far right columns because it shows how correlated each marketing campaign was with orders and revenue (if your marketing campaigns have other key goals or metrics such as ad revenue or lead generation, those metrics can be substituted here as well). You can see that Paid Search is a better Frisbee player than Social Media because Paid Search led to higher revenue more effectively than Social Media did.

So based on these scores, I can assign a weighted average of our revenue so that the revenue gets distributed according to each campaigns’ comparative performance. The formula for that would look something like this: this is a fairly simple approach, but is almost certainly better than first or last touch since the best Frisbee players will get the most credit for their skills. Of course there are more advanced models that can be explored from this special dataset that incorporate additional information such as which channels seem to work well together, how a channel’s influence decays over time, or how a channel/campaign is affected by the order in which it appears for users (i.e. some players are better catchers or throwers), but these are topics for another day.

Understanding which campaigns are actually boosting your site performance is as critical to digital analytics as knowing who you can rely on to catch your game-winning Frisbee pass. If you’re interested in building an advanced attribution model for your own company, please reach out to your Adobe sales rep, account manager, or consultant who can guide you to the next steps.