Summit Topic #1: Visitor Scoring in SiteCatalyst

Whew! Omniture Summit 2010 has come and gone, but hopefully you’re still deriving real value from some of the thoughts, tactics, and strategies that were discussed, both formally and informally, at the conference. As promised, I will be blogging about each of the three advanced SiteCatalyst solutions that my team and I covered in our breakout session. (It’s also worth noting that, while we discussed SiteCatalyst specifically in my session, these ideas become even more powerful when coupled with an advanced segmentation tool, such as Discover or ASI, or when used in the context of Test & Target.) Let’s get started with the first topic from our session.

Visitor Scoring—an overview

You probably have a number key activities on your site that constitute a highly engaged visitor. These are things that you want users to do, because they either constitute direct success (i.e., conversion), indicate the building of loyalty (e.g., social elements such as “Write a Review”), or suggest that the user is truly interacting with your site (e.g., internal searches, viewing photos, etc.). Visitor scoring in SiteCatalyst allows you to assign numerical “scores” to each of these key activities, and then aggregate these scores as visitors move through these different site elements.

The idea is that you can begin to see how varying visitor (and visit) scores affect conversion and customer loyalty. For example, you may notice that users with a score of 30 or higher spend twice as much money on each order when compared to users with score less than 30. You can also drill down to see which site elements are preferred by highly engaged visitors so that you can focus your efforts on those areas. Additionally, the solution allows you to see which visitor acquisition channels generate the highest levels of interaction with key site elements, and which campaigns are the stickiest in terms of leading users to these aspects of your site.

To do this, you will need three things:

  1. A scoring system that assigns values to each of your top 5-10 key engagement points.
  2. One “counter eVar,” which will be explained below.
  3. One custom event, set to “numeric” as its type.

The Scoring System

This is the most complex aspect of the Visitor Scoring solution because you will need to lay out a relative points system for the 5-10 top site elements that you want users to engage. Some users may have access to advanced systems for scientifically assigning relative value to different site activities, but in the absence of such a system here is how we recommend doing this:

  1. Identify the top several key “things” that you want users to do on your site (including, but not necessarily limited to, conversion). As described above, these should be activities that show engagement; the idea is to gauge how active a visitor’s experience is.
  2. Isolate either the activity that you consider most valuable or the activity that you consider least valuable on your list.
  3. Assign that value an arbitrary score, such as “5.” (This can be anything, and can be adjusted later.)
  4. Assign other scores relative to that first score. For example, if we believe that “internal search” is our important, but is our least “valuable” activity of those on our list, we might give it a value of 4. Let’s say that subscribing to a newsletter is significantly more valuable; we might give it a value of 8. Reading a review may be even less valuable in our minds than performing a search; we’ll give it a value of 3.

This chart shows an example of how this system might be laid out:

This will be entirely unique to your business and, at the beginning, may involve a bit of educated estimation. Fortunately, as you work more closely with these reports, you’ll be able to see where scores are being inflated (or deflated) by user activities that have been overvalued or undervalued, and you can adjust your scoring appropriately.

The Counter eVar

A staple of almost any SiteCatalyst implementation is the “eVar” variable, which allows you to set a value (e.g., an internal search keyword) and persist it for a customizable period of time so that you can tie subsequent success metrics back to the values. (In the convenient “internal search keyword” example, an eVar allows you to view the amount of revenue, number of leads, etc. that occurred after the given keyword was searched.) These eVar variables accept text strings by default, but also have a nifty option for accepting numbers. Adam Greco defined counter eVars in outstanding detail in a previous blog post, which contains information on enabling counter eVars as well as a number of use cases.

In the case of Visitor Scoring, we’re going to take advantage of eVars’ persistence by setting the number of points for the key activities that were defined in the previous step into the eVar each time the activity occurs. For example, based on the chart above, we would set a value of “+3” into the counter eVar immediately after the user rates a product, “+5” every time the user e-mails an item to a friend, and so forth. Implementation-wise, using eVar1 as an example, it looks like this:

// after an internal search s.eVar1="+4"

// after writing a review s.eVar1="+5"

(Not that this has anything to do with Visitor Scoring, but it’s worth mentioning here—because we forgot to do it in one of our two chances to present this topic—that you CAN pass negative numbers into counter eVars to subtract from the overall visitor value.)

As the user moves through an experience on your site, interacting with the various key elements that you have defined, he/she will accrue points. If the user converts, you’ll be able to see their “score” at the time of conversion. If they don’t convert, you’ll be able to see that, too.

Pretty confusing, right? We can see that scores of 16-17 appears to be the “sweet spot,” but it’s hard to really understand what we’re looking at here. Fortunately, you can use SAINT to “group” visitor scores, and note that SAINT is flexible and allows you to rearrange/reallocate scores into different buckets depending on your changing needs and observations. After doing this, the report is much more digestible and actionable:

Now we’ve got something we can use. This makes it much easier to see how different engagement levels affect conversion. Again, how you divide up the different scores into groups is completely up to you, and you’ll probably want to adjust it as you dig into these reports. Also, note that you can do some really powerful things here with segmentation; using Discover, ASI, or Data Warehouse, you can focus in on the user experience both for users in the “Very High” group and in the “Low” group to see what they’re doing, and optimize your site around those findings.

The Numeric Event

The final piece of our Visitor Scoring system involves a numeric (a.k.a. “incrementor”) event, which you can enable using the Admin Console. (Mr. Greco documented numeric events as well in a previous post.) This might be my favorite part of the solution.

In addition to setting a counter eVar whenever a user does something that we’re scoring, this method also sets the value in an event so that you can view the score as a metric in various reports. This does involve using the s.products string, but don’t worry; we’re not going to mess with any actual product data. The implementation would look something like this (expanding on the examples given above and using event2 as our numeric event):

// after an internal search s.eVar1="+4" s.events="event2" s.products=";;;;event2=4"

// after writing a review s.eVar1="+5" s.events="event2" s.products=";;;;event2=5"

(Make sure to note that “event2” exists both in s.events and in s.products in this case, and that there are exactly four semi-colons in s.products before event2 gets set.)

The great thing about this is that it allows you to see how various data dimensions affect engagement with key site elements. You’ll probably want to set up a calculated metric to divide this “score” metric by visits, because the raw score may be higher for different data dimensions simply due to varying levels of traffic; for example, when viewing this metric in the Campaigns report, a campaign that has 10,000 click-throughs will likely have a higher visitor score than a campaign that has 10 click-throughs simply because the overall traffic level is higher. When we view various marketing channels through the lens of this calculated metric, we immediately get a great report:

Social media sites, natural search, and partners are the clear winners in terms of bringing interested, engaged visitors to our site. Looks like we know where to focus our efforts—especially if we already know (from our experience with our counter eVar) how much more conversion a high-score visitor is likely to generate on our site. This gets even better when we focus on individual items within our top channels, breaking down this report by referring domain:

Not only do we know that social media brings eager visitors to our site, but we even know exactly which social media efforts/campaigns were most powerful (in this case, Fark.com, Delicious, LinkedIn, and Facebook).

Conclusion—and two bonus tips!

We didn’t mention this during the Advanced SiteCatalyst session at Summit, so be glad you read down this far. A really useful twist on this solution if you have two available eVars for use with visitor scoring is to configure one of them to expire at the end of the visit and another to a much longer expiration (such as “never”). The first eVar then gives a uniform view of individual visits, and how varying levels of interaction with key site elements affects conversion within the individual visit only. The second eVar would provide a view of “lifetime engagement” across multiple visits since the user last cleared his/her cookies. You can slice and dice both data sets to get some powerful views into how user behaviors may change over time.

Along these same lines, it’s possible that different teams or individuals may want to assign different scores to certain site activities. Don’t fight about it! Don’t let this scoring system destroy the harmony in your marketing department. Instead, similar to the tip just mentioned, if you’ve got an extra eVar, you can actually assign different scores to the same key site activities. Here’s a quick example using s.eVar1 and s.eVar2:

// this one is for the first team s.eVar1="+5" // this one is for the second team s.eVar2="+30"

So there you have it. One tactic down, two to go. Next time, we’ll cover perhaps the most popular of the topics that we covered—the brand-new getPercentPageViewed plug-in. If you have any questions about Visitor Scoring, either based on this blog post or on the content we shared at Omniture Summit, please let me know by leaving a comment here. You can also contact me via Twitter (@OmnitureCare).