15 for 15: Improved Metrics and Logic

The improved platform backing SiteCatalyst 15 has made possible a number of the features we’ve discussed on this blog. For example, you can build and apply segments and break down any conversion report by any other conversion report because the platform makes these things possible (as well as a number of other tremendous new benefits that we haven’t discussed here yet—or haven’t yet built on top of that platform!). Today, I’d like to shift focus a bit toward some practical improvements that the platform allowed us to make to existing features/metrics/reports in SiteCatalyst.

There are four of these that I’d like to discuss today, the goal being to help you understand how your data—and your ability to do confident, powerful analysis—is going to improve as you move in to SiteCatalyst 15. Let’s walk through each of these things; I’m sure they’ll generate some questions in your mind, so please leave comments or work with your Account Manager, who has been trained on each of these improvements and empowered to help your organization prepare to leverage them.

1. Visits for Non-Cookied Visitors

SiteCatalyst 14: Site visitors who do not accept persistent cookies via their browser are not included in the Visits total in any report, at either the site-wide or line-item level. This also applies to pathing data; for example, users who block cookies are not included in Bounce Rate calculations, the Fallout report, etc. Note that Discover has always included Visits and pathing data from non-cookied visitors. This occasionally led to a situation where your Unique Visitors could be higher than your Visits.

SiteCatalyst 15: All visitors, regardless of cookie acceptance, are included in Visits counts and pathing data.

What does this mean for me? First, it means a more complete and accurate view of many metrics. Of course, a visit is a visit, and, ideally, cookie acceptance should not determine whether a user’s time on your site counts as a visit. Second, it means an increase in the Visits metric in SiteCatalyst, although the degree of this increase varies depending largely on the type of implementation that you are using. A sample of data from our beta customers show a mean increase of less than 0.5% for first-party cookie implementations, and an increase of 5-12% for third-party cookie implementations (due to differences in cookie acceptance rates).

Keep in mind that this increase will also affect calculated metrics that use Visits or a pathing-related metric (such as Entries, Average Time Spent on Page, etc.) For example, many users have defined ” Conversion Rate” as Orders divided by Visits. After upgrading to SiteCatalyst 15, you may see a minor increase in Visits with no corresponding increase in Orders, leading to what some could view as a small decrease in Conversion Rate (which, as any analyst will tell, can often raise red flags).

This also means that you should never see your Unique Visitors count exceed Visits.

How should I prepare for the change? You can determine the increase in Visits (and pathing metrics) that you are likely to see post-upgrade by logging in to SiteCatalyst 14, going to the Site Metrics > Visitors > Daily Unique Visitors report, and clicking on the “Persistent Cookies” filter. This will break out non-cookied visitors from cookied visitors in your existing data; you can divide [non-cookied visitors] by [total visitors] to determine the percentage increase that you are likely to see when you upgrade to SiteCatalyst 15. Then, we recommend working with your internal customers to help them understand and prepare for this improvement in this commonly used metric.

2. Time Spent Metrics

SiteCatalyst 14: Both the Time Spent per Visit report and the Average Time Spent on Page metric (in the Pages report) use Page View server calls (as opposed to custom links, exit links, video views, etc.) to generate timestamps used in order to calculate the time spent. To improve processing efficiency, Average Time Spent on Page metric is calculated by assigning visit times to “buckets” (such as “30-60 seconds,” “1-3 minutes,” etc.), giving everything in each bucket the median value for the bucket (45 seconds or two minutes in the examples just given), then averaging the number of instances in each bucket.

SiteCatalyst 15: Both the Time Spent per Visit report and the Average Time Spent on Page metric use all server call types to generate timestamps, giving you additional insight whenever non-Page View data is captured. For example, SiteCatalyst 15 provides much improved reporting regarding time spent on pages that include videos. Additionally, the Average Time Spent on Page metric does not use “buckets,” but instead averages out the time spent for each and every individual page view, except for anything occurring after the final server call of the visit.

What does this mean for me? Time Spent per Visit is likely to increase, although it is impossible for me to say in a blog post how much; that answer depends on your implementation and the extent of your use of custom links, file downloads, exit links, and video views. Average Time Spent on Page may increase or decrease, since SiteCatalyst 15 does not use buckets. For example, the median value for the “1-3 minutes” bucket is two minutes; if your visits within that bucket does not follow an even distribution, but is instead weighted toward one minute, using the true average will result in that set of visits having a mean below two minutes, thus possibly lowering the Average Time Spent on Page metric for the given page.

How should I prepare for the change? If you have Adobe Discover, you can examine the difference between your Time Spent data in Discover and the same data in SiteCatalyst 14; Discover has been using the improved logic for time data since Discover 2.9.

3. De-duplicated Visits/Visitors in Classifications

SiteCatalyst 14: In classification reports, such as the Campaigns report, the Visits (or Visitors) total for a value represented the summed total of each of the Visits (or Visitors) Key values (e.g., Tracking Codes). For example, consider two Visits that happened to include the same three pages. At the page level, you’d see two visits/visitors to Page A, two visits/visitors to Page B, and two visits/visitors to Page C. Now let’s say that those three pages are grouped using SAINT classifications into a value called “Product Pages.” If you were to run the classification-based report containing “Product Pages” in SiteCatalyst 14, you would see six visits and visitors for “Product Pages” (2 + 2 + 2).

SiteCatalyst 15: Classification values correctly de-duplicate the Visits and Visitors metrics for classification values. In the example provided above, “Product Pages” would correctly show two visits/visitors.

What does this mean for me? Most importantly, it means that you can use SAINT classifications—one of the most popular and powerful features of SiteCatalyst—more heavily than before, because they provide more accurate insight into the number of visits and visitors occurring at the “group” or classification level. Where in the past you may have needed to use additional variables and/or VISTA rules obtain a visit/visitor count for a site section, a campaign, product categories, etc., SiteCatalyst now provides this insight directly through SAINT.

How should I prepare for the change? If you have Adobe Discover, which has de-duplicated visits and visitors for classifications for some time now, you can examine the difference between this data in Discover and the same data in SiteCatalyst 14. As with the previous changes discussed here, you can work with your users to help them understand why these totals are changing, and how they have been improved.

4. De-duplicated success metrics for merchandising eVars

SiteCatalyst 14: Very similar to the previous entry, if a single order/booking/lead/etc. includes n products with the same merchandising eVar value, that eVar value will receive credit for n order/booking/lead/etc. (For example, if I buy two products, both with a merchandising category of “sports equipment” in a single order, SiteCatalyst 14 will report two orders for “sports equipment,” but one total order at the site level.)

SiteCatalyst 15: SiteCatalyst 15 de-duplicates orders for merchandising eVars so that, in the example just given, “sports equipment” would receive credit for one order (since both products were purchased as part of the same order).

What does this mean for me? Note that this only applies to merchandising eVars, not to regular eVars; if you are unsure whether you are using merchandising eVars, check with your Account Manager. For those who are indeed using merchandising eVars, this improvement means that success metrics for merchandising eVar values may decrease, depending on how you are using these eVars.

How should I prepare for the change? I am beginning to sound like a broken record here: Right? Right? Anyway, here goes: If you have Adobe Discover, which has de-duplicated success metrics for merchandising eVars for some time now, you can examine the difference between this data in Discover and the same data in SiteCatalyst 14. As with the previous changes discussed here, you can work with your users to help them understand why these totals are changing, and how they have been improved.

Conclusion

Why publish a post that I know is going to raise questions in your mind? The fact of the matter is that if we only discuss the new features in SiteCatalyst 15, we aren’t telling the whole story. We expect that most readers of this blog will love the four improvements that I have discussed here, but in some cases, training internal users and preparing the business to use and understanding these metrics will take some time. We can’t assume that all of your users are fluent in analytics, and want your SiteCatalyst 15 upgrade to be the best possible thing for your analytics practice. Hopefully this post gives you some direction. Now, get with your Account Manager and talk in more specific terms about what these improvements mean to you and what you should do about them!

As always, if you have any questions about anything in this post, or about anything else related to the Adobe Online Marketing Suite, please leave a comment here or contact me on Twitter and I’ll do my best to get you the information that you need.