iOS 9 Content Blocking Follow-up: Are Your Metrics Affected?

Soon after the release of iOS 9, I made a real rookie-analyst mistake. At the request of a customer, I ran some reports to see if ad blocking was affecting their Safari traffic. It turns out, there was a drop! As iOS 9 traffic increased, the overall mobile Safari traffic decreased. I picked up the red phone. I sent emails around the company. I started a major initiative with a dozen people.

Later that night, I came back to that analysis and did something I should have done the first time. I compared Safari traffic to Android traffic. Odd. Android traffic was following the same downward trend as the Safari traffic! Oh boy. This wasn’t an iOS problem, it was something else. False alarm. Yes, there was a downward trend in traffic for one report suite, but it had nothing to do with the release of iOS 9.

So, to help you avoid making my mistake, let me share some information I’ve learned about iOS 9 traffic trends.

First, there is no measureable drop in Safari traffic across Adobe’s customer base. Second, there are some fool-proof (ahem, Bret-proof) ways to measure the ad-blocker impact on your data. (This is indirect measurement because there’s no way to send hits to Analytics when content blockers are active because those hits are, well, blocked.) Third, your apps are not affected, and there’s an easy way to prevent them from being affected in the future.

Global Impact

What is the impact of ad blocking apps across Adobe’s customer base? So far, it’s not measurable. Honestly, I’m a little surprised. This year’s edition of scare-analysts-into-thinking-their-job-is-over seems more plausible than previous years because it’s fueled by real market problems: limited battery life, costly bandwidth plans, and mediocre network speed. If these problems are so convincing, then why aren’t iOS ad blockers having a visible impact?

Apple does not publish the number of ad-blocking apps downloaded, however, Adobe’s Digital Index team has access to traffic data across hundreds of sites, and they analyzed the Safari traffic trend over the last couple months. How much has Safari traffic dropped since the release of iOS 9? Nada. Notice in the graph below that there is no noticeable drop in the share of Safari traffic in the last 13 days.

AA iOS ad blocking 1

We also took another approach to include standard deviations. In order to do this accurately, we had to sum traffic across customers, which tends to give higher weighting to our largest volume customers. It’s interesting that at a global level, the number of Safari visits has been increasing over the last few weeks. This could be a result of college football or other factors, which is why this view should be taken with a grain of salt. The point is, there doesn’t appear to be a drop.

AA iOS ad blocking 2

Adobe will be watching this trend carefully over the coming months. If it changes, we’ll be sure to notify you.

Local Impact

You’re probably not satisfied looking at global trends, or more importantly, you know your boss won’t be satisfied with a global trend. You want a concrete way to see iOS traffic trends in your report suites. I thought you’d never ask. My colleague, Justin Grover, did some wicked cool, and surprisingly simple, analysis in the Analysis Workspace feature we released last week. That analysis is described in this blog post and video. Here is the chart you’ll be able to create by following his instructions:

AA iOS ad blocking 3

Justin has run this analysis on a handful of report suites. In each case there has been no statistically significant drop in iOS Safari hits.

What Else Do We Know?

Here are some relevant pieces of information you can use to help calm concerns in your organization:

  1. iOS apps are not affected by iOS 9 ad blocking, even if you have a hybrid app that renders web pages. There is one exception to this rule. With the release of iOS 9, a new library called SFSafariViewController can be used to render content. If this library is used, then content blockers apply within the app. If you want to avoid content blockers, make sure your developers don’t use this library.
  2. Apple is planning to release ad blocking support in OSX with the release of El Capitan. Is that the next ad-Armageddon? Probably not. Desktop ad blockers have always had low adoption because battery, bandwidth and speed concerns are not an issue, and while ads can get annoying on the desktop too, there’s plenty of screen real-estate for them.
  3. All of the Adobe Marketing Cloud products appear to be affected by ad blockers, including Dynamic Tag Manager. Even if your collection domain is whitelisted in an app, the DTM container JavaScript file may fail to load. Just watch for this if you do any testing.
  4. Adobe Analytics data is blocked by the largest iOS ad blockers. This makes it impossible (or at least really hard) to create a report in Adobe Analytics that shows how many ad blockers are installed for your visitors. Instead, you’ll need to compare the Safari traffic to the expected Safari traffic as Justin explains here.
  5. As my colleague, Travis Sabin, pointed out, ad blockers require users to opt-in, which significantly lowers adoption numbers. I know that’s a repeat of information you may have, but it’s worth reminding folks you work with. You could also try asking your coworkers how many of their friends have installed an ad blocking app on their phone. Most people just don’t know or care enough to install that kind of app.

Action Plans

In summary, you now have a lot more information at your disposal. You can avoid making analysis mistakes like the one I made. You know what Adobe’s global trends are. This should help calm fears and direct efforts. But iOS ad blocking apps are a big enough concern that you may want to draft a plan in case trends change.

Here are some things we recommend considering. Some you could do now, others you may just add to the contingency plan you’re compiling in case ad blocking grows to measurable levels:

  1. Contact the app developer. If your site is rendered useless by an ad blocker (as we’ve seen in some news articles), or if the customer experience suffers, contact the app developer to inform them of this. The app developer wants positive app reviews. If their app breaks your site, tell them. This approach worked for at least one Adobe Analytics customer already, at least with one app developer. To contact an app developer, go to the app store and find the developer web site for the top ad blocking apps. Currently, only two apps are in the top 100 paid apps: Purify (#3) and Crystal (#42). Adobe is trying to contact these developers as well, but sometimes you have more influence than “the vendor.”
  2. Consider measuring something server-side. (Warning, this could be a big-ish resource investment.) A lot of companies already measure some metrics server-side, such as purchase revenue. If the difference between a server-side metric and a client-side metric is growing over time, you may want to collect some user agent information server-side to see if the iOS 9 discrepancy is growing faster than other browsers.
  3. Stay tuned in. Adobe is actively monitoring the situation and we have a few ideas about how to ensure you have accurate metrics by which you run your business. As we see changes, we’ll keep you informed about the options. As you know, we both have a pretty strong incentive to create reliable and valuable metrics.