What’s the Difference between Reporting and Analysis – A Question I Get Asked a Lot
In my last blog, I mentioned Insight as an analytical tool for big data and differentiated Insight as an analytical tool, not a reporting tool. The key question is, what is the difference between reporting and analysis? This is an important distinction for digital marketers. We, as digital marketing consultants at Adobe, get asked this question all the time. It is a difficult question to answer, because there is quite a bit of overlap between the two from one perspective and significant differences from another viewpoint.
What is common between an analytical tool and a reporting tool?
Both tools can store large data. Both tools have defined metrics (Key Performance Indicators – KPI’s) and dimensions (attributes of data). You can make decisions by looking at the data in both tools. The decision-making capability of the one, I would argue, is what differs most.
For example, let’s look at a report. This is a somewhat simple example, but gets the point across very easily. Here’s a report on campaign performance.
Number of Impressions
% Conversion
Number of Conversions
Campaign A
1000000
2.10%
21000
Campaign B
1000000
2.00%
20000
Age
Number of Impressions
% Conversion
Number of Conversions
Less 35
1000000
2.10%
21000
Greater 35
1000000
2.00%
20000
Gender
Number of Impressions
% Conversion
Number of Conversions
Male
1000000
2.10%
21000
Female
1000000
2.00%
20000
Looking at the report, it is obvious that nothing much separates campaign A from Campaign B. The conclusion would be to continue the ad-spend as is, without making any change. Let’s stick to that decision, since that is the best choice we can make with the data we have.
What separates an analytical tool from a reporting tool?
If I am using an analytical tool, I would quickly try to isolate the data for each campaign and attempt to see any trends. Let’s select “Campaign A” and examine its behavior for each age segment.
Age
Number of Impressions
% Conversion
Number of Conversions
Less 35
500000
0%
0
Greater 35
500000
4.00%
20000
Can we stick with the same decision now? Well, you may claim this is a set-up, that the report was badly designed, and that I would have designed the report better with cross-tab with age as the second dimension on the top. Well, I will grant that. Now let’s introduce Gender into the mix.
Gender
Number of Impressions
% Conversion
Number of Conversions
Male
500000
8%
20000
Female
500000
0%
0
Well, now a cross tab does not work. We are at three dimensions and reporting fails. Let’s assume there is another dimension—region.
Region
Number of Impressions
% Conversion
Number of Conversions
North
250000
16%
10000
South
250000
0%
0
East
250000
0%
0
West
250000
16%
10000
You get the point. It is a losing battle to analyze this data using these reports. In the real world, the number of dimensions available to analyze a simple campaign is on the order of 10’s of dimensions. Other attributes could include time of the day, day of the week, income category, type of creative, etc. This complexity is due to the multi-dimensionality of the data. There is no way to know up-front which dimensions impact a particular campaign. Maybe for Campaign B it is Income Category and Time of Day. There is no way I can guess which dimension will impact the campaign.
It is not only the number of dimensions that complicate things. Imagine the complexity when considering the number of campaigns, number of age categories, and number of regions. This complexity is due to the cardinality of the dimensions (number of items in each dimension). In the real world, to run hundreds of campaigns and create reports for every combination of dimensions is just not possible.
So, going back to the example, what would you do? I would run Campaign A only for males whose age is greater than 35 in the North and the West. Using the same marketing spend and reallocating the marketing spend to only the relevant segments, my analysis would have resulted in a 100% increase in conversions for the same marketing spend.
The goal of this blog is not to say that reporting does not have a role, but rather to make the point that reporting by itself is just numbers. Without an understanding of the big picture, you can’t rely on individual reports to make your decisions. So, in summary, I would encourage all of you to start analyzing your data using the most powerful analysis tools available. You’ll begin to find nuggets of gold hidden in the data that you can use to increase your profitability.
Note: In my previous blog, I promised to write a blog post about cross-sell for financial services. I got great feedback on the previous post and decided to stay on generic topics of data analysis. I anticipate that it will be another 3-5 posts before I return to the topic of cross-sell. In the meantime, I appreciate your feedback, so let me know what you think.