Tame Your Big Data from the Bottom Up

Not a week goes by when I don’t receive at least a dozen messages about Big Data. Since I focus on the Telco industry, I receive many invitations to webinars and events promising new insights into the now age-old question: “How can operators differentiate themselves from the competition by harnessing the power of Big Data?”

Don’t get me wrong, the topic is of real interest. A March 2014 analysis from McKinsey & Co (titled “Big Data in Telecoms: How to Capture Value from Customer Information”) estimates the potential opportunity stemming from Big Data and advanced analytics is on the upwards of $300 billion. The outputs of these Big Data efforts support improvements in core business as well as new sources of revenue growth. Operators have a distinct advantage over other industries in terms of the treasure trove of data that pass through their networks. Hence, they have the most to gain by pursuing Big Data initiatives.

slide
http://blogs.adobe.com/digitalmarketing/wp-content/uploads/2014/05/slide.png

Unfortunately, as is often the case with Big Data, the path to success is fraught with complexities (i.e. limitations on storing location data or mining internal data stores for commercial purposes) and examples of Big Data efforts gone wrong. (Remember when Google Flu Trends substantially overestimated instances of the flu in 2012 and missed the 2009 Swine Flu pandemic?)

I recently read a piece by Booz & Co. that suggested that the top-down method to solving Big Data questions may not be the best approach for at least two reasons: “[First] the business problem often exceeds the capacity of the avail­able data to solve it, and second, the process of gathering the right data to help solve the problem is poorly understood by many companies.” The article suggested taming data with a bottom-up approach. I’ve seen bottom-up approaches yield tangible benefits for many organizations in the Telco industry and in other industries as well. The approach also lends itself to agile friendly marketing, which has its own benefits.

Step 1. Identify Highly Predictive Data Sets

As a first step in preparing your data for bottom-up analysis, identify the data sets that are the most predictive. Some data are more likely to offer meaningful correlations and therefore should be incorporated as the foundation of robust customer profiles and predictive models. For example, a report from Pew Research shows there is a strong correlation between device features and demographics: 25–34-year-old males with household incomes of $75,000+ and education level ranges from high school education to some college have a strong preference for Android OS.

Usage data can also be a strong indicator of future behavior; as such, voice and data consumption are often incorporated as part of audience segmentation schemas. Another important activity is the mapping of behavioral, or implicit data, gathered through customer engagement with digital channels such as Web, mobile, and email. The results of data discovery will reveal hidden relationships and provide further evidence that not all data is equal.

Step 2. Run Cluster Analysis Across Data Sets

After identifying the most predictive data sets and segments, the next order of business is to perform cluster analysis across the collected data sets. Technology advances over the recent past have greatly eased the once arduous task of identifying statistically significant events. Sophisticated analytics environments with visualization capabilities (such as the Adobe Analytics Data Workbench) are now a foundational part of marketing ecosystem that supports cutting-edge marketing organizations. The practice of algorithmic marketing, whereby data generated insights are executed in real time, is taking hold and operators are scientifically managing a broad spectrum of marketing issues, such as targeted offers based on Next Product to Buy (NPTB) models and progressive offers designed to deflect churn. (See the 2012 McKinsey report, “By the Numbers: Unleashing the Power of Algorithmic Marketing.”)

Organizations that are able to tame their data are able to make decisions ahead of the curve and are successful in delivering efficiency and profitability across the entire value chain:

If you would like to read more on this subject, I have explored the topic of Big Data for Telcos in more detail in the Adobe whitepaper “Scale Big Data into Massive Insights.”