5 Tips to Transform Optimization from a Sideshow to Main Event (Part 1)
by Drew Burns
posted on 06-15-2017
Adobe’s optimization consultants often sit side-by-side with their clients, either in-person or virtually, to discover what those clients are trying to accomplish. I love the chance to catch up with them because their finger is on the pulse of what our customers are trying to do and accomplish with Adobe Target. Recently, I sat down with Debra Adams, one of our seasoned Adobe Digital Strategy consultants. We discussed a key issue that optimization programs face—optimization is just a sideshow, at the whim of senior management demands and product owner requests.
From her years of experience running and consulting for optimization programs, Debra shared tips for five key actions optimization programs can take to transform from being just a sideshow into part of the main event:
1. Build the business case for testing
2. Overcome the data dilemma
3. Do valid testing
4. Get your seat at the table before big decisions are made
5. Reinforce the value of your testing program
These actions can help you get and keep a seat at the table where IT, the development team, creative, product owners, and executives determine and prioritize the features to develop, establish a development timeline, and allocate available resources.
But first, how can you tell if your optimization program is a sideshow? Read on for a true story that illustrates what it looks like:
Your optimization program is a sideshow if…
The optimization program manager at a large retail company was perplexed: senior leadership demanded that her team run more and more tests. Business unit owners insisted that her team run the tests they wanted, many of which she knew from experience would barely move the needle. Important tests she planned to run were knocked off the table with a simple, “We don’t have room in the schedule or design resources for that.”
She knew that the optimization program had incredible business potential, but she just wasn’t in a position to influence how the organization used it. She was just a marketing manager who had been told, “Get some testing done.”
If this all feels familiar, then your optimization program is a sideshow. The good news? It can be much more—start transforming it into part of the main event with this first tip.
Tip 1: Build the business case for testing
Your testing is up against features that product owners or executives strongly believe will bring ROI. Why should they put off implementing their money-making features just so you can run a test? You have to make a believable argument that your testing has the potential for big impact and ROI—you have to build a business case for testing in general, and for individual tests.
Build the testing business case with:
- Real examples of impact and ROI businesses have experienced from testing, including testing for discovery—not just validating existing ideas.
- Calculations of possible ROI and opportunity from testing based on your traffic levels and the average RPV, AOV, and conversion lift typical of your industry.
- Supporting research from analysts, benchmark data from self-assessment tools like Adobe Target Pro, and success data like the Adobe Target Payoff infographic (shown in a recent blog post by Kevin Lindsay).
- Comparison of possible ROI from optimization against known ROI from other digital investments like display ads, search engine marketing, and other investments.
Build the business case for individual tests with:
- Test objective. Explain the test’s purpose and business value along with your rationale for the audiences you’re targeting.
- Test KPIs. Explain how test KPIs align with business initiatives and make an impact or generate ROI. Impacts can include increased sales or conversions, but also cost reductions like shifting from call center to self-service transactions.
- Supporting data. Show analytics data like drop-off in purchase paths, customer survey data, or other data sources that support your test hypothesis.
- Test timeframe. Use a sample size calculator to show that your test can prove or disprove the hypothesis in a relatively short amount of time.
- Test impact/ROI versus level of effort. Rate the test’s likely impact/ROI and level of effort to run it (low, medium, or high). Show its high potential impact/ROI for low effort.
Next up: Overcoming challenges to your business case
While making your business case, anticipate and prepare for a couple of challenges many optimization programs face—gaining stakeholder trust in your business case and test data, as well as the test design of your individual tests. In my next post, I’ll discuss tips 2 (Overcome the data dilemma) and 3 (Do valid testing) to suggest ways to overcome those challenges.