Three Hallmarks of a Successful Marketing Experiment

I have a pret­ty green thumb, but I wasn’t born with it. It took lots of tri­al, error, and exper­i­men­ta­tion to find out which plants grow best in what kind of soil, how much light they need, and when things come into sea­son. I had to test water­ing tech­niques, plant food, sun expo­sure lev­els, and place­ment in my flat—and I had to do it in an order­ly, organ­ised fashion.

The Impor­tance of Exper­i­ment­ing

This might sound like a com­pli­cat­ed way to grow a few plants, but to be suc­cess­ful and fill my house with healthy green­ery, exper­i­men­ta­tion was the best solution.

This same is true in the expe­ri­ence busi­ness. As the famous John Wana­mak­er say­ing goes, “Half the mon­ey I spend on adver­tis­ing is wast­ed; the trou­ble is I don’t know which half.” Mar­keters strug­gle to under­stand which of their adver­tis­ing expen­di­tures are dri­ving busi­ness objec­tives, and which are miss­ing the mark. Media mix deci­sions are often made based on cor­re­lat­ed met­rics, which are prob­lem­at­ic because they don’t always indi­cate causal­i­ty. Some­one who clicked on your ad might have bought your prod­uct after arriv­ing at your web­site, but what if they were already plan­ning to buy?

That’s where a care­ful­ly designed exper­i­ment can come in handy. The pur­pose of an exper­i­ment is to deter­mine a cause-and-effect rela­tion­ship. In the world of mar­ket­ing, exper­i­ments help adver­tis­ers iso­late the met­rics that mat­ter, and deter­mine what strate­gies are caus­ing con­sumer to take incre­men­tal actions. Iso­lat­ing these cause-and-effect rela­tion­ships allows mar­keters to under­stand how adver­tis­ing strate­gies are affect­ing their bot­tom line, pos­i­tive­ly or neg­a­tive­ly. Exper­i­ments help mar­keters under­stand which half of their adver­tis­ing spend is work­ing and which is going to waste.

How do you cre­ate those exper­i­ments? How do you make sure you’re mak­ing deci­sions based on causal­i­ty, instead of ran­dom­ly cor­re­lat­ed data points and key per­for­mance indi­ca­tors? Let’s take a look.

Design­ing a Suc­cess­ful Mar­ket­ing Experiment

There are three steps to exe­cut­ing a suc­cess­ful exper­i­ment that pro­vides action­able infor­ma­tion you can use to improve your campaigns.

• First, you’ll need to cre­ate two iden­ti­cal groups. One will be your con­trol group, and the oth­er will be your exposed group. In the exposed group you’ll intro­duce an inde­pen­dent vari­able, like adver­tis­ing, to see what effect it has.
• Sec­ond, you’ll want to min­i­mize con­t­a­m­i­na­tion of the exper­i­ment. If there are any exter­nal fac­tors that could influ­ence the out­come of your exper­i­ment, try to con­trol them, or, at the very least, keep them con­sis­tent for each group.
• Final­ly, you’ll need to eval­u­ate your results. What were the sta­tis­ti­cal­ly sig­nif­i­cant dif­fer­ences between your con­trol group and the group exposed to a new vari­able? You want to be sure the change you see is due to the vari­able and not some­thing that would have occurred naturally.

To bet­ter demon­strate a good exper­i­ment, let’s go back to my plant obses­sion. Say I want­ed to deter­mine the impact that adding plant food would have on growth. Here’s how I’d design my experiment:

• I’d pur­chase two iden­ti­cal plants (same age, size, soil type, ) from the same store. I would label one pot “con­trol” and one pot “exposed.”
• For the two months, the con­trol plant would only get water, while I’d give the exposed plant both water and plant food. (The plant food is the inde­pen­dent vari­able in this experiment.)
• While going about my exper­i­ment, I’d work to min­imise con­t­a­m­i­na­tion from out­side influ­encers. I’d keep the plants in the same cli­mate, expose them to same amount of light, and place them on the same win­dow sill in my home. If one plant is in the bath­room and one is on the ter­race, the exper­i­ment is flawed.
• After the time was up, I’d look at the dif­fer­ence in results between each plant. Did the exposed plant grow sig­nif­i­cant­ly more than the con­trol one, or was it small­er? Was it green­er? Or even, did it die? Keep in mind that the dif­fer­ence should be sig­nif­i­cant enough to indi­cate causal­i­ty. If the exposed plant is only slight­ly taller than the con­trol, it could eas­i­ly be a nat­ur­al occur­rence. If, how­ev­er, it’s two or three times the size, I could safe­ly assume the plant food was the reason.

I could repeat the same exper­i­ment for oth­er vari­ables, such as sun expo­sure, water lev­els, or even the brand of plant food. After enough exper­i­ment­ing, I’d be able to zero in on a sure-fire method for grow­ing strong, healthy plants.

Your Tick­et to High­er ROI

These ideas are eas­i­ly trans­lat­ed into the mar­ket­ing world—and should be. And while this might seem basic, it’s impor­tant because exper­i­ments can help you avoid cost­ly mis­takes, point to bet­ter ways of doing things, and point the way toward inno­v­a­tive thinking.

But many mar­keters find exper­i­men­ta­tion dif­fi­cult. With efforts spread across silos and plat­forms, it may be hard to dis­cern where exper­i­ments should be used, let alone how to design them. That’s where omnichan­nel soft­ware, which helps mea­sure met­rics from all plat­forms and chan­nels, as well as mea­sure­ment con­sul­tants can lead the way.

A well-designed exper­i­ment shows you the cause and effect of your efforts. It sheds light on which met­rics you should hone in on, mon­i­tor, and work to opti­mize, and can iso­late the strate­gies lead­ing cus­tomers to take incre­men­tal actions that deliv­er real return on invest­ment. With these results, you can tweak your strat­e­gy to max­i­mize those returns, mak­ing the absolute best use of your ad spend.