Why Retailers Need to Know the Difference Between a Test and Experimentation
A test isn't an experiment. While these two words are used interchangeably by retail marketers, the difference is more than semantics. There are many stark differences between the mind-set, process and outcome of one vs. the other. A test isn't an experiment, and an experiment isn't a test.
Experimentation is done without a fear of failure or expectation of outcome. Testing is done with with an expectation of a winner. Experimentation leads to new things, while testing validates assumptions. Experimentation is dangerous, while testing is safer. Retail marketers want to get credit for experimentation, but they’re held accountable to the funnel and key performance indicators (impressions, traffic, leads, purchases, revenue), so testing and experimentation become like reading and regular exercise: things we say we do, knowing that we should do more to warrant the claim.
In retail and e-commerce businesses, both testing and experimentation are seen as luxurious practices to be implemented when you have extra — extra time, money, resources, energy. However, retailers should embrace testing and experimentation as regular practices, like executing marketing campaigns and pulling reports. Experimentation to discover, testing to validate. Rinse and repeat.
It’s critical to understand the differences between tests and experiments, and know where to plug them into your weekly work. Let’s compare:
Both tests and experiments are critical for the growth funnel, with experiments discovering new tactics that tests can validate, then leading to further optimization at scale. Knowing the difference between these two approaches can help you advance your organization’s learning of what works to drive revenue.
Kasey Lobaugh, chief retail innovation officer and omnichannel retail practice leader for Deloitte Consulting LLP, recently said in an interview with Entrepreneur, “The way you win is you find opportunity, you find where there’s an unmet need, where there’s scarcity or where you can build perceived scarcity, and you go after it in an unapologetic, very focused way. But recognize that those windows of opportunity open and close at a faster rate when the competitive fervor is high. So being nimble, to me, means being able to find those, take advantage of them, move out of them and move on to new ones, fast.”
This is exactly what a culture of experimentation is all about. It’s a cycle of experimentation, learning and iterating to find untapped opportunities.
As retail marketers, we struggle with patience. We want to skip the iterative steps. We want the winner, the silver bullet, so we try to cut corners or mimic competitors to get ahead. But to discover the game changers, we have to own up to investing in growth. Here are a few ways your can begin to invest in experimentation and build a culture of innovation through experimentation in your organization:
- Establish an experimentation quota and budget. This budget can be any amount (start with 5 percent of the digital budget), but it’s reserved for trying new things.
- Establish a testing discipline. Tests can be done in-line with normal campaigns and range from small to large — 12 hours is typically all your need to test two subject lines, headlines or calls to action.
- Make learning a MBO for everyone in marketing, then tie incentives or bonuses to those that are meeting and/or exceeding those goals.
- Celebrate successes and failures of both experiments and tests. Try awarding a trophy for innovation, success and failure alike.
- Share your learnings! Blog about your tests and experiments, your discipline, share them internally and externally (as you can), and don’t be afraid to say “sorry” when a mistake is made.
Like any change, the hardest part of testing and experimenting is establishing the habit. However, a new habit is just two weeks away! Setting up clear definitions for both tests and experiments enable marketing departments to take advantage of the new world emerging around us. What will VR, chatbots, blockchain, social apps, mobile and other innovations do for your business? Start experimenting and testing to find out!
RJ Talyor is the CEO of Quantifi, a marketing R&D platform.