
This week in the final installment of this two-part series on the value of creating mail tests that produce measurable, and telling, results for your catalog, I provide takeaway lessons from last week’s example of how one catalog company tested the profitability of using an upgraded paper stock in its catalog. I’ll also list some tips to help ensure your company is conducting productive mail tests.
(For part 1, click here.)
The Moral of the Story
Even scientific tests often succumb to the subjective. Once all of your scientific testing is done, the art of interpreting the data takes over. Returning to last week’s example, as a direct marketer I never would’ve rolled out the higher-grade paper without additional testing. I would’ve wanted to confirm my results by running the same test over again. Even if the test were a runaway success, I would’ve proceeded cautiously.
In the catalog/multichannel industry, there are infinite varieties of tests that can be configured, measured and, ultimately, interpreted.
So how do you become successful and improve your company via testing? I believe the following seven principles, that if adhered to, will allow you to hedge your bets.
1. Always test against something. Your current catalog, e-mail, Web site, merchandise, among other things, can be used as your control.
2. Always create a testing hypothesis. Use this example: If we do X, we expect Y to occur.
3. Always set up logical tests based on scientific principles.
4. Test only one variable at a time. If you test more than one, you’ll never know which variable made the difference. See below for a list of variables you can test.
5. Always do your math up front. Calculate all expenses in advance of the test and set up a pro forma profit and loss and break-even analysis.
- Companies:
- Gilbert Direct Marketing
- People:
- Jim Gilbert

Jim Gilbert has been creating direct marketing programs that drive superior ROI for almost 30 years. Fluent in consumer or B-to-B, creative, operations, and analytics, he marries the strategic and tactical sides of direct and social media marketing in a seamless fashion that gets results. He's CEO of a multidiscipline direct marketing agency, Gilbert Direct Marketing, Inc., which focuses on direct mail, catalogs, DRTV, telemarketing, print, alternative direct marketing media and social media marketing. Jim has been involved in start-ups, expansions and turnarounds, and is an expert in helping multichannel marketers get to the "next level." He's a former adjunct professor, teaching direct marketing at Miami International University, and is President of the Board of Directors of the Florida Direct Marketing Association. Jim loves to talk direct marketing, and has done many lectures on direct and social media marketing.