
This week in the final installment of this two-part series on the value of creating mail tests that produce measurable, and telling, results for your catalog, I provide takeaway lessons from last week’s example of how one catalog company tested the profitability of using an upgraded paper stock in its catalog. I’ll also list some tips to help ensure your company is conducting productive mail tests.
(For part 1, click here.)
The Moral of the Story
Even scientific tests often succumb to the subjective. Once all of your scientific testing is done, the art of interpreting the data takes over. Returning to last week’s example, as a direct marketer I never would’ve rolled out the higher-grade paper without additional testing. I would’ve wanted to confirm my results by running the same test over again. Even if the test were a runaway success, I would’ve proceeded cautiously.
In the catalog/multichannel industry, there are infinite varieties of tests that can be configured, measured and, ultimately, interpreted.
So how do you become successful and improve your company via testing? I believe the following seven principles, that if adhered to, will allow you to hedge your bets.
1. Always test against something. Your current catalog, e-mail, Web site, merchandise, among other things, can be used as your control.
2. Always create a testing hypothesis. Use this example: If we do X, we expect Y to occur.
3. Always set up logical tests based on scientific principles.
4. Test only one variable at a time. If you test more than one, you’ll never know which variable made the difference. See below for a list of variables you can test.
5. Always do your math up front. Calculate all expenses in advance of the test and set up a pro forma profit and loss and break-even analysis.
- Companies:
- Gilbert Direct Marketing
- People:
- Jim Gilbert

Jim Gilbert has had a storied career in direct and digital marketing resulting in a burning desire to tell stories that educate, inform, and inspire marketers to new heights of success.
After years of marketing consulting, Jim decided it was time to “put his money where his mouth was" and build his own e-commerce company, Premo Natural Products, with its flagship product, Premo Guard Bed Bug & Mite Sprays. Premo in its second year is poised to eclipse 100 percent growth.
Jim has been writing for Target Marketing Group since 2006, first on the pages of Catalog Success Magazine, then as the first blogger for its online division. Jim continues to write for Total Retail.
Along the way, Jim has led the Florida Direct Marketing Association as their Marketing Chair and then three-term President, been an Adjunct Professor of Direct and Digital marketing for Miami International University, and created a lecture series, “The 9 Immutable Laws of Social Media Marketing,” which he has presented across the country at conferences and universities.