How to Create Catalog Test Scenarios That Matter, Part 2 of 2

This week in the final installment of this two-part series on the value of creating mail tests that produce measurable, and telling, results for your catalog, I provide takeaway lessons from last week’s example of how one catalog company tested the profitability of using an upgraded paper stock in its catalog. I’ll also list some tips to help ensure your company is conducting productive mail tests.

(For part 1, click here.)

The Moral of the Story
Even scientific tests often succumb to the subjective. Once all of your scientific testing is done, the art of interpreting the data takes over. Returning to last week’s example, as a direct marketer I never would’ve rolled out the higher-grade paper without additional testing. I would’ve wanted to confirm my results by running the same test over again. Even if the test were a runaway success, I would’ve proceeded cautiously.

In the catalog/multichannel industry, there are infinite varieties of tests that can be configured, measured and, ultimately, interpreted.

So how do you become successful and improve your company via testing? I believe the following seven principles, that if adhered to, will allow you to hedge your bets.

1. Always test against something. Your current catalog, e-mail, Web site, merchandise, among other things, can be used as your control.

2. Always create a testing hypothesis. Use this example: If we do X, we expect Y to occur.

3. Always set up logical tests based on scientific principles.

4. Test only one variable at a time. If you test more than one, you’ll never know which variable made the difference. See below for a list of variables you can test.

5. Always do your math up front. Calculate all expenses in advance of the test and set up a pro forma profit and loss and break-even analysis.