Strategy: Anatomy of a Marketing Test

A/B Split Tests

A/B splits are the basis for all testing. An A/B split is when a group of buyers and/or prospects are divided into two equal groups on an every-other-name basis. Be sure your sample size gives you statistically valid results.

Determine which groups you want to test — housefile or prospects or both. Then, once you identify the test group, the next step is to determine the quantity.

As a general rule, you need at least 100 orders from any group to have a valid read on results. But sometimes it depends on the leap you’ll make when rolling out. Assuming you’re mailing smaller quantities and looking for a minimum of 100 orders per panel at a 2 percent response rate, you need a minimum of 5,000 copies per segment or test cell.

If you’re testing, say, three segments of your housefile and five different outside lists, you need to print at least 40,000 copies for “A” group (the control group) and 40,000 for “B” group (the test group).

Next, assign key codes to every segment for the control group and for the test group; this will enable you to track your results. Proper coding is a must! When writing your merge specifications, carefully sel-ect a cross section of the housefile segments and outside prospect lists so you choose a representative sample across all ZIPs. Select both groups from the same list universe. Just change one variable at a time for a valid test.

Now you’re ready to mail. Mailing results should be tracked at least weekly. And it needs to be at least 50 percent complete before any conclusions can be drawn.

With split tests, note the net benefit of selective binding to keep the mailing in one ZIP stream to maximize your postal discount. Generally, at quantities of 300,000 or more, it’s more cost efficient to selectively bind.