Contributions to Profit: The Art and Science of Testing
Not long ago, a client of mine was all but certain that by upgrading the paper used for its catalog, sales would increase. Unfortunately, the client scoffed at the notion of running a pro forma break-even analysis to determine how much revenue was needed to offset the additional paper and postal costs. In fact, it took the client’s paper merchant, printer, service bureau reps and myself to convince the cataloger to set up a test before making a global change to this new paper grade.
Simple A/B Split Test
Once convinced, the client set up a pretty straightforward, scientific A/B split test, sending half of its customers and prospects a catalog on its regular paper, and the other half the book with the higher grade paper. To keep the test scientific, the service bureau chose every other nth name from each list segment. In the world of science, the “A” portion of names mailed were the “control” group — the goal was to beat the test “B” portion.
When we analyzed the results after the test, the client and I came to different conclusions. The cataloger concluded the test was a success. After all, there was a marginal increase in response rates, driving in a few new customers in some, but not all, of its list segments. I saw something different. Although results were up slightly in some prospecting segments, housefile response decreased — especially its single buyer segments. This meant that the cataloger was converting fewer new customers (who were just trying out its products) to multibuyers — a key metric and a clear indicator of future success.
Just as important, the incremental cost of the better paper grade wasn’t covered by the slight bump in sales. Prospects were costing more to acquire, and the company was losing slightly more money up front that previously would have to be made up by future mailings and orders. The company was making less profit on a per-order basis.
Jim Gilbert has had a storied career in direct and digital marketing resulting in a burning desire to tell stories that educate, inform, and inspire marketers to new heights of success.
After years of marketing consulting, Jim decided it was time to “put his money where his mouth was" and build his own e-commerce company, Premo Natural Products, with its flagship product, Premo Guard Bed Bug & Mite Sprays. Premo in its second year is poised to eclipse 100 percent growth.
Jim has been writing for Target Marketing Group since 2006, first on the pages of Catalog Success Magazine, then as the first blogger for its online division. Jim continues to write for Total Retail.
Along the way, Jim has led the Florida Direct Marketing Association as their Marketing Chair and then three-term President, been an Adjunct Professor of Direct and Digital marketing for Miami International University, and created a lecture series, “The 9 Immutable Laws of Social Media Marketing,” which he has presented across the country at conferences and universities.
