Different Vendors, Different Results — Who's Right?

Dear Dr. pROfIt: My company gave a marketing attribution problem to four different vendors. We were surprised to learn that each vendor provided a different solution. For instance, one vendor said that 8 percent of our sales came from email marketing, while another said that 14 percent of our sales came from email marketing. Who's right?

Dr. pROfIt: That’s a dilly of a pickle! The honest answer is that they're all right, and that’s a very frustrating answer. Attribution models use a combination of statistical algorithms, customer purchase data and promotional data (i.e., catalogs mailed, emails delivered, ads displayed, searches) to parse orders to the advertising channels that theoretically caused the orders to happen.

If the statistical expert working on attribution models creates variables in a different way than another statistical expert at another company, you're going to get different results.

Here’s another challenge: You're probably not a statistical guru, right? So, when you're working with multiple vendors, you really have no idea if the solution that one vendor arrives at happens because that vendor is using the best technology.

The best attribution strategy is to execute mail/holdout tests. If you're a catalog marketer, don’t mail catalogs for three months to a small holdout segment — e.g., 5 percent of your 12-month buyer file. If you're an email marketer, hold out email campaigns for a month to a small segment of your 12-month buyer file. If you're a display ad maven, run holdout groups as well.

Holdout groups yield the true incremental value of your marketing activities, allowing you to accurately develop attribution rules without the need for complex statistical algorithms.