How Backtesting Helps Merchants Find Their Ideal Fraud Prevention Solution

Every merchant needs protection from payment fraud, especially as online payment fraud keeps increasing. Juniper Research estimates that e-commerce merchants will lose more than $20 billion to fraudsters this year, an 18 percent rise over 2020, when fraud spiked dramatically because of the pandemic. Fortunately, there are many solutions on the market that offer merchants protection from fraudulent orders that can lead to expensive chargebacks.

However, it can be difficult for merchants to know how to choose among those options because fraud prevention models vary by provider. Furthermore, those models may perform differently from one merchant to another, depending on the parameters set, the industry and vertical, and the behavior of the merchant’s typical customers. What’s more, merchants may base their selection on simple metrics like cost per transaction or number of chargebacks rather than an assessment of the total value a particular provider can deliver. That can lead to long-term revenue loss that’s invisible to the merchant.

Finding a satisfactory solution can help merchants avoid fraud, chargeback losses and mitigation costs, elevated payment processing fees, and the long-term revenue losses caused by false declines. A solution that’s not the right fit can increase a merchant’s losses by rejecting legitimate orders with minor discrepancies to get to zero chargebacks. That practice can drive down average customer lifetime value, waste marketing spend, and increase the cost to acquire customers. With so many variables to consider, merchants may want to use backtesting to decide among the most promising candidates in their fraud prevention search.

What's Backtesting and How Can it Help Merchants?

Backtesting is a process that lets businesses test fraud prevention providers comparing their performance results via statistical models. To run a backtest, merchants give a set of past orders to fraud prevention providers, removing the results of the actual order screenings that took place when those orders originally came through. The providers use their own fraud-prevention models to score those orders as if they were new orders coming into the merchant’s store, and then they share the results of their scoring with the merchant.

The merchant can compare the results from different fraud prevention providers to decide which one to choose. However, the choice should be less about the absolute number of approvals, chargebacks and false declines than about the total expected loss that each provider delivers.

Quantifying the Costs of Each Fraud Prevention Model

By assessing the risk rate and costs of both inappropriate order approvals and inappropriate order rejections (i.e., false declines) in the backtest results, merchants can determine the true costs and value of the providers they’re considering.

Orders that are inappropriately approved during a backtest can be considered chargebacks. Every chargeback costs the merchant in several ways, including

• the loss of product and its associated handling, shipping and marketing costs;
• the cost of contesting the chargeback;
• chargeback fees paid to the processor that are nonrefundable even if the merchant’s challenge succeeds; and
• potential higher payment processing fees if the merchant incurs too many chargebacks.

These costs can all feed into the total cost of chargebacks in a backtest.

The backtest may also result in the rejection of some good orders. False declines can be hard to verify because without accurate data from the original order review, it can be impossible to determine whether the order was actually legitimate. Fraud prevention providers that don’t manually review flagged orders typically classify all rejected orders as fraud, so merchants that use their services don’t have any insight into their false decline rate.

Merchants that don’t know their false decline rate can use a formula to estimate the rate of false declines in their backtest dataset. To do this, they can identify a percentage of orders rejected as suspected fraud that they can consider lost sales instead. For example, a merchant that knows its vertical has a 30 percent fraud rate can assume that 70 percent of orders that were rejected without expert review were good orders. Then it can use the average value of those orders to calculate the immediate false decline loss.

Figuring the long-term losses caused by customer abandonment can be more difficult, however, 40 percent of consumers across five countries in the most recent ClearSale consumer attitude survey agreed that they would never return to a merchant that declined their order. Multiplying that percentage by the merchant’s average customer lifetime value can estimate those losses.

The backtest must also consider the cost-per-transaction of manual review that’s required to determine which suspect orders are fraud and which can be safely approved. Automatic order approval has a much lower per-transaction cost than manual analysis, although avoiding false declines and customer losses is worth the investment.

Evaluating Fraud Prevention Backtest Results

With results from backtests, merchants can identify the provider that does the best job of maximizing the automatic approval of good orders and minimizing chargebacks and false declines to deliver the lowest overall losses caused by fraud and false declines. This provider will typically be the best option for the merchant.

That may be true even if that solution costs a bit more than competitors, especially if it has a machine learning algorithm that can incorporate the results of manual transaction reviews. That learning can reduce the cost of manual reviews over time, as the algorithm gets better at identifying good orders, so fewer orders require expert analysis.

Backtesting, evaluating expected losses and identifying solutions that minimize them can help merchants reduce their chargeback costs, prevent false decline losses and customer churn, and maximize the value they get from their fraud prevention partner.

Rafael Lourenco is executive vice president and partner at ClearSale, a card-not-present fraud prevention operation that helps retailers increase sales and eliminate chargebacks before they happen.