Boost Your Online Store's Customer Experience and Retention With A/B Testing

In today’s competitive retail landscape, customer experience (CX) can make or break a brand. The numbers are clear that CX is now a primary factor in growing revenue and retaining customers. That means your store’s CX is too important to leave to chance. To deliver the experience your customers expect and then surpass their expectations to keep them coming back, you need to build an A/B testing program.

A New Era of Customer Experience Expectations

A look at the data makes the value of good CX clear. It’s not only that consumers have lost patience with glitchy websites — 95 percent of shoppers will quickly abandon an online store with screen-blocking ads or broken links — it’s also that customers have never had higher expectations for their shopping experiences. A 2018 PwC survey found that:

• 65 percent of U.S. consumers say good CX influences them more than ads;
• great CX lets retailers charge up to 16 percent more; and
• one bad experience with a brand they love will drive away one-third of consumers worldwide — and nearly half of LATAM shoppers.

One of the most effective data-driven ways to improve CX is through A/B testing. A/B testing is simply comparing two versions of a page or touchpoint element (e.g., a landing page image or an “add to cart” button) to see which one compels more customers to take the action you want.

How Much Can A/B Testing Improve CX?

Even if your store’s CX is already good, you may be able to identify areas for improvement — if you test the right elements against good alternatives.

• One German software training provider saw a 40 percent increase in customer engagement in an A/B test of one of its homepage elements. Option A featured the company’s three training tracks; option B featured the eight most popular courses. Option B earned much more engagement because it immediately showed visitors what they were looking for — training in specific programs.
• Health insurer Humana saw a 433 percent increase in customer clickthroughs on a website banner after A/B tests on the banner copy and image. Humana then tested another change that delivered another 192 percent gain in clickthroughs.

A/B tests that identify ways to improve CX can also boost retention. How much? Again, it depends on your baseline and the changes you make, but one study found that 85 percent of customer churn could be prevented by better customer service and better CX. With A/B tests, you're able to put your trust in real data and not only in your team and designers’ good taste.

Planning Your CX A/B Tests

Every A/B testing program should start with a plan. If you have known issues, like a checkout step with substantial drop-off, those items should top your to-test list. You may also choose to run tests based on:

• customer feedback from customer service conversations and survey results;
• site analytics that show where customers leave;
• user testing results; and
• heat maps of customers’ site navigation.

As you build a list of elements to test, decide how to prioritize them. After any obvious issues like checkout drop-off, consider prioritizing the CX factors that 15,000 consumers told PwC they value most.

• Efficiency and convenience: Does your homepage immediately help consumers find what they want? You may want to test different top-level categories, menu options and other navigation elements.
• Friendly, knowledgeable service: Do your current customer service options satisfy customers who need help finding products and getting answers to questions? You may want to A/B test your customer service menu, chat features and the placement of your customer service contact information.
• Easy payment options: Does adding payment methods and reducing the number of steps in your checkout process increase orders? Consider running A/B tests for each of these elements.

You can also A/B test the following:

• Sometimes a subtle wording change can increase engagement, as Humana’s landing page test showed.
• Different lengths and levels of detail within site body copy.
• Call-to-action copy and design. An easy to find “buy now” or “add to cart” button can reduce friction and increase conversions.
• Product page design. Test different placements of product images and different formats for product descriptions.
• Do customers prefer a percentage discount or a fixed dollar amount off their total order? Run an A/B test to learn the answer.

Whatever you include in your A/B testing plan, focus on options that you think will improve CX. Don’t simply test changes for the sake of change.

Best Practices for CX A/B Tests

• Having a plan will keep your testing program on track. However, before you implement any tests, decide first if you can get the information you need by asking your customers for their opinion. For example, you might survey your customers to find out which payment options they would like to use, rather than A/B testing multiple options without customer input.
• When you do an A/B test, be sure to get a large enough data set over a long enough period of time to get valid results. A few hundred responses over a few hours or a couple of days are unlikely to give you enough information to make a good decision.
• Start testing only one element at a time so that your results are clearly related to the options you’re presenting.
• Keep testing. A good A/B testing program helps brands continuously optimize their CX. Something that was working in the past might not be the best for today. Your data management platform should enable you to run A/B tests and automate some of the work involved.

Consumers today are more selective than they’ve ever been, and they have more alternatives than ever to turn to when their expectations aren’t met. By developing an A/B testing plan and following best practices for testing and evaluation, your site can always be improving CX. The result is your customers spend more and keep coming back.

Alex Soncini serves as the North American GM of VTEX, the leading global commerce platform that unifies customer experience into one comprehensive enterprise solution.