4 Lessons Learned From the 2010 Online Retail Holiday Shopping Season

By Jonathan Ranger

Many retail websites during the 2010 holidays failed to meet the performance demands of consumers who expect web pages to download in two seconds or less — even during peak traffic periods. A recent analysis of web page visits found that end user abandonment rates rise 8 percent with an extra two seconds of wait time, meaning that slow websites can have a negative impact on revenues and profitability, as well as brand and customer loyalty.
This year’s underperforming retailers share three common mistakes:

load testing isn't properly prioritized;
key stakeholders aren't involved in the load testing process; and
load testing isn't conducted from the end user’s perspective.

In contrast, this year's most successful retailers have taught us several key lessons (applicable to any peak traffic event) which can help others realize a profitable holiday season. These lessons include the following:

1. Early and thorough planning is critical to your online success. You must allocate the appropriate time and effort to execute an effective load test. Otherwise, your load testing is rushed, ineffective or incomplete. These tests must take into consideration what pages, applications, customer segments and geographies need to deliver the best performance. By solving these problems first, you'll engineer a more streamlined, strategic load test that identifies the most critical (and potentially costly) bottlenecks. You'll also have adequate time to address these issues.

2. Both marketing and IT execs must be considered in the plan. Before executing a load test, define what constitutes acceptable performance. Conduct the test based on those parameters. Representatives from your marketing team should be involved so they can share information on anticipated sales and growth projections in order for the load test to be scaled properly. Likewise, representatives from your IT department must communicate the capabilities and limitations of the environment, not so much in terms like network capacity and CPUs, but how heavy loads will impact the end user experience given the realities of the current infrastructure.

Ensuring effective web performance is a shared marketing and IT responsibility. Through informed, proactive communication, marketing and IT execs can ensure the best possible end user experience within an acceptable cost structure.

For example, let’s say your marketing team wants to add new site content, such as graphics or video, during an anticipated period of peak traffic. The IT team can then demonstrate how the addition of this content will likely impact the end user experience under load for critical revenue-generating applications. This type of cost/benefit analysis can expand to cover various factors such as third-party services investment, internal resources and potential downstream influence on performance. Ultimately, this will help you achieve a critical balance between website functionality and performance — one that aligns to end user expectations.

Similarly, your marketing team can communicate which website sections and applications are apt to be most crucial during a peak period (well before that peak period hits), so IT can align resources more strategically and flexibly to the areas of greatest need. A common language between technical and nontechnical groups centered on the end user experience is absolutely essential.

3. Protect your customers from poor third-party performance. Today’s websites are a combination of elements served by an average of eight external hosts. These elements include shopping carts, ratings and reviews, analytics, and more. Any one of these services performing slowly will slow down your entire web page. It's imperative that your test plan, test tools and test team include all third parties providing features and functionalities to your site. After all, if a third-party service fails to scale under load, your brand and business results will be affected, not theirs.

4. Geography, browsers and devices … you must include them all. Your load test must offer direct perspective into the end user experience under various load sizes across the most common usage scenarios and key geographies. For example, are the majority of your customers Internet Explorer users in North America or Firefox users in Europe? Which mobile devices do you receive the most traffic from — BlackBerries, iPhones or Sidekicks? End users in developed geographic regions generally expect the same high level of performance regardless of their usage scenarios. Therefore, your test plan must cover the most prevalent scenarios.

As the economic recovery gains momentum, you don’t want to just survive the holiday season, you want to thrive. Planning early and thoroughly combined with collaboration between your marketing and IT teams creates a recipe for effective load testing. There’s no better time than the present to consider how the four lessons outlined here can improve on the work you did last year as you prepare to make the most of this year’s peak shopping season.

Jonathan Ranger is Gomez benchmarks practice director at Compuware. Jonathan can be reached at jonathan.ranger@compuware.com.