Steps to Building a Loyal and Profitable Customer Base
During the first big wave of data warehousing in the 1990s, companies were sometimes described as being "data rich and information poor," a reference to the lack of business insights that still prevailed even after large amount of data had been collected and centralized. With the explosion of communications channels in recent years, that phrase has new currency, as many retailers have been struggling to integrate all these new data sources — and to make use of them to drive business results.
Big data technologies such as Hadoop were developed to help large internet companies (Facebook, Google and Yahoo) sift through staggering amounts of data, but many of the applications at first seemed remote from the challenges faced by retailers. Web companies may have huge data sets, but these tend to be "skinny" — billions of rows about a few things. Retailers, on the other hand, may not have the same scale of data, but their data is richer and more complex. Fortunately, as these tools mature, they're proving to be equally as valuable in solving retail marketing problems.
One midsize retailer with a strong online presence recently found that just the first step in their planning process was taking 30 hours to 50 hours per month to run. This process required rolling up hundreds of millions of line-item purchases by stores, product categories, planograms and customer segments. Because the job was so intensive, it had to be scheduled around other processes, which meant that it could take a long time to complete. Migrating this process to a Hadoop cluster reduced total run time to four hours. Now the planning group has far more time to spend on subsequent steps in merchandising planning, making the retailer more nimble in its merchandising.
In this case, the significant gains in throughput come from Hadoop's ability to split a big problem into manageable chunks, spread the workload across multiple servers and, at the end, reassemble all of the pieces to provide business insights. Retailers have many such complex, intensive processes — e.g., offer management, cross-channel analysis, demand forecasting, to name a few — that present good candidates for the technology. For those retailers with a large online presence, integrating traffic data with shopping and customer data is well-suited to these technologies.