Earning the Right to Change Your Data

The CEO of a public company had just said it was a terrible time for us to change our data. What do we do? I asked. Not change the data? He paused. Change it, he said. Just don’t leave us hanging.

Without intending to speak for the industry, he had. The industry would absorb change — but with terms that felt steep, yet reasonable. The wrinkle was that we had never done it before. No one in our industry had. With that, the biggest change to the leading consumer purchasing data in North America gave rise to an equally big shift in change management.

Backing up, data change was obvious. Consumer purchasing behavior, shifting for over a decade, accelerated during COVID just as the first majority-minority generation hit adulthood. It demanded a larger dataset for detail; higher participation thresholds for quality; and an omni-view to capture growth. So bigger, better, beyond data was a no brainer. Or was it?

Despite the obvious need, no one else was doing it. No one had ever obsoleted their own consumer purchasing data with better data before. Why not? What were we missing? Oh. Because it changes everything.

Even when making products that much better, how change is handled matters. We accepted the challenge and did the hard things — and with it, earned the right to change based on four principles we recommend for everyone:

Lesson 1: Anticipate the future to optimize change frequency.

We designed the new standard to capture a tighter view of consumer purchasing as consumers kept some of their new COVID buying behaviors, while introducing some requirements specific to digital-only transaction standards that will be more important in the future. Together with our single-source data architecture, this enables our data to keep pace as consumer behaviors and paper receipt legislation evolve.

Lesson 2: Design product and data changes to anticipate change management — inside your customer.

We back-transcribed and attributed over 20 million additional purchasing transactions so that the new higher standard would be available all the way to the start of 2019 to create a pre-COVID baseline. It was an investment, time and money. But it was right. It’s what we would want if we were our customers — which was our internal yardstick.

Lesson 3: Forget what worked last time. In our case, we had to throw out a core belief.

To understand change, the data needed to exist side-by-side. It required creating a parallel environment to house twin datasets. Internally, this had the look of a religious war as we're committed to one best set of data. We still are. But for this moment, for the purpose of change management, we let them both live. Profound openness helps.

Lesson 4: Don’t just lead change – lead with what drives it.

We have hundreds of customers with thousands of brands. They would all want to know what changed, and why for their brands — ideally at the time of launch. Gulp. This required automated change reports for a moment in time done at scale. All new — at least in our industry. For the first time, we published over 5,000 pages of custom reporting to familiarize hundreds of clients with change in core metrics. For their brands. In advance.

Changing data that impacts your clients — even ones like ours who champion change in the name of growth — is hard. Start by changing how you think about it.

Kelly Smith Dotson is president of Numerator, a data and tech company reinventing market research.