Holiday Shopping With Gen AI. Here’s How I Fared

It comes with the cold and the falling leaves: The mounting horror that time is running out. It was already December and I hadn’t thought once about Christmas presents. I'm not an especially talented gift-giver, and my family doesn't have easily satisfied tastes. Every year I find myself daydreaming about a polite, hypercompetent professional who could do this for me.

I didn’t have such a professional this past holiday season. I did, however, have access to a roster of polite, eager, generative artificial intelligence-powered chatbots. As a data scientist, I’ve been fascinated by the impact these AI products and their rapid proliferation have already had, but I’ve been more of an observer than a participant in the great generative AI revolution. So last month was as good a time as any to let these chatbots try to help me.

Shopping With Gen AI

My family is quite non-materialistic, a tendency I'm fond of any other time but holiday shopping season. But maybe these generative AI assistants could surface something useful. I started by telling them that I needed gifts for three family members, listing a few of their likes and hobbies. I asked for three different kinds of gift ideas: a book they might find interesting published in the last five years, an item they might like, and an experience they might enjoy. I also mentioned a few things they already had, hoping to exclude them as suggestions (my dad already has a fancy chess set and my sister doesn’t need more knitting supplies).

I gave the same prompt to OpenAI’s ChatGPT 3.5, Google’s Bard, and Anthropic’s Claude. Every one of them met the brief, for the most part.

My notes about what my family already had or wouldn’t want went mostly unheeded. My poor sister got the worst of it; the one bot that didn’t recommend more knitting needles instead recommended … a reading nook. Not a Nook, as in an e-reader, or things to put in the reading nook, but the whole nook. This bot also suggested that a fun experience for my mom would be a weekend getaway — to England.

The bots’ suggestions were mostly uninspired. There was one exception when one got confused about the historical period that I said my mom enjoys reading about — the Edwardian/Tudor period. The bot combined that detail with mom’s interest in gardening to suggest buying Tudor-era antique botanical prints to put in her garden. A set of flowers in the style of old scientific illustrations is a cute idea, actually (for inside, though).

None of the bots recommended the same set of three books for each person, although some of the titles did repeat. One of the bots hallucinated a title and synopsis entirely ("The Chess Wars: Mao's Game of Revolution" by Peter Skjoldager (2019) does not, in fact, exist). I asked a follow-up question to all three bots: Where could I buy their book suggestions if I wanted to shop local in Seattle? One of the bots pleaded the fifth and refused to answer, but the other two showed impressive knowledge of Seattle’s indie bookstores.

These bots were astoundingly fluent and comprehending. Rarely did they suggest anything novel or surprising (hallucinations aside). In my case, I did receive a genuinely helpful suggestion that wouldn’t have occurred to me. Point to the bot, even if the suggestion did seem to emerge out of some confusion over my instructions.

The Future of Search?

There's another way we can interface with generative AI — enhanced search. So, I ran one final comparison. On a fresh browser with all of my personal history purged, I used Bing, GPT-enhanced Bing, Google search, and Bard-enhanced Google search to look for places to buy antique-style botanical prints. I wanted to see how results from search alone would compare to recommendations from the generative assistants.

All the searches surfaced links directly to products from a handful of top retailers. Not every retailer represented in the direct search appeared in the AI assistants’ recommendations. Bing had high correspondence, but a low degree of retailer diversity. Google’s results matched decently well between the two search modes, but the retailer I eventually chose didn’t appear in the generative recommendations.

All this has interesting implications for the customer journey. Where and how a retailer appears in search can have a huge impact on its traffic. Does success with traditional search, via search engine optimization and AdWords, translate to success with generative search? How many people find their way to a product or retailer page via generative vs. traditional search?

Should generative AI come to play a larger role in recommending where to buy, merchants will need to figure out ways to stay in the conversation. For now, the mega-retailers enjoy big advantages. Owing to their immense size, they've amassed a disproportionate amount of AI expertise. We can’t assume the same for the local indie bookstore.

As this future gets sorted out, merchants should collect as much data as they can. Ask whether customers found your product or website via a Large Language Model. A lot of sites ask how shoppers heard about their product — including the various social networks. It may be worth adding a bullet asking about generative AI.

If the results don’t uncover a meaningful amount of traffic, merchants can watch and wait. But if other retailers in the same sector are generating non-trivial amounts of AI-related recommendations, don’t dismiss the signals. It’s an early sign that the world is changing faster than you assume.

And remember: We’re in the dawn of the consumerization of AI-powered chatbots — ChatGPT was released just one year ago. The adoption and impacts of generative AI today may bear little resemblance to how the world looks in the full light of day of the generative AI era.

Katherine Wood is a staff data scientist at Signifyd, an e-commerce fraud protection platform.