The Challenges and Pitfalls Retailers Face Starting With Machine Learning

Introduction

For organizations new to ML, getting a solution into production usually takes far longer and requires far more resources than expected—if they make it to production at all. According to Forbes, a shocking 87% of ML models never make it out of the experimental stage and into production.

Why? It’s worth mentioning again that organizations tend to woefully underestimate the time and effort required for data acquisition and preparation, not to mention the train-and-test cycles. Truly understanding the problem and the questions you want to answer is a close second.

But there are other common difficulties and pitfalls for retailers to avoid.

Data bias

Besides having a large enough dataset to train (and improve) your models, your data must accurately represent the audience you’re trying to serve. There are many ways bias can creep into the dataset, or it may be there from the beginning. If this is the case, your model may not give the answer you want or need. You can overcome this by brute force / sheer volume—for example, Amazon has so much retail data that bias virtually disappears. Or you can recognize that a certain amount of bias is inevitable and tune the model using hyperparameters to overcome that bias. Also, depending on the quantitative maturity of your organization, you may use other more sophisticated statistical methods of removing bias.

Insufficient data collection architecture/pipeline

You can provision all the servers with machine learning services you like, but until you provide your model the data it needs to solve the problem, it’s not going to give you the answers you need.

Sometimes the problem a retailer is trying to solve simply requires more data than they are collecting. For example, if you are trying to provide real-time, in-the-moment product recommendations to an online customer, you need a data pipeline to collect clickstream data. This data includes the product they are looking at, how long they’ve spent on the page, how they are scrolling, and much more. To provide this requires building a real-time system to grab, ingest, process and transform it, then inject it into the ML system to spit out the recommendations. In the absence of such a data pipeline/ architecture, even the most excellent ML algorithm cannot effectively yield the quality of answers you desire.

Limiting the data to transactional sales data

Sometimes sales data—which many retail ML models rely on—isn’t enough to produce the accuracy of results you need. If you can augment static sales data with customer behavioral data, you can make better recommendations. For example, if you detect that an online customer went to the same product page three times without clicking “Add to Cart,” perhaps your model can recommend something—a bundle perhaps?—to entice the buyer to take the final step.

Speaking of product recommendations, you can also leverage third-party or local data to enhance the quality and effectiveness of the results. For example, local weather data isn’t an item stored in most order entry systems, but using it could boost sales for a clothing store or sporting good e-commerce site. The lack of these types of “extra” data might not cause a model to fail, but it can limit its effectiveness. But as mentioned earlier, your data pipeline has to acquire such data somehow before the model can leverage them.

Lack of the right personnel and expertise

Platforms like Amazon AWS and Microsoft Azure offer pre-built, “ready to use” ML services, which can relieve the need for an army of software developers to build algorithms from scratch. However, any company that wants to leverage ML has to deal with data engineering and data architecture. On top of that, they may simply lack familiarity with planning, executing, and deploying an ML solution. Hiring a technology partner can fill these gaps during the project, and they can also work with you to “train up” your staff to own and maintain the solution.

Attacking the toughest or most complex problem first

It is tempting for retail IT shops to tackle the most significant problem first to show this new technology’s value. But remember that “new” is the operative word. Start small and start simple until you’ve developed some experience with ML. For example, with its single yes-no answer, fraud detection is far simpler than creating, testing, and deploying a product recommendations solution.

Trying to do too much, too quickly (all at once)

If you pick only one of the four retail-oriented ML solutions we presented earlier, you might be okay. (If you’re new to ML, pick the easiest one first.) But if you try to build, train, test, and integrate all four scenarios simultaneously, you will almost certainly fail. It is simply too giant a leap for most organizations right out of the gate.

Building on a shaky application foundation

If your application architecture isn’t well thought out in the first place, you’re going to run into issues. Integrating ML into a poorly designed piece of software is like building a house on a shaky foundation. Figuring out the source of problems—is it the old code or the new?—is difficult in the best case and impossible (or cost-prohibitive) in the worst. If things do work, you’ll run into performance issues, poor reliability, and a lack of scalability. Building new features—including adding ML—is simpler and more organic with proper application architecture.

The once-and-done assumption

It is dangerous to assume that you are “done” once an ML solution goes into production. You must continuously monitor the model’s output, retune it as needed, and change it altogether if you cannot correct any drift using hyperparameters. Be aware that a significant shift in customer behavior—such as that caused by a global pandemic—can render an otherwise accurate model useless without adjustment or outright change.

After reading, you’re probably thinking: Exactly how does a retailer start benefiting from ML while avoiding the myriad of potential pitfalls? Read more about the benefits here or you can download our white paper: Getting Started With Machine Learning in Retail.