Avoid The 7 Most Common Mistakes in Retail and CPG Demand Forecasting
Tue Sep 24 2024
Author
Sumith Matharage
Demand forecasting for retail and CPG (Consumer Packaged Goods) is a complex yet critical function that can significantly impact an organisation’s operational efficiency and profitability. Accurate forecasting enables better inventory management, reducing both stockouts and overstock situations, optimising supply chain operations, and minimising waste. This, in turn, helps streamline production schedules, improve cash flow, and enhance customer satisfaction by ensuring the right products are available at the right time, ultimately driving cost savings and operational effectiveness.
However, many businesses face numerous challenges in transforming their demand forecasting efforts from basic estimations to actionable, accurate predictions that drive value. The journey from simply wanting a forecast to producing a highly effective, production-ready model often encounters obstacles. As a result, organisations frequently struggle to achieve the full potential of their demand forecasting processes, leaving gaps in their ability to make informed decisions that respond to market dynamics such as planning for upcoming events or responding to changing weather conditions.
In this article, we will explore the top 7 mistakes that can undermine the effectiveness of demand forecasting in retail and CPG, and how to overcome them to maximise success.
Top 7 Mistakes
1. The one-size-fits-all approach
- What it means:Dairy products, fashion items, and candies each follow distinct patterns of customer behavior. There is no single machine learning model or architecture that can accurately capture the nuances of all these product categories. Applying a one-size-fits-all approach often results in inaccurate forecasts and inefficient planning and execution.
- How to avoid:Design your forecasting system with flexibility to configure features, algorithms, and behaviors based on product categories, stores, and sales channels, among other variables. Incorporate automated processes that explore different configurations to identify the optimal model for each specific problem, ensuring greater accuracy and effectiveness
2. Too many forecasts
- What it means:Some organisations, along with many forecasting SaaS providers, generate multiple forecasts for different functions - such as replenishment, workforce planning, price optimisation, and budgeting. This can lead to inconsistencies, with teams spending more time debating which forecast is correct rather than focusing on improving the outcomes
- How to avoid:Implement a single, unified core forecast for demand, and then apply specific transformations on top of it (e.g., converting quantity demanded into revenue or profit). This ensures that the entire organisation is aligned around one consistent forecast, allowing teams to concentrate on improving performance rather than reconciling different numbers. It also ensures that key features of the forecast, like holiday demand spikes, are addressed correctly, once and for all.
3. Lack of adoption
- What it means:No matter how accurate a forecast is, if people don’t use it or trust it, it will fail to deliver its potential value. Many organisations overlook the importance of change management in the implementation of new forecasting systems, undermining the effort put into development. Without clear communication, proper training, and a focus on the benefits of the forecast, even the most advanced models risk being underutilised or mistrusted, ultimately failing to achieve their full impact.
- How to avoid:Prioritise change management from the outset. Understand user needs, potential barriers, and current workflows to design programs that build trust and confidence in the forecast. Ensure that change managers are well-versed in how machine learning initiatives differ from other projects, allowing them to tailor the rollout and adoption strategies accordingly. This approach will foster smoother integration and higher user engagement.
4. Outdated algorithms
- What it means:Many SaaS forecasting solutions rely on traditional algorithms like ARIMA and Exponential Smoothing. While useful in some cases, these models struggle to capture the complex interactions between various demand drivers, limiting their ability to provide highly accurate and insightful forecasts compared to more advanced methods.
- How to avoid:Embrace modern algorithms such as gradient boosting and deep learning, which offer greater accuracy and the ability to model complex interactions between various demand factors. These advanced algorithms not only improve forecast accuracy, but also enable simulations for previously unseen events, such as new product launches or promotions, providing businesses with deeper insights and more robust planning capabilities.
5. Not using the full potential of your data
- What it means:Many forecasting solutions rely on basic features such as "Price" or "Promotional Saving %." This simplistic approach often overlooks valuable information like relative pricing, pricing trends, and historical price changes. As a result, the system fails to capture the full impact of these factors, limiting the accuracy of the forecast.
- How to avoid:Develop a rich set of feature engineering to extract more value from each data point and better reflect the business context. For example, even a single factor like price can generate hundreds of distinct useful features, leading to a 1-5% improvement in forecast accuracy.
6. Cost of ownership
- What it means:Organisations often underestimate the complexity of the forecasting process, especially when creating separate forecasts for different use cases. This approach leads to duplicated IT infrastructure, increased computing costs, and repeated efforts to maintain and operationalise forecasts, resulting in unnecessary redundancy and inefficiency.
- How to avoid:Strive for a unified forecasting system to minimise duplication. Additionally, develop a scalable framework to streamline the build where adding a new business unit incurs only incremental costs, rather than replicating the full expense repeatedly. This reduces operational complexity and maximises cost efficiency.
7. Shortage of retail AI/ML experts
- What it means:Building a team of data scientists who not only grasp the complexities of forecasting algorithms and architectures but, more importantly, possess a deep understanding of retail dynamics and consumer behavior is a major challenge. Finding experts who excel in both areas is crucial yet often difficult, which can lead to potential gaps in the effectiveness of forecasting systems.
- How to avoid:Foster a strong data science culture that attracts top talent and provides clear career development pathways for growth. Additionally, consider seeking external support from experts with both retail/CPG knowledge and advanced AI expertise to accelerate time-to-value and enhance forecasting capabilities.
While these challenges represent the top obstacles in demand forecasting, there are additional considerations such as establishing robust MLOps practices, aligning stakeholders on priorities, reusing data science assets, and implementing effective tooling. Despite the complexities involved, these challenges are surmountable, and the effort invested is well worth the rewards. Achieving an accurate and responsive view of customer demand can unlock significant value and drive substantial improvements in operational efficiency and strategic decision-making.
If you’d like to know more about demand forecasting or how jahan.ai can help you, reach out to us on info@jahan.ai and ask for our Demand Forecasting Handbook. Learn all about setting up your organisation with the best forecasting solution for you.
At jahan.ai, we build end-to-end AI twins for retail and supply chain businesses. We take the concept of a digital twin, a virtual replica of business processes, further with advanced AI, which not only mirrors these processes but also automatically optimises them. In this way, we support businesses in driving efficiency, going beyond their potential.