Why you aren't Driving Enough Value from your Forecast?
Fri Nov 10 2023
Author
Noah Xiao
In the intricate dance of supply and demand, a company’s ability to forecast effectively is not just a competitive advantage; it is the rhythm that keeps the business moving smoothly. However, many businesses find themselves out of step, not quite able to synchronize their moves to the ever-changing beat of the customer. The result? Shelves either barren or overflowing, capital tied up in unsold inventory, and opportunities missed. If your demand forecast isn’t yielding the value you expect, it may be time to tune into the deeper issues.
The forecast is not used at its potential
Undiscovered use cases
Many think of forecasting as merely a tool to predict the future, but its use cases are far beyond that. When leveraged effectively, forecasts can carve out significant value in various critical areas.
Pricing planners can use forecasts to gauge price elasticity at future points, enabling more strategic pricing decisions. Supply chain planners can identify which levers to pull to prevent shelf availability and overstocking issues, ensuring that inventory levels are optimised. Senior management can delve into forecast insights, both for the history and future, to understand why company sales didn’t or may not have met expectations and take informed actions to improve performance. These use cases all hinge on the forecast’s ability to learn and capture the intricate relationship between demand and a multitude of factors.
And of course, this requires that the model is designed and built with wisdom and care, ensuring it can accurately support these varied and vital applications.
The isolated use cases
A significant barrier to delivering more value from forecasting is the lack of integration into the broader business process. When forecasting tools stand apart from the daily workflows, their insights remain isolated from the action points they should inform. This disconnect renders the forecasting exercise academic rather than actionable, leading to missed opportunities for operational adjustments and strategic pivots.
For instance, consider a grocery retailer that uses forecasting to predict sales trends for fresh produce. If the forecast indicates a surge in demand for organic apples but isn’t integrated with the store’s supply chain system, the purchasing team remains unaware and doesn’t adjust orders accordingly. The apples sell out faster than expected, customers are left frustrated, and potential sales are lost.
The other common situation is having multiple forecasts across different business functions can create a disjointed view of the future if they conflict with one another. For instance, if the supply chain forecast predicts a high demand while finance projects a downturn, it can lead to confusion and misaligned strategies, such as overstocking or underfunding. This lack of coherence can limit the value of forecasts by causing departments to pull in different directions, undermining the potential for a unified, effective business strategy.
To maximise value, forecasts must be integrated and reconciled, ensuring all departments move forward with a shared understanding and a common goal.
Failing to customise to meet your business's unique needs, strategies, and processes
Customisation is key. A common shortfall in demand forecasting is using a one-size-fits-all model that doesn’t consider the unique aspects of your business. For example, if your retail business operates on an Everyday Low Prices (EDLP) strategy, you need a model tailored to understand how consistent low pricing, as opposed to periodic promotions, affects demand. In such a situation it is important to capture nuances like the impact of a product switching from a Hi-Lo to an EDLP promotional strategy, or the other way round, the impact of how long a product has been on EDLP, and the impact on substitutable and complementary products, to name a few.
By ensuring the model is customised to the intricacies of your business strategy, like promotions, EDLP, weather, Ends and space allocation, Assortment Decisions, Events, and many more, you harness the full power of machine learning — where indeed, the whole is greater than the sum of its parts.
Antiquated model architecture
Underrated architecture
The architecture of a demand forecasting model is crucial, yet often undervalued. It’s not merely about choosing the right algorithms or parameters; it’s about designing a model blueprint that adeptly reflects intricate demand patterns.
A well-architected model balances accuracy with running costs, ensuring that the benefits outweigh the expenses. It also provides the flexibility needed to detect and adapt to previously unseen trends and scenarios. Integration between systems is another critical aspect, allowing for seamless communication and data flow. Additionally, the architecture must strike the right balance between complexity and the ease of debugging and explanation, which is essential for maintaining transparency and trust in the model’s predictions. It should follow sound software design patterns to ensure scalability as the business grows and data volume expands. The model must be robust enough to process large datasets and diverse scenarios, and sensitive enough to detect subtle shifts in consumer behavior. Most importantly, every key design decision should be reviewed to meet high scientific and engineering standards.
Outdated architecture
Many would be surprised, but a significant number of retail software systems powering retailers today still heavily rely solely on traditional statistical models. Statistics are the foundation of data science, and these models have been the backbone of forecasting for decades. However, alone they can suffer under the complexity of today’s fast-evolving retail landscape. By adding in more advanced techniques, combined with refactoring the overall architecture for how these models work together, greater predictive power can be unleashed.
While traditional statistical models form a solid base, they typically do not account for the cross-learning that modern retail operations require. They may struggle to capture the full spectrum of behaviors across different products and regions, which is crucial for identifying overarching trends, as well as ensuring the effect of unseen prices and promotions are accurately estimated. Traditional models also tend to operate on a simplified set of data and assumptions, not fully acknowledging the complex interactions of market forces. This simplicity can lead to blind spots, such as missing the impact of a competitor’s pricing strategy or the ripple effect of one product on the sales of another.
Modern machine learning methods are setting a new standard in demand forecasting. Techniques like global models draw from a wide range of products to ensure unseen promotions or events can still be accurately predicted, while tree-based models dissect decision paths with precision. Deep learning delves into the complexity of nonlinear relationships, unearthing patterns traditional models may miss. Ensembling leverages the collective power of various models to enhance accuracy and robustness, mitigating bias and errors. Meanwhile, sophisticated feature engineering uncovers hidden relationships within the data. When these approaches are seamlessly integrated with robust and scalable architectural design and good business acumen, they unlock new levels of forecasting accuracy and reliability, supporting businesses toward more informed decision-making.
Lack of trust from users
An accurate forecast is nothing if there are no users and winning user trust is crucial to drive business adoption.
The black box
A common factor causing a lack of trust is the poor explainability of machine learning algorithms. Advanced models can appear as “black boxes,” offering little insight into how conclusions are drawn. This opacity can lead users to distrust the results, particularly if the outcomes challenge their intuitions or past experiences. Without transparency and useful tools for explainability, stakeholders may be hesitant to base their decisions on the model’s predictions, which undermines the potential benefits of ML-powered forecasting.
Lack of change management
Moreover, the absence of effective change management exacerbates this issue. Transitioning from the old forecasting methods to new ones isn’t just a technical upgrade; it’s a paradigm shift that affects roles, processes, and decision-making frameworks. Without proper change management, this shift can be disorienting for users, leading to resistance and skepticism. Organisations need to invest in training and communication strategies that help users understand and embrace the new tools.
In essence, to harness the full power of ML in demand forecasting, companies must address these human factors with as much rigor as the technical ones. Enhancing model explainability and investing in comprehensive change management are critical steps in building trust and ensuring that users are not just comfortable with, but also confident in, adopting the new ways of working.
Out-of-date insights
A critical challenge with some demand forecasting models is their inaccessibility to the everyday user. If a model is constrained to only run in batch processes overnight or sometimes even less frequently like weekly or monthly, it creates a lag in accessing fresh insights and actions. This delay can be problematic, especially when swift decision-making is required in response to market changes, or if the user wants to run simulations to understand ‘what-if’ they take some action
Whether this is due to the limited underlying infrastructure, or business misalignment, the challenge can largely undermine the value of the forecast.
Architecting the forecast so that it can support not only batch processes but also real-time interactive simulations, is key to driving the full value from the forecast.
Garbage In, Garbage Out
High-quality data is the lifeblood of accurate forecasting; it must be clean, comprehensive, and current. Ensuring data quality is a pivotal step, as even the most sophisticated models will falter without a solid foundation of reliable data, or without the automated anomaly detections to identify when something is off.
Data quality should be addressed in both the source and the consumption side. A must-have step in the forecasting pipeline is the validation and detection of data quality issues. Early detection and diagnosis help mitigate the risks of misleading forecasts.
Conclusion
In essence, driving value from your demand forecast is not just a technical endeavor; it’s a holistic approach that blends technology, people, and processes. It’s about ensuring that every note of the model’s prediction aligns with the music of your business operations, creating a melody that resonates with accuracy and propels your business forward.
If you’re looking to fine-tune your forecasting strategy and maximize your business outcomes, reach out to us at jahan.ai. We are a team of world-leading AI/ML scientists, engineers, and consultants with deep retail, supply chain, and manufacturing expertise. Drop us a line to get the conversation started.
At jahan.ai, we build end-to-end AI twins for retail and supply chain businesses. We take the concept of a digital twin, a virtual replica of business processes, further with advanced AI, which not only mirrors these processes but also automatically optimises them. In this way, we support businesses in driving efficiency, going beyond their potential.