Empirical Strikes Back

There is a tendency in supply chain to assume that a more sophisticated calculation must produce a better answer. In purely mathematical terms that may well be true, but supply chains are not run in purely mathematical terms. They are run by people who have to understand the outputs they are being given, make decisions from them and, occasionally, recognise when those outputs do not make sense.

This creates a trade-off that is sometimes overlooked. A sophisticated statistical model might, theoretically, get 99% of the way to the best possible answer, while a much simpler empirical calculation gets 98% of the way there. On mathematical performance alone, the first model wins. But if very few people understand how the 99% answer has been calculated, while planners, managers and other functions can understand the 98% answer, the comparison becomes rather less straightforward.

Understanding has practical value. If people can see how an answer has been derived, they are better able to challenge it, explain it and recognise when the circumstances behind it have changed. The calculation becomes part of the decision-making process rather than something that simply produces a number which everyone is expected to accept.

This is one of the reasons empirical methods can be particularly useful in forecasting and inventory management. They are not necessarily the most statistically elegant way of approaching every problem, and nor should they replace good statistical analysis where that analysis provides genuine value. Their attraction is that they can often get surprisingly close to the answer using relationships that can be observed, tested against what actually happened and, importantly, understood by the people expected to use them.

Take an inventory calculation. It is possible to create increasingly sophisticated ways of representing uncertainty in demand and lead time, and there are circumstances where doing so genuinely improves the decision. Equally, it is possible to look empirically at the demand and lead-time variation the business has actually experienced, test what different stock policies would have done historically and establish whether a more sophisticated calculation materially changes the result. If greater sophistication materially improves the decision, there is a good case for using it. If the improvement is marginal, there may not be.

The distinction matters because explainability is useful in its own right, even though it rarely appears as a measure when models are compared. A calculation that planners understand can be questioned when demand behaviour changes, when lead times move, when a promotion distorts the history or when the commercial context makes the calculated answer inappropriate. None of that means allowing gut feel to replace analysis. It means allowing informed judgement to be applied to something people can actually understand.

AI changes the equation

Historically there has been something of a natural barrier to unnecessary mathematical complexity: somebody had to be capable of developing it. Advanced models generally required people who understood the mathematics well enough to build them, which at least created some connection between the sophistication of the calculation and the expertise behind it.

AI is beginning to remove that barrier.

It is increasingly possible to use AI to develop a mathematical approach, algorithm or piece of code that goes considerably beyond the mathematical understanding of the person asking for it. In many respects that is a remarkable opportunity. AI-enabled supply chain systems can make approaches that would previously have required specialist development capability much more accessible.

The danger is that the ability to create complexity becomes confused with the ability to justify it.

AI can produce something mathematically impressive and turn it into working code. But the fact that a model can be built does not establish that the mathematics is appropriate to the supply chain decision, that its assumptions are sensible or that a marginal improvement in the answer is worth the loss of understanding.

That could leave businesses in a rather odd position. They may have access to the most mathematically sophisticated planning tools they have ever had while progressively fewer of the people using them understand why they produce the answers they do. The results may look increasingly authoritative at exactly the point at which the ability to challenge them is becoming weaker.

Start with the requirement

The answer is not to resist sophisticated modelling or AI. It is to start somewhere different.

The starting point should be the decision that needs to be made, the degree of accuracy that genuinely matters and the operational consequences of getting it wrong. An approach can then be built around that requirement and tested against what has actually happened. If a relatively straightforward empirical model performs well, that provides a useful baseline against which additional sophistication can be judged.

Complexity should then have to earn its place.

If a more sophisticated statistical approach materially improves forecasting, inventory, service or another decision that matters to the business, it is justified. If the improvement is marginal, what is being given up in return should also be considered. Moving from 98% to 99% may look like progress when viewed through the mathematics, but it is not necessarily progress if the organisation moves from understanding the calculation to trusting something it can no longer properly interrogate.

That distinction will become increasingly important as AI becomes more capable. The question will no longer be whether a more sophisticated model can be built. In many cases, it almost certainly can. The more useful question is whether the additional sophistication produces a better decision.

Sometimes it will. Sometimes a pragmatic empirical approach will get sufficiently close while remaining explainable, testable and useful to the people actually making the decision.

In practice, 98% that people understand may prove rather more valuable than 99% that they don’t.