The severe global economic meltdown has taken the governments, corporations and public by surprise. It is even more surprising, however, that the meltdown has been preceded and accompanied by severe forecasting errors. For example, it was forecast that the crude oil will jump to USD 200 per barrel; now it is forecast that it would go as low as USD 20 per barrel. Similar gross variations between forecasts and actual levels have occurred in the currency exchange rates as well as in various economic parameters. Governments, central banks, economic agencies, consultants and experts have uniformly failed in generating forecasts that could have reasonably mirrored the likely actual levels.
Given the enormous progress made in archiving and analyzing data through information technology and the internet and the equally significant progress made in quantitative and stochastic modeling using mathematics, heuristics and simulations, the failure of forecasting is truly amazing. Perhaps one has to go back to the era of simpler models and their evolution into complex quantitative techniques to understand the reasons.
In the 1960s and 1970s, the forecasting models were based on mathematical models which analyzed a small set of past data to project a future trend. Within this, improvements were made to identify the impact of trends such as seasonality and cyclicality and smoothen the forecasts. As the quantitative capability enhanced thanks to information technology, the forecasting models became more sophisticated and moved into the domain of simulation. Eventually generic as well as domain specific simulation models, covering multiple sectors from FMCG to industrial equipment and from energy to economy, evolved.
Despite the increased sophistication, the failures of forecasting and simulation models have only increased. The reasons are many; some of which can be addressed and others which are difficult to address as discussed below.
Complexity of variables
A model is considered more sophisticated if it considers as many relevant variables as possible. Here lies the essential fallacy of complex models. Each variable is in itself dependent on a series of sub-variables and needs to be forecast based on its own simulation. As one would expect, any decrease in the forecasting efficiency of the primary variables will only decrease the accuracy of the final forecasting outcome in a multiplicative manner. And in several cases, as the variables and sub-variables arise from multiple sectors of the economy and/or multiple geographies, it is nearly impossible to achieve a uniform level of rigor in quantifying the variables and sub-variables. It is therefore necessary to limit the variables that are considered in modeling to only those that would have a significant impact on the outcomes, preferably by using ABC analysis.
While identification of cyclicality and seasonality do help in generating a better model, the techniques do not help when inflection points emerge in respect of certain variables. For example, in today’s scenario purchasing power could have reached an inflection point with reference to the housing sector. The prices of commodities, especially of agricultural products could have a determining impact on the evolution of biofuel space.
Not all inflection points are economic in substance, however. Some inflection points are caused by technology. Ability to redefine the form factor of a device (whether a mobile phone or a laptop computer, or both) could substitute or combine the demand for such products each of which hitherto functioned as independent product-market segments. Ability to combine multiple functionalities in a single product could lead to generation of new product-market structures that are completely different and path-breaking. It is therefore necessary to conduct appropriate qualitative macro-economic evaluation and Delphi type technological analysis prior to embarking on detailed forecasting models.
Inflection points are hard to detect if organizations are mired in managerial dogmas. One is aware of the IBM chief’s observation made decades ago in the context of building of the first computing device that the world would not need more than a handful of computers. Cellular technology collaborators who introduced mobile telephony to India a few years ago felt that India could not support more than a few thousand cellular connections. In both these cases, separated by several decades, the dogmatic beliefs were beaten hollow by the new technology waves that these two products represented. In each case, sheer utility, affordability and proactive market segmentation led to levels of market expansion and penetration, which no forecasting models or leadership judgement could identify.
In today’s complex world where technological and economical factors generate lateral (and sometimes tectonic) shifts in supply side and consumption side factors, the cause-effect relationships are particularly complex to identify. These changing relationships impact the outcomes in several inexplicable ways.
The demand for oil, for example, is not determined merely by economic growth factors such as industrialization, or the infrastructure factors such as roads and automobiles but by the emergence of alternative cleaner energy sources such as biofuels, hydrogen energy, electric energy, solar energy or several combinations thereof. As seen earlier, each sector has its own variables and each variable is influenced by several sub-variables. The more pervasive, and the more multi-component a variable is, the more difficult would it be to define or quantify. Forecasting the overall demand for energy is not a sufficient solution in the alternative because, demand estimation for individual components is essential for sustainable economic and industrial planning.
Amongst all the intriguing aspects of forecasting and simulation, the cause-effect relationships are the most difficult to handle. It is important to aim at scenarios rather than specific outcomes to handle this issue.
The common perception is that forecasting is a statistically supported precise science. This has been a major contributor for the inadequate appreciation and incorrect application of this science. Forecasting is not an exact science. In fact, the subject of forecast errors is a fundamental part of the overall forecasting science. Identification of forecast errors and their sources helps the elimination of systematic deficiencies and refinement of the forecasting model.
Integration of the application domain and forecasting expertise would generate a feedback loop that helps the forecasters and users understand the benefits and limitations of forecasting with accuracy levels that should reasonably be expected. If forecasting through a model and its application in practice are treated as two different disciplines there would be no scope to study forecast errors in a real time framework. Modification of models based on a systematic study of forecast errors is essential to develop robust forecasting models.
Marketing of forecasts
Forecasts were seen in the past by the decision makers as well as lay public as guideposts for an uncertain future. Over time, these have come to be creatively positioned and eagerly lapped up by decision makers to make economic or business moves. Even lay men are increasingly basing their consumer purchase or retail investment decisions on slickly marketed forecasts.
Nowhere is this seen more evidently than in forecasts about corporate performance or stock movements. Over the last few years forecasting of corporate performance has become a “speculative science” built on the foundations of management projections and simple financial planning. The glaring absence of economic and business analysis in such forecasting is covered up under an excessive projection of the so called growth drivers and financial triggers.
At another level, analysts, institutions and agencies use forecasts as a tool to market themselves. It is a matter of concern that despite their apparent data orientation, these bodies rarely compare the forecasts with actual developments and seldom use them for effecting changes in their forecasting methodologies.
In recent years, forecasting has moved into a new domain called derivatives. Based on an underlying transaction, be it related to exports or demand-supply match, calls are taken on the future course of current investments which are, indexed to forecast movements in variables such as currency exchange rates. Scores of companies have lost millions of dollars individually, and billions in the aggregate, in exotic derivative deals. It is with great wisdom and appropriate that Warren Buffet called the modern day derivatives as weapons of mass economic destruction.
Intrinsically, forecasting offers no guarantee in derivatives despite the element of “future projection” that is involved. Movements in foreign exchange rates are subject to unexpected substantial changes and volatility in macro and cross-border economic developments such as demand for dollars, foreign direct investments, balance of trade, industrial recession and so on. Even the most sophisticated experts failed to predict the drastic change in dollar-euro parity or the unrelenting strengthening of Japanese yen. No wonder then that the banks that sold exotic derivative products and the companies which bought them for attractive service fees and profit opportunities respectively are nursing huge losses.
Healing the physicians
Being in the forecasting business does not mean that the wisdom to ensure a future of certainty, and insure an uncertain future accrues through forecasting. We have the case of Goldman Sachs causing a tremor in the global economy with its dramatic forecast of USD 200 per barrel crude not so long ago. While crude is now in a rapid reverse trend towards a low that is just 15 per cent of the forecast, Goldman Sachs could not forecast the downfall of its own institution caused by its erroneous economic and asset forecasts, and related management decisions.
There appears to be a valid case for forecasters to appreciate the science of forecasting in a more temperate and holistic sense, carefully charting all the variable and sub-variable trees, understanding the inflection points and tectonic shifts, delineating cause-effect relationships and recognizing the science of forecast errors. Resisting the temptation to hyper-market forecasts or misusing them to destroy economic value, the forecasting experts have to introspect on the capabilities and the limitations of the science of forecasting, and function as reliable and sensitive guides to considered decision making by the governments, corporations and people at large.
Posted by Dr CB Rao on December 29, 2008