Preparing forecasts using data that contain one or more unusually large or small demand periods can be challenging. Depending on your forecasting approach, these “outliers” can have a significant impact on your forecasts. This article surveys three different approaches to forecasting data containing unusual demand periods, discusses the pros and cons of each and recommends when it is best to use each approach.
Option #1: Use a forecasting method that models what happened
Unusual demand can be caused by events of which you have knowledge (e.g., promotions, one-time orders, strikes, catastrophes, etc.) or can be caused by events of which you have no knowledge (i.e., you know that the point is unusual, but you don’t know why). If you have knowledge of the events that created the unusual demand, you should consider using a forecasting method that explicitly models these events.
Event models are an extension of exponential smoothing that are particularly well suited to this task. They are easy to build and lend themselves well to automation. Another option is dynamic regression.
Unlike time series methods, which base the forecasts solely on the items’ past history, event models and dynamic regression are causal models, which allow you to bring in additional information such as promotional schedules, the timing of business interruptions and (in the case of dynamic regression) explanatory variables.
By capturing the response to the events as part of the overall forecasting model these techniques often improve the accuracy of the forecasts and provide insights into the impact of the events.
Recommendation:
In instances where the causes of the unusual demand are known, you should consider using a forecasting method that explicitly models the events.
Option #2: Separate the demand streams
At times, unusual data points can occur as a result of the demand stream being driven by distinctly different sources. In these instances, it may be useful to separate a time series into two different demand streams and forecast them separately. Consider the following three examples.
Example A: A pharmaceutical company’s demand for a given drug consists of both prescription fills (sales) and free goods (e.g., samples distributed free of charge to physicians). The timing of the distribution of free goods introduces outliers in the time series representing total demand. Separating the demand streams yields an outlier-free prescription fills series and allows different forecasting approaches to be used for each series—which is appropriate since the drivers generating the demand are different for each.
Example B: A manufacturing company’s demand normally consists of orders from its distributors. In response to an unusual event, the government places a large one-time order that introduces a significant outlier into the demand series, but does not impact base demand from the distributors. Separating the demand streams yields an outlier-free distributor demand series and allows the forecast for the government’s demand series to be simply set to zero.
Example C: A food and beverage company sells its products from both store shelves and promotional displays (e.g., end caps, point-of-sale displays, etc.). It has access to the two separate demand streams. Although it is tempting to forecast these two series separately, it may not be the best approach. Although the promotional displays will increase total demand, they will also cannibalize base demand. In this example it may be better to forecast total demand using a forecasting method that can accommodate the promotions (e.g., event models, regression, etc.).
Recommendations:
1. Separating the demand streams should only be considered when you understand the different sources of demand that are introducing the outliers.
2. If the demand streams can be separated in a “surgically-clean” manner, you should consider separating the demand streams and forecasting them separately.
3. In cases where the demand streams cannot be cleanly separated, you are often better off working with a single time series.
Option #3: Adjust the history
A simple solution to lessen the impact of an unusual data point is to replace the observation with a more typical value prior to generating the forecasts. This process is often referred to as Outlier Correction. Many forecasting solutions, including Forecast Pro, offer automated procedures for detecting outliers and “correcting” the history prior to forecasting. Correcting the history for a severe outlier will often improve the forecast; however, if the outlier is not truly severe, correcting for it may do more harm than good. When you correct an outlier, you are rewriting the history to be smoother than it actually was and this will change the forecasts and narrow the confidence limits. This will result in poor forecasts and unrealistic confidence limits when the correction was not necessary.
Forecast Pro TRAC screenshot showing both an outlier report and a graph displaying a “corrected” outlier.
Recommendations:
1. If the cause of an outlier is known, alternative approaches (such as option #1 and #2 above) should be considered prior to resorting to outlier correction.
2. Outlier correction should be performed sparingly. Using an automated detection algorithm to identify potential candidates for correction is very useful; however, the detected outliers should ideally be individually reviewed by the forecaster to determine whether a correction is appropriate.
3. In cases where an automated outlier detection and correction procedure must be used, (for example if the sheer number of forecasts to be generated precludes human review) then the thresholds for identifying and correcting an outlier should be set very high. Ideally the thresholds would be calibrated empirically by experimenting with a subset of the data.
Summary
Ignoring unusual demand periods in your data often leads to poor forecasts. The best approach to forecasting data containing unusual demand periods depends on the nature of the outliers and the resources of the forecaster. In this article, we have discussed three approaches—modeling what happened, separating the demand streams and correcting outliers—which can be used for creating forecasts based on data containing unusual demand periods.
If you would like to see how Forecast Pro TRAC can help your organization handle outliers and address other forecasting challenges, schedule a personalized Web-based demo.
About the author:
Eric Stellwagen is the co-founder of Business Forecast Systems, Inc and the co-author of the Forecast Pro software product line. He has consulted widely in the area of practical business forecasting and spends 20-30 days a year presenting workshops on the subject. He has worked with many leading firms including Coca-Cola, Procter & Gamble, Merck, Blue Cross Blue Shield, Nabisco, Owens-Corning and Verizon. He has presented seminars and workshops under the aegis’s of many groups including the Institute for Professional Education, the American Production and Inventory Control Society, the University of Wisconsin, the Institute for Business Forecasting, the World Research Group, the International Institute of Research, the Electric Power Research Institute, the International Communications Forecasting Association and the International Institute of Forecasters. He is currently serving on the board of directors of the International Institute of Forecasters and on the practitioner advisory board of Foresight: The International Journal of Applied Forecasting.
