A New Metric for Evaluation of Forecasting Methods : Weighted Absolute and Cumulative Forecast Error

Aggregate Production Planning determines levels of production, human resources, inventory to maximize company's profits and fulfill customer's demands based on demand forecasts. Since performance of aggregate production planning heavily depends on accuracy of given forecasting demands, choosing an accurate forecasting method should be antecedent for achieving a good aggregate production planning. Generally, typical forecasting error metrics such as MSE (Mean Squared Error), MAD (Mean Absolute Deviation), MAPE (Mean Absolute Percentage Error), and CFE (Cumulated Forecast Error) are utilized to choose a proper forecasting method for an aggregate production planning. However, these metrics are designed only to measure a difference between real and forecast demands and they are not able to consider any results such as increasing cost or decreasing profit caused by forecasting error. Consequently, the traditional metrics fail to give enough explanation to select a good forecasting method in aggregate production planning. To overcome this limitation of typical metrics for forecasting method this study suggests a new metric, WACFE (Weighted Absolute and Cumulative Forecast Error), to evaluate forecasting methods. Basically, the WACFE is designed to consider not only forecasting errors but also costs which the errors might cause in for Aggregate Production Planning. The WACFE is a product sum of cumulative forecasting error and weight factors for backorder and inventory costs. We demonstrate the effectiveness of the proposed metric by conducting intensive experiments with demand data sets from M3-competition. Finally, we showed that the WACFE provides a higher correlation with the total cost than other metrics and, consequently, is a better performance in selection of forecasting methods for aggregate production planning.

수요 예측 평가를 위한 가중절대누적오차지표의 개발