Part 2. MM4XL Tools > 1. Strategic Tools > Forecast Manager > 3. Technicalities > Opening the black box of Forecast Manager > Reliability and accuracy measures

Forecast Manager

Reliability and accuracy measures

Forecast Manager can print several indices useful for evaluating the accuracy of curve fit, the overall fit reliability, and the consistency of the quality of fitted values.

Accuracy is measured with the following indices:

 Index Formula Meaning MAD Mean Absolute Deviation Source: Ragsdale It assigns equal weight to all errors, so it is easy to compare, but it is difficult to interpret its scale of measurement. It fails to take under- and over-representation into account. Meaning: the smaller the more accurate the fit. MAPE Mean Absolute % Deviation Source: Ragsdale Like MAD but in percentage, so it overcomes the scale of measurement problem. Meaning: the smaller the more accurate the fit. MSE Mean Square Error Source: Ragsdale It minimizes the occurrence of a major error. But penalizes techniques that produce only a small number of large errors, perhaps at start. Meaning: the smaller the more accurate the fit. RMSE Root Mean Square Error Source: Ragsdale Like MSE. It is sometimes preferred to MSE because it is easier to interpret, for it has the same unit of measurement as the actual series. Meaning: the smaller the more accurate the fit. R Squared Coefficient of Determination Source: Jarrett It represents the part of actual data explained with the fitted data. This is also the square of the correlation between actual and fitted data. Meaning: the larger the more accurate the fit. Overall reliability is measured with the following indices:

 Index Formula Meaning U-statistics Source: Jarrett  Variance actual series Variance fitted series If the method forecasts perfectly then U=0. If generating erroneous forecasting U=1 Durbin-Watson coefficient Source: Jarrett DW tests the correlation of residuals (error terms). If DW > U, conclude Ho. If DW < L, conclude Ha. Ho: error terms are independent. Ha: error terms are correlated. U = Upper level and L = Lower level can be found in DW tables. Sample size, significance level, and number of independent variables are needed to locate the appropriate values.

Consistency of fitting performance is measured with the following indices:

 Index Formula Meaning Turning-Point Performance Source: Bails False Signals: Missing Signals: A turning-point is a change in the direction of either the curve of actual data or of predicted values. False signals occur when a change in the direction of the fitted data is not due to a change in the direction of the actual data. Missed signals occur when a change in the direction of the actual data is not paired with a change in the direction of the fitted data. Consistency of Performance Source: Jarrett Level set by the user and comprised in the range 0-100% of fitted data. It counts the number of fitted values lying above and below the boundary level set by the user, typically 1-10% of fitted values.
Price: 238.00
 Your vote  1 2 3 4 5 Email: Gender: M F Age: --Select-- below 2525-3536-4546-5555+ Position: --Select--ManagerConsultantProfessorStudentOther Department: --Select-- MarketingSalesFinanceITOther Comment: