When it comes to forecasting and estimating data, accuracy measures play a vital role in evaluating the performance of predictive models. One commonly used error measure is the Mean Absolute Percentage Error (MAPE). However, there are scenarios where the Normalized Mean Absolute Error (NMAE) outshines MAPE as a more appropriate metric. Here I will explore why NMAE is preferred over MAPE, particularly when dealing with the estimation of data that can be negative or close to zero, which is the case when forecasting electricity load demand, where the rising popularity of rooftop solar usage can drive the demand to be close to zero or even negative at times.
Understanding the MAPE Limitations
Mean Absolute Percentage Error (MAPE) is a commonly employed metric for evaluating the accuracy of forecasting models. It measures the average absolute percentage difference between predicted and actual values. However, MAPE has certain limitations that make it less suitable in specific scenarios.
- Division by Zero: MAPE is not well-suited for situations where the actual values are close to zero. When the denominator in the percentage calculation is small or zero, it can lead to infinite or extremely high values, rendering the MAPE meaningless. This issue arises frequently when estimating data that can approach zero or even turn negative.
- Asymmetric Impact of Errors: MAPE treats over-estimation and under-estimation errors symmetrically. However, in certain domains, such as energy forecasting, over-estimation and under-estimation errors can have significantly different consequences. This imbalance in the impact of errors makes MAPE less suitable for such scenarios.
Introducing NMAE as an Alternative
Normalized Mean Absolute Error (NMAE) is an alternative metric that overcomes the limitations of MAPE in situations involving data that can be negative or close to zero. NMAE normalizes the error by dividing it by the range of the actual values, providing a more balanced measure of accuracy. Let's explore why NMAE is particularly useful in forecasting electricity load demand, where rooftop solar usage can drive demand close to zero or negative.
Electricity load demand forecasting is crucial for grid operators to ensure a stable and reliable power supply and for other participants in the electricity market to optimally reduce their risk exposure to changing market behavior. With the increasing adoption of rooftop solar panels, some regions experience situations where electricity demand can drop close to zero or even become negative when solar generation exceeds local consumption. This poses a challenge for traditional accuracy metrics like MAPE.
By employing NMAE instead of MAPE, the accuracy of load demand forecasting can be evaluated more effectively. NMAE normalizes the absolute error by the range of the actual values, accommodating situations where demand approaches zero or becomes negative. This normalization provides a better reflection of the forecasting model's performance, even during periods of low or negative demand.
Example Scenario
Let's consider a scenario where an electricity supplier uses electricity load demand forecast for a specific zone with significant rooftop solar penetration. On a sunny day, the solar generation can be substantial, potentially driving the load demand close to zero or even resulting in negative values. In such cases, MAPE would encounter the division-by-zero problem and fail to provide a meaningful accuracy measure.
However, NMAE would handle this situation with ease. By normalizing the absolute error based on the range of actual values, NMAE would accurately evaluate the forecasting model's performance, even when dealing with near-zero or negative demand.
.png)
.png)
