When making decisions based on electricity generation prediction, particularly for single-site solar and wind installations, precision is of utmost importance. Predictive error, though inevitable, can have a profound impact on the system's effectiveness and profitability. Therefore, having a clear understanding and approach to error measurement can make all the difference. Here, I review a few key error measurement techniques, and highlight an alternative that we use at Amperon for the benefit of our clients: the capacity normalized mean absolute error (cnMAE).
The following are common error measurement techniques:
1. Mean Absolute Percentage Error (MAPE) is often employed for assessing the accuracy of load forecast models. It measures the average absolute percentage difference between predicted and actual values. MAPE is especially useful when evaluating strictly positive values, but stumbles when the values approach zero or go into negatives due to its mathematical instability.
2. Normalized Mean Absolute Error (nMAE) is the average absolute difference between the predicted values and the observed values normalized by the mean value observed. It’s straightforward and easy to comprehend; however, it is sensitive to the average values across the evaluation period. For instance, in solar generation, it can produce exaggerated errors for winter evaluations, when the average generation is minimal, and depreciated errors in summers when generation spikes.
3. Root Mean Square Error (RMSE) calculates the root of the average squared differences between predicted and actual values. This error measurement is particularly useful when accounting for significant prediction misses but is scale-sensitive, and needs calibration when comparing error across different domains or assets.
While these measurements have proven useful, they fall short when dealing with renewable load generation, especially for single-site solar or wind installations. They fail to take into account the system's capacity (maximum power that the system can produce), leaving error values inconsistent across different time periods and assets.
Enter the capacity normalized mean absolute error (cnMAE). The cnMAE scales the forecast error according to the system's capacity. This presents a relative error measure, ensuring that the scale of the system and the values observed during the estimated period do not influence the error score. Simply, cnMAE = MAE/Capacity.
Why cnMAE?
There are several good reasons why cnMAE should be considered the ultimate alternative for renewable generation:
1. Support Comparability: cnMAE enables performance comparison across systems with different capacities. This makes it an ideal metric for diversified portfolios of single-site installations.
2. Prevent Unintended Bias: Contrary to traditional measurements cnMAE maintains an unbiased field by providing capacity-normalized insights.
3. Enhances Planning and Budgeting: By providing a more accurate picture of forecast accuracy relative to the system’s size. This assists operators in planning and budgeting maintenance and operations on a capacity-adjusted basis.
4. Time-insensitive Performance Evaluation: cnMAE is less sensitive to the time period evaluated, making it a more robust estimator of error. This prevents bias in the selection of periods for reporting.
To understand the values expected from each of the error measurements, here’s an example from an Amperon client with a relatively small single solar installation: the park’s capacity is 21 MW, and the mean hourly generation over the last two months of 2023 was almost 1.1 MW. With that, the mean hourly generation over this period when only taking into account hours with any radiance was about 2.9 MW. Computing the error values, when MAPE is only considered for times with any generated load we get:
MAPE - 517%
NMAE - 45%
NMAE using mean value of hours with generation - 17%
RMSE - 0.95 MW
cnMAE - 2.6%
In conclusion, the capacity normalized mean absolute error (cnMAE) offers a considerable advantage over traditional error measures for predicting electricity generation, especially in single-site solar and wind installations. By introducing the system capacity into error measurement, cnMAE alleviates the limitations of traditional methods, thus providing a versatile and equitable error metric for the renewable energy industry.
.png)
.png)
