How to validate PV fleet forecast accuracy when you cannot directly measure it

The capacity of behind-the-meter PV systems is growing steadily. Figure 1 illustrates that there is over 5 GW of PV in California alone. The number is continuing to increase.
Graph showing evolution of behind-the-meter installed PV capacity

The result is that behind-the-meter PV fleet forecasts are becoming increasingly important for utility operations. The impact of behind-the-meter PV production has been well documented through the California Independent System Operator’s “Duck Curve.”

Graph of predicted California ISO load changes

The result is that independent system operators (ISOs) and utilities need accurate PV fleet forecasts. Day-ahead forecasts are needed for utility load forecasts that underlie electricity markets and determine power plants’ commitment. Short-term forecasts (minutes to hours) are important for spot market and load following operations, including demand response and spinning reserves management.

PV fleets include both utility-scale power plants and behind-the-meter PV systems. The utility-scale plants are precisely monitored. Data from these plants can be used to evaluate the accuracy of solar forecasts at specific locations. PV fleets also include a large number of behind-the-meter PV systems.

These behind-the-meter systems are distributed throughout utility service territories. In many cases, they are not directly monitored. The result is that their aggregated output is unknown and cannot be used directly for solar forecast validation. So how can behind-the-meter PV fleet forecasts be validated?

Map of distribution and capacity of behind-the-meter PV systems in California

How to validate behind-the-meter PV fleet forecasts

The accuracy of PV production forecasts is primarily driven by the accuracy of solar irradiance forecasts. This is because irradiance is the main determinant of PV production. PV output can be determined from irradiance if PV specs are known, even if PV production data cannot be measured. Other determinants of PV output such as temperature and wind speed are respectively second and third order effects. Temperature also can be dependably forecasted.

Historical satellite-derived irradiance data could be amenable for dispersed PV fleet forecast validations. This is because satellite data are time- and site-specific. The data are also available at high resolution throughout entire regions. The key question is therefore whether historical satellite-derived irradiance data constitute a quantitatively acceptable benchmark of fleet forecast accuracy.

Recent evidence from an article by Perez et al., indicates that yes, historical satellite-derived irradiance data can be applied reliably to evaluate forecast model uncertainty.1

Figure 4 compares forecast validation results obtained at point locations when using actual irradiance measurements and satellite data as a benchmark. The satellite data consist of SolarAnywhere® Version 3.0 data. The forecast validation metric is the percent root mean square error (RMSE). The RMSE is evaluated for several forecast models at different point locations (Eastern and Western US), and for forecast time horizons ranging from one-hour ahead to one-day ahead. Aggregate results over all locations and time horizons are shown.

The figure shows that comparing forecast data to historical satellite and ground data have nearly identical results. This suggests that satellite-derived irradiance data can be used to assess forecasts. More importantly, because satellite-derived irradiance accuracy is largely independent of location, the data can be reliably applied to assess the performance of forecast models over regions of arbitrary size and shapes–for example, substation territories, utility territories or ISO balancing areas–where distributed PV fleets operate.

In addition, with satellite-derived irradiance data, one does not need to deal with potential inaccuracies in data collected using ground-based instruments. These instruments must be carefully calibrated and well-maintained to guarantee accuracy.

Graph Comparing single-point RMSE statistics for different solar forecast models across a range of time horizons and locations as benchmarked with ground measurements and satellite data.

Key take away or action

Next time you plan to validate PV fleet forecasts, consider using historical fleet data based on satellite-derived irradiance as a benchmark. This will provide a reliable way to evaluate the performance of PV fleet forecasts models.

1 Perez, R., J. Schlemmer, K. Hemker, Jr., S. Kivalov, A. Kankiewick and J. Dise, (2016): “Solar Energy Forecast Validation for Extended Areas & Economic Impact of Forecast Accuracy.” Presented at 43rd IEEE PV Specialists Conference, 2016.