Evaluation of conformal-based probabilistic forecasting methods for short-term wind speed forecasting

We apply Conformal Predictive Distribution Systems (CPDS) and a non-exchangeable version of the traditional Conformal Prediction (NECP) method to short-term wind speed forecasting to generate probabilistic forecasts. These are compared to the more traditional Quantile Regression Forest (QRF) method. A short-term forecast is available from a few hours before the forecasted time period and is only extended a couple days into the future. The methods are supplied ensemble forecasts as input and additionally the Conformal methods are supplied with post-processed point forecasts for generating the probability distributions. In the NECP case we propose a method of producing probability distributions by creating sequentially larger prediction intervals. The methods are compared through a teaching schedule, to mimic a real-world setting. For each model update in the teaching schedule a grid-search approach is applied to select each method’s optimal hyperparameters, respectively. The methods are tested out of the box with tweaks to few hyperparameters. We also introduce a normalized nonconformity score and use it with the conformal method that handles data that violates the exchangeability assumption. The resulting probability distributions are compared to actual wind measurements through Continuous Ranked Probability Scores (CRPS) as well as their validity and efficiency of certain prediction intervals. Our results suggest that the conformal based methods, with the pre-trained underlying model, produce slightly more conservative but more efficient probability distributions than QRF at a lower computational cost. We further propose how the conformal-based methods could be improved for the application to real-world scenarios.