SIParCS 2022 - Justin Willson

Neural Network for Winter Weather Precipitation Type Prediction

Recorded Talk

Winter weather events can be dangerous and are often difficult to predict. The goal of this project was to leverage machine learning techniques to predict four winter weather precipitation types (p-types): rain, snow, ice pellets, and freezing rain. Two artificial neural networks were developed. The models received data from the Rapid Refresh (RAP) model as input and predicted on data from ASOS, a system of automated weather stations, or mPING, a mobile application where users submit weather reports. Hyperparameter optimization was conducted using Earth Computing Hyperparameter Optimization (ECHO), a custom AIML group package. After hyperparameter optimization, the ASOS model predicted rain, snow and freezing rain with accuracies greater than 80%. The mPING model predicted rain and snow with accuracies at or above 75%. Freezing rain accuracy in mPING was significantly lower. Both models were unable to predict ice pellets. The expected calibration error (ECE), which is the sum of the deviations of the accuracy from the confidence of the predictions, was also analyzed. It indicates whether a model is overconfident, underconfident, or calibrated. In ASOS, ECE for rain, snow, and freezing rain was small, suggesting a calibrated model, but ECE for ice pellets was substantially larger. In the mPING model, all p-types had a large ECE. The accuracy of ice pellet predictions decreased significantly when the confidence approached 1 in both models, opposite of the expected relationship. Future work will involve furthering the development of a neural network capable of estimating its own evidential uncertainty.

Mentors: John Schreck, David John Gagne II

Slides and poster