NWP Satellite Validation

The goal of this project is to develop an automated ranking method that can be used to quickly assess the accuracy of High Resolution Rapid Refresh (HRRR) model forecasts through comparison of observed and simulated Geostationary Operational Environmental Satellite (GOES) infrared brightness temperatures. The simulated brightness temperatures provide detailed information about the spatial distribution of clouds and water vapor across the continental United States. Given the hourly update cycle of the HRRR model, it is difficult for forecasters to routinely assess the accuracy of each HRRR model forecast cycle because of the large data volume. To help remedy this problem, the satellite-based model analysis tool developed during this project provides forecasters a new method to quickly assess the accuracy of the overlapping HRRR forecast cycles at the current time. A more complete description of the project methodology exists below. If you have questions or comments related to the project, please contact group researchers ( Jason Otkin, Justin Sieglaff, Sarah Griffin, and Lee Cronce ) at the Cooperative Institute for Meteorological Satellite Studies (CIMSS) at the University of Wisconsin-Madison.

The accuracy of the model forecasts is being assessed through comparison of simulated and observed brightness temperatures from two GOES infrared bands. These include the 6.7 µm “water vapor” channel that is sensitive to cloud and water vapor in the middle and upper troposphere (maybe mention approximate height in meters) and the 11 µm infrared “window” channel that is sensitive to clouds when they are present but will see the surface when skies are clear. Simulated HRRR BTs are being generated for each forecast cycle at the Earth System Research Laboratory (ESRL) in Boulder, CO, and typically arrive at CIMSS about 1.5 hours after the forecast cycle begins. The simulated and observed datasets are remapped to a common grid and then statistics are computed using data from the entire model domain and also for specific regions corresponding to those used on the Storm Prediction Center Mesoscale Analysis webpage. Currently, three metrics are being used to assess the accuracy of each HRRR forecasts cycles, including the Root Mean Square Error (RMSE), Bias, and Mean Absolute Error (MAE).

The simulated and observed GOES brightness temperature imagery and evaluation statistics are displayed on this webpage. The webpage updates with each new HRRR model forecast cycle (which is run 24 times per day) and shows imagery and statistics at hourly intervals during the 0-15 hour forecast cycle. Because the HRRR model data is available approximately 1.5 hours after the start of each forecast cycle, the most recent HRRR model forecast available on the webpage is typically a 2-hour old forecast. Additional short-term disruptions in the experimental HRRR model forecasts may also occur.

The simulated HRRR satellite brightness temperatures became available to UW-CIMSS in April 2015. During the next two years, we plan to explore the use of object-based and neighborhood verification methods and the most useful of those metrics will be added to the webpage. In addition, as the project progresses, we will also use the long-term data record to assess the model accuracy with respect to the diurnal cycle, thermodynamic environment, etc.

Please see the Tutorial section for a brief tour of how to use the data viewer and interpret the statistics.