Observing System Simulation Experiments (OSSE) for Geostationary Satellite Retrievals


  1. Introduction
  2. OSSE Design
  3. Preliminary Results
  4. Pilot Study

1. Introduction

The Cooperative Institute for Meteorological Satellite Studies (CIMSS) has designed and implemented a software system that will allow investigators to quantitatively assess the value of an atmospheric observing system to operational mesoscale numerical forecasts. The system uses the construct of the Observing System Experiment (OSE), which allows for the objective assessment and comparison of existing operational observing systems in a controlled software environment. Observations that represent the characteristics of the observing system being tested are synthesized from forecasts generated by a sophisticated numerical prediction model that is independent of the operational assimilation system being used. These forecasts are referred to as the "true", or "nature" atmosphere. The forecast model used to produce the "nature" atmosphere must have a known performance history, and must be calibrated against reality. The observations that are synthesized from the "nature" atmosphere must mimic, as close as possible, those observations from the real observing system that is being assessed. The OSE system described here is unique in that it attempts to measure the impact of a designated observing system over a limited area. Previous applications of the OSE construct have been performed using global prediction models, which are not influenced by pre-specified lateral boundary conditions. Every attempt was made to isolate the assimilating forecast model from the influence of pre-specified lateral boundaries.

The software system described above can be extended to include proposed, next-generation, observing systems. These assessments are performed as Observing System Simulation Experiments (OSSEs). Observations for these experiments are simulated using projected instrument characteristics. In this context future-observing systems can be compared to existing systems to determine if there is value added in the form of improved forecast skill.

CIMSS is also performing additional value assessments on proposed observing systems using information content theory. The potential for an observation to contribute to an operational assimilation system can be quantified by computing how much the synthesized observations reduce the "information entropy" of the assimilation system. If the "information entropy" reduction (IER) is minimal, the observations are not adding information to the system. If the IER is significant, the observations are contributing information.

A "Pilot Experiment" is now being performed at CIMSS. The hypothesis states: Information from a spaced-based interferometer, with the ability to measure radiation at high spectral resolution, will significantly improve the accuracy of numerical weather forecasts as compared to using observations with limited spectral resolution. The interferometer system was simulated from a geostationary orbit. Its ability to add information to an operational numerical forecast system is being compared to the current operational filter-wheel radiometer onboard the Geostationary Operational Environmental Satellite (GOES). Initially, retrieved temperature and moisture profiles will be assimilated. They are derived by superimposing typical error structures on the "true" profiles generated by the "nature" run. This will be done for both geostationary instruments. Eventually, synthetic radiances will be computed, using accurate forward radiative transfer models, and assimilated.

2.OSSE Design

An OSSE can be subdivided into four basic steps: 1) Generate a "nature" atmosphere; 2) Compute synthetic observations; 3) Assimilate the synthetic observations; and 4) Assess the impact on the resulting forecast. Each step is performed with the goal of minimizing any external influences, which may compromise the value of the synthesized observations, the assimilation process, or the results of the numerical forecasts.

a. Generate a "nature" atmosphere

The University of Wisconsin, Non-hydrostatic Mesoscale Model (UW-NMS) has been selected to create a four dimensional atmosphere consisting of the model dependent variables. These data define "truth" for the OSSE. The UW-NMS was configured with a horizontal resolution of 60 km for the Pilot Experiment. Ideally, the "nature" atmosphere should be two to four times the resolution of the simulated observing system (e.g. 20-km resolution for simulating 80-km data). The horizontal domain is chosen to be as large as practical to isolate the influence of the pre-defined lateral boundary conditions. The model vertical resolution is chosen to be a minimum of two-times the resolution of the observing system to be simulated. For the pilot study the UW-NMS will be initialized with fields from the National Centers for Environmental Prediction's (NCEP) Eta forecast model, which is available on the 40 km AWIPS 212 grid. A 12hr forecast will be generated to allow the model to "spin up". Upon completion of the 12hr integration, forecast fields will be saved at hourly intervals, for the next 12hr period. These fields define the "true" atmospheric state.

b. Simulate observations

Temperature and moisture profiles from the "true" atmosphere are converted to radiances using a forward radiative transfer model. Both GOES and interferometer radiances are generated at hourly intervals over the 12hr "nature" period. Realistic observation errors will be introduced into the synthesized observations. An example of data coverage is shown. Gaps in the coverage simulate cloudy fields-of-view. A simple cloud mask was used based on dew point depression.

c. Assimilate the synthesized observations

The synthesized radiances will be assimilated at hourly intervals over a 12hr period into the 40km Mesoscale Analysis and Prediction System (MAPS) of the Forecast Systems Laboratory (FSL), also known as the Rapid Update Cycle (RUC) (see Benjamin et al., 1998). This operational forecast system was designed to assimilate observations at hourly intervals. Boundary conditions will be provided by the NCEP Eta model, projected onto the AWIPS 211 grid (80km resolution). These fields are smoother than the AWIPS 212 grid used in the "nature" run. Following the assimilation period a 12hr forecast will be generated. The assimilation cycle will be repeated four times. 1) a run with no observations (NO); 2) a perfect observations run (PO), assimilating temperature and moisture profiles extracted directly from the "nature" run; 3) a GOES Radiometer (GR) run, assimilating temperature and moisture profiles adjusted to emulate GOES retrievals; 4) an interferometer run (GI), assimilating temperature and moisture profiles adjusted to emulate retrievals from a geostationary interferometer. The NO run and the PO run represent the range of performance that can be expected from the MAPS. The GR and GI runs should fall inside this range. The previous 1hr forecast from MAPS will be used as the background profile for the next analysis cycle, just as it would be in an operational system.

d. Assess the impact of the synthesized observations

The impact of the observations on the assimilation system will be assessed by comparing forecasts from each of the experimental runs. The ability of each observing system to steer the forecast toward "nature" will be evaluated by comparing each assimilation run to the "true" atmosphere generated by the UW-NMS model.


3. Preliminary results

a. Observation density assessment

The ability of the RUC to assimilate large amounts of information was assessed. An analysis was performed for 28 August 1998, at 00UTC, using perfect UW-NMS profiles at six increasing observation densities. Data counts ranged from 250 to 6000 profiles of temperature and moisture. The resulting analyses were then compared to "truth" to determine the fit to the data.

  • Analysis Fit of heights as a function of observation density

  • Analysis Fit of temperature as a function of observation density

  • Analysis Fit of relative humidity as a function of observation density

  • Analysis Fit of wind speed as a function of observation density


    b. Perfect observations experiment (PO)

    Perfect temperature and moisture profiles, generated by the UW-NMS, were assimilated over a 12hr period, at one hour intervals. Approximately 5000 profiles were assimilated each hour. Data quality control was turned off. All observations were assumed to be perfect. A one hour forecast was used as the background for each data insert. The root-mean-square fit of the background and the analysis to the "nature" run is shown at each insertion time. This experiment represents the perfect assimilation cycle, or the best temperature/moisture assimilation we can expect from the RUC analysis.

    Data Retention Figures

    Field 850 mb 700 mb 500 mb 300 mb
    Height rms vs hour rms vs hour rms vs hour
    Temperature rms vs hour rms vs hour rms vs hour rms vs hour
    Relative Humidity rms vs hour rms vs hour rms vs hour
    U-Component rms vs hour rms vs hour
    V-Component rms vs hour rms vs hour

    c. Geostationary Radiometer Versus Geostationary Interferometer

    Four experimental simulations have been completed that compare the impact of assimilating retrievals from a geostationary radiometer (GR) to those from a geostationary interferometer (GI). A 'no observation' (NO) simulation and a 'perfect observation' (PO) simulation were also generated to provide a range of expected performance. GR and GI temperature/dew point profiles were simulated by mapping expected errors onto the PO profiles. These were assimilated at one hour intervals, for a period of 12 hours, into the Rapid Update Cycle (RUC) forecast model. Each data set contained approximately 5000 profiles. A 12 hour forecast was generated after the assimilation period. The root-mean-square error of each simulation was computed against 'perfect observations' from the 'nature' run. These results can be displayed from the table below.

    Validation Against The True Atmosphere

    Parameter 850 mb 700 mb 500 mb 300 mb
    Height RMS error RMS error RMS error RMS error
    Temperature RMS error RMS error RMS error RMS error
    Relative Humidity RMS error RMS error RMS error RMS error

    Skill (S1) Score

    Parameter 850 mb 700 mb 500 mb 300 mb
    Height S1 Score S1 Score S1 Score S1 Score
    Temperature S1 Score S1 Score S1 Score S1 Score
    Relative Humidity S1 Score S1 Score S1 Score S1 Score

    Initial 6 hours forecasts runs indicate that the GI results are significantly improved over those from the GR. 500 hPa temperature errors are reduced by 0.2 C root mean square (rms) over the extended CONUS (contiguous United States) and 700 hPa relative humidity errors are reduced by 2%. The shape and location of the GI forecasted temperature and moisture fields were also enhanced over the GR fields.

    d. Validation Against 'Truth'

    The impact of the simulated observing systems was assessed by comparing each experiment run to 'perfect observations' generated by the UW-NMS run. RMS errors and S1 skill scores were computed. Plots are shown for the 12 hour assimilation period and the 12 hour forecast period.

    No Observation (NO) = NO OBS

    Geostationary Radiometer (GR) = GEO-R

    Geostationary Interferometer (GI) = GEO-I

    Perfect Observations (PO) = OPTIMAL

  • GOES-8/10 Composite Image from 28Aug98 12UTC. Rapid Update Cycle (RUC) domain shown in yellow.

     12 Hour Assimilation Period

  • RMS error: 700hPa Relative Humidity

  • S1 skill score: 700hPa Relative Humidity

  • RMS error: 500hPa Temperature

     12 Hour Forecast Period

  • RMS error: 700hPa Relative Humidity

  • S1 skill score: 700hPa Relative Humidity

  • RMS error: 500hPa Temperature

    To assess the relative benefit of the geostationary interferometer over the geostationary radiometer a rating score (1 to 10) was computed. The RMS errors for temperature and relative humidity were summed over four layers (700hPa, 500hPa, 400hPa, 300hPa) and normalized between the RMS error sums from the No Observation (NO) run and the Perfect Observation (PO) run. A score of 10 is perfect.

     12 Hour Assimilation Period

  • Normalized RMS score: 4-layer Temperature

  • Normalized RMS score: 4-layer Relative Humidity

     12 Hour Forecast Period

  • Normalized RMS score: 4-layer Temperature

  • Normalized RMS score: 4-layer Relative Humidity

    4. Pilot Study

    The pilot study consists of three assimilation experiments, all of which include conventional observations. The BEST experiment results are included on the following figures to a give a bound to the best performance possible by the assimilation forecast system. The BEST experiment does not include conventional observations.

  • CONV: Conventional observations only (RAOB, surface, ACARS, and profiler).
  • GEO-R: Conventional observations plus simulated geostationary radiometer observations.
  • GEO-I: Conventional observations plus simulated geostationary interferometer observations.
  • Pilot Study Validation

    Level Field RMS S1 Score Bias
    300 mb U-component RMS error S1 Score Bias
    300 mb V-component RMS error S1 Score Bias
    500 mb Temperature RMS error S1 Score Bias
    500 mb Height S1 Score
    700 mb RH RMS error S1 Score Bias



    Benjamin, S.G., J.M. Brown, K.J. Brundage, B.E. Schwartz, T.G. Smirnova, and T.L. Smith, 1998: The operational RUC-2. Preprints, 16th Conference on Weather Analysis and Forecasting, AMS, Phoenix, AZ, 249-252.

    Hibbard, W. L., B.E. Paul, D.A. Santek, C.R. Dyer, A.L. Battaiola and M-F. Voidrot-Martinez, 1994: Interactive Visualization of Earth and Space Science Computation. Computer, Vol. 27, 65-72.

    Huang, H.L., W.L. Smith and H.M. Woolf, 1992: Vertical Resolution and Accuracy of Atmospheric Infrared Sounding Spectrometers. J. Appl. Meteo, Vol 31, No.3, 265-274.

    Huang, H. L. and R. J. Purser, 1996: Objective Measures of the Information Density of Satellite Data. Meteorology and Atmospheric Physics, 60, 105-117.

    Ma, X., T.J. Schmit and W.J. Smith, 1998: A nonlinear physical retrieval algorithm - Its application to the GOES-8/9 sounder. J. Appl. Meteor., in press.

    Purser, R. J. and H. L. Huang, 1993: Estimating Effective Data Density in a Satellite Retrieval or an Objective Analysis. J. Appl. Meteor., 32, 1092-1107.

    Rodgers, C. D., 1990: Characterization and Error Analysis of Profiles Retrieved from Remote Sounding Measurements. J. Geophys. Res., 95, 5587-5595.

    Tripoli, G.J., 1992: A nonhydrostatic mesoscale model designed to simulate scale interaction. Mon. Wea. Rev., 120, 1342-1359.