Fig. 1
P5.63
Calibration and Initial Results from the OSSEs for NPOESS
M. Masutani*, J. S. Woollen*, J. Terry+, S. J. Lord*,
T. J. Kleespies#, G. D. Emmitt**, S. A. Wood**, S. Greco**,
J. C. Derber*, R. Atlas+, M. Goldberg#
*NOAA/NWS/NCEP/EMC, +NASA/GSFC/DAO,
#NOAA/NESDIS, **Simpson Weather Associates
http://www.emc.ncep.noaa.gov/research/osse
1. INTRODUCTION
The future National POES System (NPOESS) is scheduled to fly during the 2007-2010 period. For the next 10 years, a considerable amount of effort must take place to define, develop and build the suite of instruments which will comprise the NPOESS. The forecast impact of current instruments can be assessed by Observing System Experiments (OSEs), in which already existing observations are denied or added to observations from a standard data base. However, the impact of future instruments must be assessed with experiments using simulated observations. These experiments are known as Observing System Simulation Experiments (OSSEs).
This project is a collaboration among, the National Centers for Environmental Prediction (NCEP), NASA/Data Assimilation Office (DAO), Simpson Weather Associates (SWA), and the National Environmental Satellite, Data and Information Service (NESDIS). Through this collaboration, the data assimilation and modeling communities can be involved in instrument design and can provide information about the expected impact of new instruments. Furthermore, through the OSSEs, operational data assimilation systems will be ready to handle new data in time for the launch of new satellites. This process involves preparation for future data volumes in operations, the development of the data base and data-processing (including formatting) and a quality control system. All of this development will accelerate the operational use of data from the future instruments (Lord et al. 1997).
2. THE NATURE RUN
For the OSSE, a long integration of an atmospheric general circulation model (GCM) is required to provide a "true atmosphere" for the experiment. This is called the "nature run" (NR). The nature run needs to be sufficiently representative of the actual atmosphere and but different from the model used for the data assimilation. The observational data for existing instruments is simulated from NR and impact tests are performed for both real and simulated data.
For this project, the nature run was provided by the European Centre for Medium-Range Weather Forecasts (ECMWF). The description and evaluation of the nature run is provided. (Becker et al., 1996.) A one month model run was made at resolution T213 and 31 levels starting from 5 February 1993. The version of the model used for the nature run is the same as for the ECMWF reanalysis.
The nature run was found to be representative of the real atmosphere but with a few exceptions (Masutani et al. 1999a,1999b). Low level marine stratocumulus required some adjustment. A localized warm SST in southern hemisphere in real data was not included in the nature run. This SST could potentially cause some inconsistent results in OSSE verification results.
3. SIMULATION OF OBSERVED DATA
Details of procedures to simulate observational data are described in Masutani et al. (1999b) and Lord et al. (2001a, 2001b). These papers are available at the OSSE web site. In this paper, simulation of Atmospheric Infrared Sounder (AIRS) and cloud motion vectors (CMV, Velden et al. 1998) are mainly described. AIRS is scheduled to be included in the NPOESS Preparatory Project (NPP) instrument suite.
NASA/DAO is taking the lead for the simulation of realistic conventional observations, including CMV and ACARS. The initial simulation uses real observational data distributions available in February 1993.
Doppler wind lidar (DWL) wind data are produced as line-of-sight (LOS) winds by SWA using their Lidar Simulation Model (LSM). Bracketing sensitivity experiments are being performed for various DWL concepts to bound the potential impact (Lord et al. 2001a). Scanning, and various data sampling strategies are being tested with these experiments. No measurement error is assigned for these initial tests but will be added in the future. Strategies for systematic errors are discussed by Emmitt (2000).
3.1 Simulation of cloud motion vector
The CMV available in 1993 were generally obtained by manual tracking and thus were limited in numbers compared to what is available today. The experiments being presented in this paper used the locations of actual CMVs assimilated in 1993 to pick a wind observation from the nature run. While this approach may be reasonable for the first few days of NR, the locations of clouds suitable for CMVs from NR are different in the out days of NR compared to the actual conditions in 1993. Thus a new CMV simulation algorithm has been developed by NASA/DAO and SWA that will identify NR clouds that would be suitable for motion vectors. This algorithm also assigns both random and systematic error (e.g., navigation, slow bias, height assignment) to the CMVs to insure a more realistic data quality for the OSSE calibration.
The locations of CMV observations, unlike most conventional observations, are derived from cloud and water vapor feature in the atmosphere. A good correlation of the CMV observations locations with the cloud information from NR will add realism to the simulated data for NPOESS OSSE's. An estimate of this correlation was made by determining the distribution of cloud fractions, derived from the model first guess from a real data assimilation, in the vicinity of real CMV observations. This is done separately for several tropospheric layers. These statistics provide statistical constraints for determining the simulated CMV locations when using cloud fractions from NR. Other constraints, internal to the overall wind retrieval process, are also considered. These include: latitude cut-offs; filtering of CMV's based on topography; discrete levels for height assignment; and a nominal count of observations by layer. To obtain realistic simulated CMV locations while maintaining a resolution comparable to real CMV observations, two methods are being developed. One method begins with a fixed grid at a resolution approximating the horizontal spacing of real CMV observations. The grid points are then filtered using the above constraints. The second method, which may be of more practical with higher density observations, involves the creation of a large "pool" of real CMV observation locations which are randomly sampled until the representative distribution is achieved, again, based on the above constraints. A procedure for introducing a correlated height assignment error is also being developed since these errors contribute significantly to overall CMV observational error (Merrill et al. 1991). The height assignment error is assumed to be associated with large-scale atmospheric features (Schmetz and Holmlund, 1992) which are capable of being resolved by NR; therefore one or more NR parameters will likely be used for this procedure.
3.2 Simulation of AIRS radiances
The AIRS simulation package was originally developed by Evan Fishbein of JPL. The simulation (i.e. forward calculation) is based on radiative transfer code
developed by Larrabee Strow (UMBC). The package was modified by Walter Wolf to generate thinned radiance datasets in the BUFR format. The effort to provide AIRS data to NWP centers in near-realtime is being led by Mitch Goldberg (NESDIS). Because the AIRS instrument will not be launched until 2002, the NESDIS AIRS near-realtime system is currently based on simulated data. The NCEP AVN six-hour forecasts are used to specify the state variables needed for the forward calculation. The simulation package runs in real-time and products (thinned radiances and retrievals) are produced in near-realtime. This same package is being used to generated AIRS radiances for the OSSE (Goldberg et al. 2001).
4. DATA ASSIMILATION SYSTEM
The data assimilation system at NCEP is based on the "Spectral Statistical Interpolation" (SSI) of Parrish and Derber (1992), which is a three-dimensional variational analysis (3-D var.) scheme. The TOVS level-1b radiance (T1B) is used (McNally et al., 2000, Derber and Wu 1998) for data assimilation and the March 1999 version of NCEP's operational Medium Range Forecast (MRF) and data assimilation system are used for the data impact test. LOS winds from DWL are directly used in the data assimilation.
The following upgrades of the NCEP operational data assimilation system are in progress.
Development of situation-dependent background error covariances for global and regional systems (Purser and Parrish, 2000).
Bias correction of background field.
Improved moisture background error covariances.
Development of cloud analysis system.
Data from NPP/NPOESS instruments, quickscat observations, GCP radio-occultation observation, GIFTS, DMSP (SSM/IS), and imager radiances (MODIS, GOES, AVHRR) are all planned to be included at a later time.
5. OSSE CALIBRATION
OSSE calibrations are performed for existing instruments by comparing the simulated and real impact tests (Lord 2001a, Lord 2001b). Assessments for future instruments will be performed by comparing relative impacts from the calibration experiments. Denial of RAOB wind (R_wind), RAOB temperature (R_temp), and T1B with various combinations are tested. The analysis with all conventional data and T1B is used as control (CTL). The results show generally satisfactory agreements between real and simulated impacts.
Anomaly correlations for 500hPa height fields for 72 hour forecast skill for the experimentss without T1B (NTV), experiments without R_wind (1BNWIN), experiments without R_temp (1BNTMP) are presented in Fig.1. The forecast skills are verified against experiments with all data (CTL). The geographical distribution of RMSE from the CTL are presented in Lord et al. (2001b). For both real and simulated experiments, 1BNWIN shows least skill in the northern hemisphere (NH) and globally less skill compare to 1BNTMP showing R_wind have more impact compared to R_temp in both simulation and real. The geographical distribution shows that the impact of R_wind is slightly weaker in simulation and the impact of R_temp is slightly stronger in the simulation.
In the NH, T1B shows impact over the Pacific for both the real and simulated analysis, where the RMSE between CTL and NTV is larger in the simulation (Lord et al. 2001b). In the 72 hour forecast the impact of T1B spreads out over the NH and NTV shows a similar magnitude of impact compared to 1BNTMP. One of the reasons for the larger impact of T1B in simulation is the lack of measurement error in the simulated data. Under-estimation of the cloud effect in the simulation is another possible reason for the large impact in simulation.
In the southern hemisphere (SH) T1B has the largest impact. Although simulated T1B is supposed to be too good and a stronger impact is expected, the skill reduction in NTV is far larger in real experiments. It is noted that there is a localized large warm SST in south Pacific in the end of February. However, SST in NR is fixed through the period. The impact of this difference is being evaluated.
In the tropics, a large analysis impact of R_temp in the low troposphere. This large analysis impact is partially related to the bias between the NCEP model and NR. Including a bias correction in the data assimilation may be required for reliable OSSEs. (Purser and Derber, 2001).
In order to test sensitivity to observational error, the difference between observation and analysis (o-a) from the real data assimilation is used as the error for the simulated data. This error will give a large-scale correlated error. With (o-a) error, the rejection statistics of simulated experiments become closer to those for real data. With random error too little data are rejected by quality control.
Another deficiency noted was caused by envelope orography. It was found that the large portion of surface data in the real world are located in underground in NR. As results there are much less surface data in the simulation. It is necessary to test with equal number of surface data for simulation and real.
6. SOME INITIAL RESULTS FROM THE OSSE FOR DWL WINDS
Among many candidate instruments for the OSSE, DWL winds heve been simulated by SWA. According to the strategy for bracketing sensitivity experiments (Lord 2001a, Lord 2001b), scanning or non-scanning, various wave lengths and the number of LOS per measurement, are being tested. Sensitivity to weight in the data assimilation is also being tested. For the first four days of assimilation, 14 combinations of DWL with TIB and conventional data were compared. Assimilation for the total one month period has been conducted for selected cases.
DWL data improved the wind fields globally at all levels for all experiments. Major improvement are over tropics while Marseille et al. (2001) showed major impact in SH but did not use T1B data. T1B data are included in this paper and large improvements are achieved by T1B in the SH. The results also show an advantage of a scanning instrument over a non-scanning instrument in the upper troposphere. The number of measurements an indication of observation quality, becomes more important in the lower troposphere. The analysis impact is sensitive to the weight of observations in the assimilation.
DWL winds also need to be evaluated with both the current data distribution and the anticipated future data distribution corresponding to when the DWL data will be used. For this reason, simulated data from at least one advanced sounder (e.g., AIRS), a scatterometer (e.g., ASCAT) and at least the current ACARS data, must be added in the future. More realistically distributed, high-density, CMV data also need to be included.
7. FUTURE PLANS
The calibration will be continued to gain further confidence in OSSE. Various techniques for adding systematic errors will be tested. The simulation procedure of T1B requires further evaluation, including the formulation of observational errors.
In addition to a DWL and AIRS., the Cross Track Infrared Sounder (CrIS), Conically-scanning Microwave Imager/Sounder (CMIS), and the Advanced Technology Microwave Sounder (ATMS) have been proposed as candidate instruments to be tested by OSSEs. We are proceeding to develop appropriate forward models for these instruments.
In order to make reliable recommendations, the techniques for creating simulated observation need to be refined. Addition of large-scale spatially correlated error and systematic error in simulated data may alter the results.
OSSEs also need to be tested with an upgraded techniques for data handling and data assimilation system. Since the amount of data involved in the future instruments increases drastically, effective super-observations to reduce the sizes of data sets needs to be studied (Purser et al. 2001). Including an adaptive correction for the bias in the data assimilation will also be tested (Purser and Derber, 2001).
Future instruments need to be tested with 2001 and future data distributions since the 1993 data distribution is outdated. An alternative NRs for the same period and summer time have also been generated by NASA/DAO and can be used to investigate additional atmospheric regimes. NRs to test northern summer time response is important especially to study the impact on tropical storm prediction.
The evaluation metrics will be expanded to include diagnostics of strength and position of cyclones and jets and a study of extreme events, as well as standard forecast skill scores. Cost-benefit and flight planning will also be studied.
ACKNOWLEDGMENT
We received much assistance from the Data Services Section and Dr. Anthony Hollingsworth of ECMWF in supplying the nature run. Throughout this project NOAA/NWS/NCEP, NASA/DAO and NOAA/NESDIS staffs provided much technical assistance and advice. Especially, we would like to thank W. Yang, R. Treadon, Y. Zhu, W.-S. Wu, M. Iredel, D. Keyser, W. Collins, and R. Kistler of NCEP, G. Brin, S. Bloom and N. Wolfson of DAO, and V. Kapoor, P. Li and W. Wolf of NESDIS. We would like to thank Evan Fishbein of JPL for providing simulation code for AIRS data. Drs. E. Kalnay, W. Baker, J. Yoe and R. Daley provided expert advice. We appreciate the constructive comments from members of the OSSE Review Panel. This project is sponsored by the Integrated Program Office (IPO) for NPOESS and by the NOAA Office of Atmospheric Research (OAR) and the NOAA National Environmental Satellite, Data and Information Service (NESDIS). We thank Drs. Stephen Mango and Alexander MacDonald, and John Gaynor, Jim Ellickson and John Pereira for their support and assistance in this project.
REFERENCES
Becker, B. D., H. Roquet, and A. Stofflen 1996: A simulated future atmospheric observation database including ATOVS, ASCAT, and DWL. BAMS, 10, 2279-2294.
Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287 - 2299.
Emmitt, G. D., 2000: Systematic errors in simulated Doppler wind lidar observations. http://www.emc.nceo.noaa.gov/resarch/osse/swa/sys_errors.htm
Goldberg, Mitchell D. , Larry McMillin, Walter Wolf, Lihang Zhou, Yanni Qu, and Murty Divakarla, 2001: Operational radiance products from AIRS, AMS 11th Conference on Satellite Meteorology and Oceanography 15-18 October 2001, Madison, Wisconson
Lord, S. J., E. Kalnay, R. Daley, G. D. Emmitt, and R. Atlas 1997: Using OSSEs in the design of the future generation of integrated observing systems. Preprint volume, 1st Symposium on Integrated Observation Systems, Long Beach, CA, 2-7 February 1997.
Lord, S.J., M. Masutani, J. S. Woollen, J. C. Derber, R. Atlas, J. Terry, G. D. Emmitt, S. A. Wood, S. Greco, T. J. Kleespies, 2001a: Observing System Simulation Experiments for NPOESS, AMS Preprint volume the Fifth Symposium on Integrated Observing Systems. 14-19 January 2001, Albuquerque, NM
Lord, S.J., M. Masutani, J. S. Woollen, J. C. Derber, R. Atlas, J. Terry, G. D. Emmitt, S. A. Wood, S. Greco, T. J. Kleespies, 2001b: Observing System Simulation Experiments for NPOESS, AMS Preprint volume the 14th Conference on Numerical Weather Prediction, 30July-2August 2001, Fort Lauderdale, Florida. 167-171.
Marseille, G. J., A. Stoffelen, F. Bouttier, C. Cardinali, S. de Haan and D. Vasiljevic, 2001: Impact assessment of a doppler wind lider in space on atmospheric atalysies and Numerical weather prediction. KNMI, Contract No.13018/98/NL/GD.
Masutani, M., J.C. Woollen, J.C. Derber, S. J. Lord, J. Terry, R. Atlas, S. A. Wood, S. Greco, G. D. Emmitt, T. J. Kleespies 1999a: "Observing System Simulation Experiments for NPOESS", AMS Preprint volume for the 13th Conference on Numerical Weather prediction. September 1999, Denver Colorado,1-6.
Masutani, M. K. Campana, S. Lord, and S.-K. Yang 1999b: Note on Cloud Cover of the ECMWF nature run used for OSSE/NPOESS project. NCEP Office Note No.427
McMillin, L. M., L. Crone and T. J. Kleespies, 1995: Atmospheric transmittance of an absorbing gas. 5: Improvements to the OPTRAN approach. Appl. Opt. 34 (36) 1995.
McNally, A. P., J. C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1 radiances in the NCEP SSI analysis system. Q.J.Roy. Metorol. Soc. , 129, 689-724.
Merrill, R. T., W. P. Menzel, W. Baker, J. Lynch, and E. Legg, 1991. A report on the recent demonstration of NOAA's upgraded capability to derive cloud motion satellite winds. Bull. Amer. Meteor. Soc., 72, 372-376.
Parrish, D. F. and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120, 1747 - 1763.
Purser, R. J. and J. C. Derber, 2001: Unified Treatment of measurement bias and correlation in variational analysis with consideration of the preconditioning problem. AMS Preprint volume for the 14th Conference on Numerical Weather prediction. July 2001, Fort Lauderdale, Florida. 467-470.
Purser, R. J. and D. F. Parrish, 2000: A Bayesian technique for estimating continuously varying statistical parameters of a variational assimilation. NCEP Office Note 429. (Also submitted to Meteor. And Appl. Phys.)
Purser, R. J., D.F. Parrish and M. Masutani 2001:Meteorological observational data compression; an alternative to conventional "Super-Obbing". NCEP Office Note 430.
Saunders, R.W., M. Matricardi and P. Brunel, 1998: An improved radiative transfer model for assimilation of satellite radiances Q. J. Roy. Meteor. Soc. 124.
Schmetz, J. and K. Holmlund, 1992. Operational cloud motion winds from METEOSAT and the use of cirrus clouds as tracers. Adv. Space Res., 12, 95 - 103.
Velden, C. S., T. L. Olander and S. Wanzong, 1998: The impact of multispectral GOES-8 wind information on Atlantic tropical cyclone track forecasts in 1995. Part 1: Dataset methodology, description and case analysis. Mon. Wea. Rev., 126, 1202-1218.