International MODIS/AIRS Processing Package

International MODIS/AIRS Processing Package (IMAPP)

Software from the Space Science and Engineering Center, University of Wisconsin-Madison

Latest update on March 30, 2004: Nanjing Remote Sensing Symposium announced.

Requirements | Release History | File Formats | Credits | Links

The NASA Terra and Aqua spacecraft provide new remote sensing capabilities for the observation of planet Earth. Both of these platforms have a direct broadcast X-band downlink that allows MODIS (Terra and Aqua) and AIRS/AMSU/HSB and AMSR-E (Aqua) data to be received in real time by sites having the proper reception hardware.

The International MODIS/AIRS Processing Package (IMAPP) allows ground stations capable of receiving EOS direct broadcast to create the following products:

MODIS Level 1 calibrated and geolocated radiances (Terra or Aqua)
MODIS Level 2 geophysical products (Terra or Aqua)
AIRS/AMSU/HSB Level 1 calibrated and geolocated radiances (Aqua only)

IMAPP is derived from the operational EOS processing software developed at NASA GSFC and JPL, and has been modified to be compatible with direct broadcast data. The main differences between IMAPP and the operational software are

IMAPP has been ported to a range of UNIX platforms,
the only tool kit required by IMAPP is NCSA HDF4,
the IMAPP processing environment is greatly simplified,
downlinked spacecraft ephemeris and attitude data may be used for real-time geolocation,
overpasses of arbitrary size may be processed.

Requirements

IMAPP software and test data are available from our FTP site (also see the links below for each IMAPP release). Please read the top level README file for installation instructions. IMAPP is licensed under the terms of the GNU General Public License. IMAPP MODIS Level 1 and Level 2 software supports the following UNIX platforms:

SGI MIPS, IRIX 6.5
Sun Ultra, Solaris 7, 8
IBM RS/6000, AIX 4.3
HP PA-RISC, HP-UX B.10.20
Intel Pentium, Red Hat Linux 7.2 (with gcc/g77 2.96)
Intel Pentium, Sun Solaris 8 (with gcc/g77 2.96)

IMAPP AIRS/AMSU/HSB software supports the following UNIX platforms:

Red Hat Intel Linux: 7.3, 8.0, 9
Sun Solaris (SPARC): 7, 8, 9

The input expected by IMAPP is EOS Level 0 data, defined as reconstructed time-ordered CCSDS packets with all communication artifacts (including duplicate packets) removed. This includes PN and Reed Solomon decoding as well as reassembly of the raw telemetry packets. Processing of the data stream to this stage is the responsibility of the user. Data in this format is known as a Production Data Set (PDS). Many X-band ground station vendors supply processing hardware and/or software that creates IMAPP compatible Level 0 files in PDS format. Pat Coronado's group at GSFC has also developed a Java application named RT-STPS which handles CCSDS packet reassembly. For more information, see the GSFC direct readout website.

MODIS Level 2 science products require MODIS Level 1B HDF radiance and geolocation files in either IMAPP or GSFC/DAAC format (both formats are supported). These HDF data are extracted into flat binary files and used as input to the science product software. Some products also require external dynamic ancillary data files as input. For example, the cloud mask product requires the Near Real-Time SSM/I EASE-Grid Daily Global Ice Concentration and Snow Extent (NISE) dataset for optimal performance over snow covered regions. All required dynamic ancillary data sets are updated daily and are available from our ancillary data FTP site.

Release History

March 30, 2004: Nanjing Remote Sensing Symposium June 6-12, 2004 (Conference Flyer, SeaSpace Announcement)

March 3, 2004 ( MODIS Level 2 v1.5) Aqua/Terra Sea Surface Temperature Product Release:

First release of MODIS Sea Surface Temperature (SST) product software compatible for both Aqua and Terra.

The technique used in the creation of the IMAPP SST product differs from the official DAAC MOD28 product. Please see SST_DOC.pdf for more information.

January 16, 2004 (MODIS Level 1 patch) Update for Terra MODIS calibration following spacecraft safe hold:

Following the restoration of Terra MODIS direct broadcast on 2004/01/15, we discovered a problem with the IMAPP MODIS Level-1 calibration code.
The primary symptom is that most of the earth view image data in the Level-1B product files will be set to "Invalid", even though the raw earth view counts are valid.
To fix the problem, minor patches are required to three of the IMAPP MODIS v1.4/v1.5 calibration source code files.
We strongly recommend that all IMAPP MODIS Level-1 users install this patch as soon as possible. Please note that Aqua MODIS data is unaffected by this problem, and the patch has no effect on the processing of Aqua MODIS.

November 19, 2003 (MODIS True Color) New Tutorial on Producing MODIS True Color Images:

The tutorial describes in detail a method for creating high quality true color MODIS images using freely available software.
Required input data includes MODIS Level 1B 1000, 500, and 250 meter radiances, and 1000 meter geolocation.
Either IMAPP or DAAC format MODIS Level 1B data may be used.
Tutorial includes documentation, source code, and sample data.
Examples of MODIS true color images created using this method are available at the SSEC MODIS Gallery.
Created by Liam Gumley, Jacques Descloitres, and Jeffrey Schmaltz.

November 6, 2003 (MODIS Level 1 v1.5) Update for Terra and Aqua MODIS:

Included in this release is an updated version of the geolocation code; an updated set of calibration lookup tables (LUTs); and an updated MODIS Level 1 master script.
An important feature of the new geolocation algorithm is that it resolves some recent problems (Oct/Nov 2003) which have been observed with Terra MODIS geolocation. Symptoms reported by IMAPP users in the United Kingdom, Germany, Australia, and Alaska include (a) geolocation algorithm exits prematurely, (b) geolocation data produced for Terra MODIS is incorrect. In all cases, the new version of the geolocation algorithm has fixed the problem.
This release contains a new set of calibration lookup tables (LUTs) for the MODIS reflected solar bands and thermal emissive bands. The new versions of the LUTs are Terra: V 4.1.2.6 (April 15, 2003); Aqua: V 4.1.3.5 (April 29, 2003). For a history of lookup table changes, please see the MCST website under the "LUTs Version" links for Terra and Aqua.
Note that the calibration LUTs are not identical to the current versions used at the GSFC DAAC, as they do not include the new time-dependent coefficients for the MODIS ocean bands. A new version of the IMAPP calibration algorithm is under development which includes the most recent MODIS LUTs. The time-dependent coefficients have a small effect on the calibration of the MODIS ocean bands only.

November 5, 2003 (AIRS/AMSU/HSB Level 1 v1.0) First AIRS/AMSU/HSB Release:

Allows any ground station capable of receiving direct broadcast from Aqua to produce calibrated and geolocated AIRS, AMSU, and HSB Level 1B products. This package is functionally identical to the operational AIRS/AMSU/HSB Level 1B package running at the GSFC DAAC.
This is a binary-only release for Red Hat Intel Linux and Sun Solaris (SPARC) platforms. No source code is included.
The package has been tested on the following platforms: Red Hat Intel Linux: 7.3, 8.0, 9; Sun Solaris (SPARC): 7, 8, 9.

September 18, 2003 (MODIS Level 2 v1.4) Aqua/Terra Atmospheric Profiles and Water Vapor Product Release:

Updated MODIS Aqua and Terra compatible atmospheric vertical profiles of temperature and moisture, and total precipitable water vapor, total column ozone and atmospheric stability software (MOD07). A "Platform Name" argument is now required to run the algorithm. Coefficients and bias corrections are chosen based upon the satellite platform name. This direct broadcast profiles release is concurrent with DAAC atmospheric profiles MOD07 operational version 4.1.0.

Updated the forward model codes and coefficients used in computing radiances from the training profiles with the latest version using the HITRAN2000 spectral database.

Bias corrections are now read in from an ascii file, instead of hardwired into the code, and a new input ascii file containing a list of the detectors to be used for each band has been created. All changes, including the improvement of surface emissivities and the handling of integration of near-surface layer precipitable water are described in detail in the HISTORY file.

First Aqua and Terra compatible band 26 correction. Aqua coefficients are now available and a "Platform Name" argument is now required to run the algorithm.

Images of the test Aqua and Terra direct broadcast data sets from 24 August 2002 are included:


Terra IMAPP L2 v1.4 Images	Aqua IMAP L2 v1.4 Images
Total Precipitable Water	Total Precipitable Water
Cloud mask	Cloud mask
Band 2	Band 2
Band 26 corrected	Band 26 corrected
Band 26 un-corrected	Band 26 un-corrected

February 19, 2003 (MODIS Level 2 v1.3) Aqua/Terra Cloud Mask and Cloud Top Properties Release:

First MODIS Aqua and Terra compatible cloud mask and cloud top property release. A "Platform Name" argument is now required to run the algorithms.

Cloud mask retrievals have been improved over problem areas such as sun glint and nighttime land. Cloud top retreivals use different noise thresholds based upon satellite platform. As a result, more CO2 retreivals are expected for Aqua due to the reduced noise in the longwave IR bands. Please see HISTORY for more information.

A software routine to correct for radiation from band 5 (1.2 microns) leaking into band 26 (1.38 micron) detectors is included in this release (DESTRIPE_BAND26.tar). The correction to the radiances helps when imaging band 26, and aids in the correct identification of thin cirrus in the cloud mask algorithm.

IDL software is included which can convert output binary flat files into a stripped down HDF file (create_fake_mod35.pro , create_fake_mod06.pro ).

The IMAPP Cloud Mask software is compatible with DAAC MOD35 version 4.2.0 that was delivered to the DAAC on November 2002. The IMAPP Cloud Top Properties and Cloud Phase science software is identical to the MOD06CT package v4.0.4 delivered to the DAAC in December 2002.

Images of the test Aqua and Terra direct broadcast data sets from 5 January 2003 are included:


Terra Cloud Mask	Aqua Cloud Mask
Cloud mask	Cloud mask
Band 2	Band 2
Band 26 corrected	Band 31
Band 26 un-corrected	Land Sea Bits (6 and 7)
Snow Bit (5)	Snow Bit (5)
Day/Night Bit (3)	Visible Ratio Test Bit (21)
1.38 micron High Cloud Test bit (16)
4-11 micron Test bit (19)


Terra Cloud Top Properties	Aqua Cloud Top Properties
Cloud Top Pressure	Cloud Top Pressure
Cloud Phase	Cloud Phase

October 1, 2002 (MODIS Level 2 v1.2) Atmospheric Profiles and Water Vapor Product Release:

Second MODIS science product release. The key parameters of this product (MODIS product MOD07) are retrievals of vertical profiles of atmospheric temperature and moisture, total column ozone, total column water vapor, stability parameters (lifted index, K index, total totals), as well as low level moisture (water vapor between surface and 700 hPa) and high level moisture (water vapor between 400 and 100 hPa).This software is equivalent to the current DAAC version 3.1.0 delivered to GSFC on 4 March 2002.

September 13, 2002 (MODIS Level 1 v1.4) First version to support both Terra and Aqua MODIS:

The Terra MODIS calibration and geolocation algorithm and lookup tables are unchanged from the MODIS Level 1 v1.3 release.
The Aqua MODIS calibration algorithm and lookup tables are versions 3.0.1 and 3.1.0.2, respectively. This version includes post-launch calibration data acquired following the Aqua MODIS nadir door opening. It will handle all Aqua MODIS data acquired to date.
The Aqua MODIS geolocation algorithm requires that ephemeris and attitude files be used, since spacecraft position is not encoded in the Aqua MODIS Level 0 PDS files. For realtime processing, the GBAD data (APID 957) transmitted as part of the X-band downlink may be used in conjunction with the GSFC GBAD processor to produce IMAPP-compatible ephemeris and attitude input files. Our experience has shown that realtime geolocation using GBAD derived ephemeris and attitude data is accurate to within 500 meters. For non-realtime processing, the definitive Aqua ephemeris and attitude files (see Links below) may be used, providing the same geolocation accuracy as the operational processing at GSFC. Note that the definitive ephemeris and attitude files are usually not available for at least 24 hours.
The IMAPP MODIS Level 1 processing script (util/imapp.csh) has been updated to handle both Terra and Aqua data. This script combines the functionality of the level1a.csh, geolocate.csh, and calibrate.csh scripts and allows all input files to be specified on the command line. The user is responsible for identifying the satellite (Terra or Aqua).

May 1, 2002 (MODIS Level 2 v1.1) First science product release for Terra MODIS:

This release includes the MODIS Cloud Mask (MOD35) and Cloud Top Properties (MOD06CT) algorithms. The release consists of an ancillary data extractor, a set of flat file radiance/geolocation extractors, the cloud mask software and the cloud top properties software. The extractors serve as data preprocessors for the science software packages. Production scripts exist as part of both the cloud mask and cloud top properties packages which will execute the extractors and product software in sequence. Please see the README files. The cloud mask software is updated to production version 3.1.1, whereas the cloud top properties and cloud phase software is updated to version 3.1.0.

December 3, 2001 (MODIS Level 1 v1.3) Third update for Terra MODIS:

The calibration algorithm and lookup tables are updated to versions 3.0.0 and 3.0.0.7, respectively. This version includes calibration data for the A-side electronics on MODIS following the outage which occurred from June 15 to July 3, 2001. The calibration algorithm in IMAPP v1.3 is date sensitive, and may be used for all Terra MODIS data back to February 2000.
The geolocation algorithm now has improved interpolation/extrapolation of ephemeris and attitude information. This prevents bad lat/lon values toward the beginning and ending sections of a pass (originally released as a patch for IMAPP v1.2 on August 31, 2001).

April 13, 2001 (MODIS Level 1 v1.2) Second update for Terra MODIS:

Calibration algorithm and lookup tables are updated to versions 2.5.5 and 2.5.5.1 respectively, which includes calibration data for the B-side electronics on MODIS (the switch to B-side electronics occurred on 1 November 2000). The calibration in IMAPP v1.2 is date sensitive, and may be used for all Terra MODIS data back to February 2000. This version also includes a bug fix for aggregation of the 250 m and 500 m spectral bands to higher spatial resolution.
Geolocation is significantly improved when using only the Level-0 platform ephemeris and attitude information (i.e. in near-realtime). Terrain correction is now available as an option (requires that DEM data files be installed: see the installation instructions). Definitive Terra ephemeris and attitude data are now available via FTP for input to IMAPP (see Links).
Level-1A algorithm is more resistant to Level-0 input file anomalies. Processing now terminates gracefully if a packet with an improper length is encountered in the Level-0 input file.

November 1, 2000 (MODIS Level 1 v1.1) First update for Terra MODIS:

Calibration algorithm and lookup tables are updated to version 2.4.3. This includes many post-launch improvements and bug fixes from the MODIS Calibration Team at GSFC
Added support for definitive ephemeris and attitude data
Solarisx86 is now supported on Intel Pentium platforms

May 12, 2000 (MODIS Level 1 v1.0) Initial release including the following functionality for Terra MODIS:

Reformatting from time-ordered CCSDS Level-0 packets to Level-1A
Geolocation for every 1000 m pixel
Calibration for every pixel in bands 1-2, 1-7, and 1-36 at 250, 500 and 1000 m resolution respectively

File Formats

The format of the IMAPP MODIS Level 1B files is very similar (but not identical) to the format of the GSFC DAAC Level-1B output files. In particular, IMAPP output files are written using HDF only (not HDF-EOS), and IMAPP output files store metadata in separate global attributes. The difference can best be seen by comparing an alphabetized listing of the HDF SDS and Vdata objects in an IMAPP Level 1B 1000 meter file and the corresponding DAAC file. Similarly, the format of the geolocation files produced by IMAPP is very similar (but not identical) to the GSFC DAAC format, as seen in the alphabetized listing of the HDF SDS objects in an IMAPP geolocation file and the corresponding DAAC file.

The IMAPP MODIS Level 2 science product files are not in HDF format. They consist of 8-bit unsigned integer (cloud mask) or 32-bit floating point (cloud top properties) files plus header files. Quality assurance files are also produced. Comparison between the HDF SDS listings of the operational versions and the direct broadcast product flat file header files assist in revealing the differences. The radiance/reflectance/geolocation and cloud top properties flat files can be displayed using the freely distributed FreeLook application or with the commercial ENVI application. Please see the Cloud_Mask_Visualization.txt file for suggestions on viewing the cloud mask product file.

The IMAPP AIRS/AMSU/HSB Level 1B files are in HDF-EOS format, identical to the operational format produced at NASA/GSFC.

Credits

The IMAPP team at SSEC includes Tom Rink (Level 1 software development); Kathy Strabala and Russ Dengel (Level 2 software development); Liam Gumley (program management); and Allen Huang (principal investigator). Eurico D'Sa at the University of South Florida was instrumental in porting IMAPP to Linux, and Alexander Shumilin of ScanEx ported IMAPP to Windows. Development of IMAPP is supported by NASA under grant NAG5-13402.

The operational versions of the MODIS Level-1 software were developed at NASA GSFC by the MODIS Science Data Support Team (SDST) and Characterization Support Team (MCST) under the direction of Vince Salomonson (MODIS Science Team Leader). The operational versions of the MODIS Level-2 cloudmask, cloud top properties and cloud phase software and atmospheric profiles were developed at the University of Wisconsin-Madison by Kathy Strabala, Richard Frey, Liam Gumley, Bryan Baum, Jun Li and Suzanne Wetzel-Seemann under the direction of Paul Menzel and Steve Ackerman (MODIS Science Team Members).

The AIRS/AMSU/HSB package was developed at NASA Jet Propulsion Laboratory by Quyen Nguyen, Richard Forrister and Evan Manning under NASA RTOP 622-40-45-03 in support of the AIRS Project at NASA Jet Propulsion Laboratory under NASA NPO-21132.

Related Links (updated 2004/05/06)

IMAPP References:
Conference Paper SPIE Remote Sensing of the Atmosphere, October 2002
Conference Paper AMS 12th Conference on Satellite Meteorology and Oceanography, February 2003.

IMAPP ancillary data:
leapsec.dat | utcpole.dat | Level-2 product ancillary data

GSFC definitive attitude and ephemeris:
Terra | Aqua

MODIS Level 1B data:
Univ. of Wisconsin: Terra FTP, HTTP, DODS; Aqua FTP, HTTP, DODS (what is DODS?)
NASA/GSFC: Terra and Aqua Direct Broadcast | Terra Global | Aqua Global

AIRS Level 1B data:
Univ. of Wisconsin: FTP, HTTP, DODS (what is DODS?)

MODIS calibration:
Algorithm Theoretical Basis | Calibration Support Team | Level-1B Product Information

Spectral Response Data:
Terra | Aqua

EOS direct broadcast information:
Home | DAAC | Univ. Dundee FAQ

Deep Space Network:
Home | Turn-off schedule

AIRS:
Home | ATBDs

Liam Gumley (last update May 6, 2004)