AVHRR Imagery and Products in D-2DInstructions for AWIPS InstallationSpace Science and Engineering Center
|
 |
| Project members: Scott Bachmeier, Jordan Gerth, Scott Lindstrom | |
Announcements
This data is provided as part of the GOES-R Proving Ground. For assistance and suggestions on obtaining and using the data, please contact Jordan Gerth.
This data has been undergoing evaluation at the National Weather Service in Milwaukee/Sullivan since 2010.
In addition to AWIPS, this data and a short archive is also available online:
Acquiring Data at a NWSFO
This page contains information for obtaining data from the NOAA satellites equipped with a Advanced Very High Resolution Radiometer (AVHRR) for display in AWIPS.
For AVHRR imagery (CGINI)
Data is currently available to Weather Forecast Offices in all regions. Check with scientific services at your regional headquarters to see if the feed is available to you. Contact Jordan Gerth for more information.
Data is transmitted in compressed GINI (CGINI) format on the EXP feed of the Local Data Manager (LDM). This data is nearly identical in format to the NOAA polar orbiter AVHRR imagery that was approved for transmission via NOAAPORT in a 2006 Technical Implementation Notice (TIN). We have been careful to work with NESDIS to produce this imagery in a similar routine to how it would have been created prior to broadcast over NOAAPORT. NOAA was unable to deliver this data due to bandwidth constraints on the ground. Because the delivery method (via LDM) is different than proposed in the TIN, this is not an operational dataset, so unplanned data errors and outages may result. Contact Jordan Gerth once you are receiving data so we can add your office to the e-mail announcement list. We value any feedback.
The LDM feed of the CGINI data from CIMSS/SSEC contains additional data from NOAA-15 and NOAA-19 not referenced in the TIN. Thus, data will be more frequent than stated in the TIN.
The Space Science and Engineering Center is not continuously staffed. Consequently, data outages and processing issues may result. This imagery should be considered non-operational. For AVHRR products
Data is currently available to Weather Forecast Offices in Central Region, Eastern Region, Southern Region, Western Region, and Alaska Region. We are unable to supply this to Pacific Region WFOs at this time. Data is transmitted in compressed netCDF format on the EXP feed of the Local Data Manager (LDM). This is not an operational dataset, so unplanned data errors and outages may result. Contact Jordan Gerth once you are receiving data so we can add your office to the e-mail announcement list. We value any feedback.
The Space Science and Engineering Center is not continuously staffed. Consequently, data outages and processing issues may result. The products should be considered non-operational. Required LDAD/Processing Configurations
Perform the following actions to configure your Local Data Acquisition and Dissemination (LDAD) server.
In /data/fxa/LDAD/data as user ldad on server px2:
- Append to LDADinfo.txt
In /awips/ldad/bin as ldad on px2:
- Add preProcessCGINI.pl and assign executable permissions
- Execute "stopLDAD.sh;startLDAD.csh" to restart the LDAD service
In /usr/local/ldm/etc as ldm on ls2:
- Modify ldmd.conf consistent with the TIN (position sensitive; change ldm.crh to your regional LDM server)
- For WFOs in Alaska Region, change "TITE" to "TITB".
- For WFOs in Pacific Region, change "TITE" to "TITI".
- For WFO SJU San Juan, PR, change "TITE" to "TITQ".
- For CONUS WFOs, execute "cat $FXA_HOME/data/localizationDataSets/$FXA_LOCAL_SITE/whichSat.txt" as fxa on dx3:
- If the output is "WEST", change "TITE" to "TITW" (West CONUS WFOs).
- Otherwise, leave the ldmd.conf as-is with "TITE" (East CONUS WFOs).
- Append to pqact.conf
- Copy ldmd.conf and pqact.conf to ls3 with scp
- Execute "ldmadmin restart" to restart the LDM service
- With "ldmadmin watch -f EXP", watch the LDM queue up to 4 hours for CGINI data (may take longer)
A zip file of these is available.
If all you wish to display is CGINI data (Visible, IR Window, Shortwave IR, and Fog), you are finished.
The remaining steps in this section are only necessary to obtain the AVHRR products.
As ldad on ls2:
- Create /data/ldad/AVHRR and give it user/group writable permissions
In /usr/local/ldm/etc as ldm on ls2:
- Modify ldmd.conf (position sensitive; change ldm.crh to your regional LDM server)
- Append to pqact.conf
- Copy ldmd.conf and pqact.conf to ls3 with scp
- Execute "ldmadmin restart" to restart the LDM service
- With "ldmadmin watch -f EXP", watch the LDM queue up to 4 hours for AVHRR data (may take longer)
Caution: The LDAD is not used to process AVHRR files; there is no preprocess script and subsequently there is no entry in the LDADinfo.txt file. However, the LDAD is used to process CGINI files. The AVHRR files are filed in their LDM state (compressed) on ls1, but moved to px1 for processing and filing. This is done by setting and running a cron task at least once hourly, which will be done in the following steps. It is important to assure this cron task exists and is executing properly or AVHRR products may quickly overwhelm ls1. The process script writes to a file in the log directory /data/logs/fxa.
As fxa on px1:
- Create the directory /data/local/localapps/AVHRR
- Copy processAVHRR.pl into the new directory and assign executable permissions
As root on px1:
- Add a cron task to /etc/cron.d/SITEpx1cron and /etc/ha.d/cron.d/SITEpx1cron
As root on px2:
- Add the same cron task to /etc/ha.d/cron.d/SITEpx1cron in case of a failover
Your operations should roughly follow this outline.
su - fxa cd /data/local/localapps # Choose the directory you wish mkdir AVHRR mv /tmp/processAVHRR.pl.gz AVHRR cd MODIS # Move to that directory gunzip processAVHRR.pl.gz chmod 775 processAVHRR.pl su - root vi /etc/cron.d/SITEpx1cron # Make cron change in two files vi /etc/ha.d/cron.d/SITEpx1cron # After the first file complete ssh px2 vi /etc/ha.d/cron.d/SITEpx1cron # In case px1 fails <Page Up> # vi command to move to bottom of file a # vi command to append <Enter> # vi command to add blank line # Process AVHRR products from CIMSS/SSEC 5,20,35,50 * * * * fxa csh -c '/data/local/localapps/AVHRR/processAVHRR.pl >& /dev/null' <Esc> # vi command to stop append mode :wq! # vi command to write and quitA zip file of these is available.
Required AWIPS Customizations
Perform the following actions to configure your Advanced Weather Interactive Processing System (AWIPS) workstations/servers.
The display for AVHRR imagery has already been implemented in AWIPS. No extra work is required. Check the satellite menu.
The steps in this section are only necessary to display the CONUS AVHRR products.
Based on instructions.txt, use the two Perl scripts included with color table manipulation package manage-ct.tgz to extract the color tables from the included AvhrrColors.nc file and add into /data/fxa/workFiles/customColorMaps.nc. You may wish to use a subset of these color tables as the default enhancement for the AVHRR products which will be added via localDepictKeys.txt below.
- Download and un-gzip manage-ct.tgz
- Additional instructions about how to use the Perl scripts for adding color tables to your office repository are available
In $FXA_CUSTOM_FILES as fxa (should be mounted on any workstation/server):
- Add conusOne.sup to provide supplementary mapping information
- Append to localDataKeys.txt to map product keys to directory locations
- Append to localDepictKeys.txt and modify the 999x color table keys for visualization
- Append to localImageStyle.txt to provide sampling
- Append to localProductButtons.txt to label the menu items
- Append to localPurgeInfo.txt to keep only recent files
- Append to otherSatMenus.txt or dataMenus.txt (see "Known Pitfalls") for selection
A zip file of these is available.
When complete with the changes, run '$FXA_LOCALIZATION_SCRIPTS/mainScript.csh f -tables -dataSups -dirs' on dx3 and dx4, create a directory /data/fxa/sat/SBN/netCDF/LOCAL/AVHRR/temp as fxa with write permissions, then execute '$FXA_LOCALIZATION_SCRIPTS/mainScript.csh f -tables -dataSups' on all lx and na workstations. If you experience a problem displaying the data, you may also need to run each of the mainScript.csh commands with the "-clipSups" argument.
As fxa on dx1 and dx2, perform a '-tables -purge' localization. On dx1, restart the purgeProcess.
As fxa on dx1, restart the notificationServer to complete.
Restart D-2D on the lx/na workstations to access the new products from the satellite menu (or custom menu if dataMenus.txt modified).
Optional — You can have GUARDIAN send an announcement when a new pass has arrived with these instructions.
Known Pitfalls
- The instructions assume local files in $FXA_CUSTOM_FILES do not have the WFO identifier as the first three characters of the file name (LLL-).
- The satellite menu may be too long. This can be fixed.
If you change LLL-otherSatMenus.txt to LLL-dataMenus.txt in your /data/fxa/customFiles directory, and at the top of that file add a line such as, menu: "Local Sat" you can move all the CIMSS/SSEC products to their own menu between Maps and Help. Then localize with mainScript.csh -tables.If problems arise — You may need to run dos2unix on some scripts and files.
Learn More