Creating RGB imagery using SIFT and Geo2Grid

July 8th, 2021 |

The use of routine multispectral geostationary satellite imagery over the United States has increased the routine use of Red/Green/Blue composite imagery to describe and evaluate surface and atmospheric conditions. This blog post will detail how to create new (or old) RGB composites using two UW-Madison/CIMSS/SSEC-developed tools: The Satellite Information and Familiarization Tool (SIFT; Journal article link) and Geo2Grid (Previous blog posts showing Geo2Grid examples are here). The scene to be highlighted is shown above in the GOES-16 Cirrus Band; it was chosen because of the interesting parallel bands in the Cirrus, features that can identify regions of turbulence. A larger-scale view of the data (created using CSPP Geosphere) is here (for the 1.37 µm Cirrus band) or here (for True Color).

SIFT has a very useful (and easy!) RGB generator.  For this case involving cirrus, I decided to create an RGB using the Split Window Difference (10.3 µm – 12.3 µm, Band 13 – Band 15) (shown here) that has been used to identify cirrus for quite a while (link to journal article), the cirrus band 4, and also the Snow/Ice channel Band 5 (1.61 µm).  After downloading SIFT and importing the data (and creating the split window difference field — here’s a blog post that describes how to do that), a SIFT user can create an RGB and tinker with the bounds.  Changing the bounds and the gamma causes a simultaneous change in the RGB in the SIFT display window, so it’s not difficult to iterate to a satisfactory solution.  As shown below, the RGB created has the Split Window Difference as the red component, with values from 0 (no red) to 12.0 (saturated red) and a Gamma value of 2;  the cirrus channel (C04) is the green component with values from 0.27 (no green) to 0 (saturated green) and a Gamma value of 2;  the snow/ice channel (C05) is the blue component with values from 0.0 (no blue) to 0.40 (saturated blue) and a Gamma value of 1.

SIFT RGB Creation window

The RGB created in SIFT using these values is shown below.  Maybe using maximum green — a color one’s eyes are usually particularly adept at viewing — for no signal in the cirrus channel was not the best choice.  But there is nice contrast between the background and the thin cirrus, and an obvious difference between the parallel lines of cirrus in the middle of the image and other clouds, such as the cirrus at the western edge of the image!

“Cirrus” RGB at 1411 UTC on 8 July 2021 (click to enlarge)


How do you create something similar using Geo2Grid?  Step 1, of course, is always to download and install the software package.  To see what products can be created with geo2grid, enter this command:  ./geo2grid.sh -r abi_l1b -w geotiff --list-products -f /path/to/the/directory/holding/GOESR/Radiance/Files/*syyyydddhhmm*.nc .  Let’s assume all 16 channels from ABI are available.  Important caveat: Geo2Grid will only work on one data time at a time, so specify your year/julian day/hour/minute with sufficient stringency.

RGB product definitions are found in yaml files within the Geo2Grid directory. Ones for abi in particular are found in $GEO2GRID_HOME/etc/satpy/composites/abi.yaml in which file you would enter something what is shown below for a product called ‘cirrustest’;  note that it has three channels:  the first is a difference between C13 and C15 (that is, the Split Window Difference);  the second is C04 (cirrus channel) and the third is C05 (snow/ice channel). This is the same as in the SIFT definitions.

Within $GEO2GRID_HOME/etc/satpy/enhancements/abi.yaml there is a further definition of this RGB.  The crude stretch defines the bounds of the RGB:  Red includes values from 0 – 12;  Green from 27 — that is, a reflectance of 0.27, or 27% — to 0 (note that it is inverted);  Blue from 0 to 40.  In addition, Gamma values are specified:  0.5, 0.5 and 1.

Two important things to note:  Gamma in SIFT follows National Weather Service and JMA conventions.  Gamma in Geo2Grid follows EUMETSAT conventions. Thus, one is the reciprocal of the other.  Also, note the _abi suffix in the abi.yaml file name in enhancements, i.e., cirrustest_abi, to specify the satellite.

After making these changes to the two abi.yaml files, and rerunning this command:  ./geo2grid.sh -r abi_l1b -w geotiff --list-products -f /path/to/the/directory/holding/GOESR/Radiance/Files/*syyyydddhhmm*.nc, you should see a new possibility: cirrustest (or whatever you have named your new RGB). Then you run Geo2Grid commands to create the cirrustest RGB (with the -p cirrustest flag.  The commands below sequentially create the grid for the analysis, create the tiff file, georeference it with coastlines (none, in this case over the Gulf) and latitude/longitude lines, and annotate it.

../p2g_grid_helper.sh CIRRUSRGBtest -88.3 26.6 500 -500 960 720 > $GEO2GRID_HOME/CIRRUSRGBtest.conf
#
../geo2grid.sh -r abi_l1b -w geotiff -p cirrustest C04 -g CIRRUSRGBtest --grid-configs $GEO2GRID_HOME/CIRRUSRGBtest.conf --method nearest -f /arcdata/goes_restricted/grb/goes16/2021/2021_07_08_189/abi/L1b/RadC/*s20211891411*.nc
../add_coastlines.sh --add-borders --borders-outline='blue' --borders-resolution=f --add-grid --grid-text-size 20 --grid-d 5.0 5.0 --grid-D 5.0 5.0 GOES-16_ABI_RadC_cirrustest_20210708_1411??_CIRRUSRGBtest.tif
convert GOES-16_ABI_RadC_cirrustest_20210708_1411??_CIRRUSRGBtest.png -gravity Southwest -fill yellow -pointsize 14 -annotate +8+24 "1411 UTC 8 July 2021 Cirrus RGB" GOES-16_ABI_RadC_cirrustest_20210708_1411_CIRRUSRGBtest_annot_2.png

The final image from Geo2Grid is shown below. Its geographic coverage is slightly different than in SIFT, above, but the two RGBs have similar looks.

‘Cirrustest’ RGB at 1411 UTC on 8 July 2021 (Click to enlarge)

Creating and Displaying gridded GLM fields using data from NOAA CLASS

March 23rd, 2021 |

GOES-16 Gridded GLM imagery of Total Optical Energy for the 1 minute ending 1501 UTC on 22 March 2021 (Click to enlarge)

This blog posts describes how to use NOAA’S CLASS (Comprehensive Large Array-data Stewardship System) system (link) that contains Level-2 GLM data, (under the GOES-R Series GLM L2+ Data Product (GRGLMPROD) tab) to create useable GLM imagery. GLM processing produces three Level 2 files each minute, and those files can be processed to produce imagery. First, choose the time range you want in CLASS, and get the global imagery.  For this blog post, I chose GOES-16 data on 22 March 2021 between 15:00 an 15:15 UTC.  On the CLASS website, I clicked the GLM L2+ Lightning Detection Data and didn’t filter by any values (CLASS allows you to filter by minimum/maximum flash, event and group counts, if you want).  This request returned 47 different files, but that is only about 10 Mbytes.  Some of the file names — two minutes’ worth — are shown below: LCFA files from julian Day 081 (that is, 3/22/2021) starting at 15:00:00:00, 15:00:00:20, 15:00:00:40, … etc.


OR_GLM-L2-LCFA_G16_s20210811500000_e20210811500203_c20210811500218.nc
OR_GLM-L2-LCFA_G16_s20210811500200_e20210811500404_c20210811500425.nc
OR_GLM-L2-LCFA_G16_s20210811500400_e20210811501003_c20210811501016.nc
OR_GLM-L2-LCFA_G16_s20210811501000_e20210811501205_c20210811501226.nc
OR_GLM-L2-LCFA_G16_s20210811501200_e20210811501403_c20210811501419.nc

Code to convert these files (that contain raw-ish group, event and flash fields) to gridded GLM fields (that can be displayed with, for example, Geo2Grid, or AWIPS) is within the CSPP Gridded GLM software package that can be downloaded here (free registration may be required; the Gridded GLM tarball to download includes a short and useful README). To create a data file that is properly configured for Geo2Grid (or AWIPS), with software that uses the open-source glmtools software developed by Dr. Eric Bruning at Texas Tech, use this command:

cspp-geo-gglm.sh ../../data/OR_GLM-L2-LCFA_G16_s20210811501*

That will create a file with a name like this:

CG_GLM-L2-GLMF-M3_G16_s20210811501000_e20210811502000_c20210821745120.nc;

Geo2Grid can then be used to create imagery from the newly-created netCDF file. The Geo2Grid code used is below.

../p2g_grid_helper.sh TestGridded -75.0 8. 1000 -1000 2000 750 > $GEO2GRID_HOME/TestGridded.conf
../geo2grid.sh -r glm_l2 -w geotiff -p total_energy -g TestGridded --grid-configs $GEO2GRID_HOME/TestGridded.conf --method nearest -f /home/scottl/CSPPGeo/GGLM/cspp-geo-gridded-glm-1.0b1/bin/CG_GLM-L2-GLMF-M3_G16_s20210811501000_e20210811502000_c20210821745120.nc
../add_colormap.sh ../../../enhancements/TotalEnergy.txt GOES-16_GLM_total_energy_20210322_150100_TestGridded.tif
../add_coastlines.sh --add-coastlines --coastlines-resolution=h --coastlines-outline='black' --add-grid --grid-text-size 12 --grid-d 1.0 1.0 --grid-D 1.0 1.0 --add-colorbar --colorbar-tick-marks 250.0 --colorbar-text-size 1 --colorbar-no-ticks --colorbar-align bottom GOES-16_GLM_total_energy_20210322_150100_TestGridded.tif
convert GOES-16_GLM_total_energy_20210322_150100_TestGridded.png -gravity Southwest -fill white -pointsize 24 -annotate +8+30 "1501 UTC 22 March 2021 Total Energy" GOES-16_GLM_total_energy_20210322_1501_Labeled.png

The Geo2Grid package commands above (1) created the grid (‘TestGridded’) onto which the data were interpolated; (2) created the imagery from the netCDF file output from the Gridded GLM package; (3) Added a pre-defined colormap (within ‘TotalEnergy.txt’); (4) Added coastlines, a lat/lon grid, and a colorbar and (5) annotated the image. This last command used ImageMagick.

Note that the GLM image created, shown at top, is mostly transparent. Three areas of GLM observations are apparent, two over South America, one over the Pacific Ocean south of Panama. The transparency is handy if you want to overlay GLM data on top of ABI imagery!

Using NUCAPS lapse rates to evaluate atmospheric stability

February 26th, 2021 |

GOES-17 Visible Imagery (2300 UTC), NOAA-20 NUCAPS-derived lapse rate (925 – 700 mb, 23:03 UTC) and NUCAPS sounding points (2249 UTC) on 25 February 2021 (Click to enlarge)

NUCAPS profiles derived from CrIS and ATMS data on NOAA-20 provide model-independent estimates of atmospheric thermodynamics globally, including, for this case over the central Pacific Ocean, in regions otherwise bereft of data.  NUCAPS lapse rates show a minimum in stability in low-levels in between two cloud features; the region includes mostly ‘green’ NUCAPS retrieval points:  where infrared and microwave retrievals have both converged.  It is difficult in the case above to relate differences in cloud features to differences in the diagnosed stability.

Four minutes later (shown below), NOAA-20 was closer to the Pole on this ascending pass and the diagnosed stability does relate well to differences in cloud structures.  In particular, the change from lapse rates around 5 C/km northeast of Hawai’i to lapse rate closer to 2 or 3 C/km even farther northeast aligns with a boundary between cloud types.

GOES-17 Visible Imagery (2310 UTC), NOAA-20 NUCAPS-derived lapse rate (925 – 700 mb, 23:07 UTC) and NUCAPS sounding points (2249 UTC) on 25 February 2021 (Click to enlarge)

The subsequent NOAA-20 pass was west of the main Hawai’ian Island chain.  Again, differences in lapse rates are related to cloud features in the visible imagery.  Stable air — with lapse rates between 3 and 4 C/km — overlies a region of very little cumuliform development.  A region of larger lapse rates over the eastern 1/3rd of the pass, just to the west of the Hawai’ian Islands is accompanied by cumulus development.  NUCAPS thermodynamic fields, even though they have limited resolution in the vertical (at most 10 layers in the enter tropopause), can give useful information on stability over the ocean that can help in the real-time diagnosis of the atmosphere.

Using ATMS data to observe lake ice coverage over the Great Lakes

February 16th, 2021 |

ATMS Ice Concentration over the Great Lakes, from overlapping ascending Suomi-NPP passes on 30 January and 15 February 2021 (Click to enlarge)

CIMSS produces Advanced Technology Microwave Sounder (ATMS) Lake Ice concentration images in a format that can be inserted into AWIPS.  These images are created from data downloaded (available at this ftp site;  imagery is also available here) at the DB antennae at CIMSS, and processed with MIRS algorithms (all MIRS products are available at this NOAA Website) that are incorporated into CSPP.   This document (from ESA) includes a figure showing how emissivities of ice and water differ, allowing for discrimination between ice and open water.  The differences are especially large at lower frequencies.

The animation below from NOAA/NESDIS of ice concentration over the USA (including the Great Lakes) (source) shows obvious increases and decreases in ice concentration;  given the general very cold conditions over the Great Lakes during this time (especially over Lakes Michigan and Superior), the reduction in ice cover on 14-15 February is inconsistent with the cold weather.

Ice coverage increases from 10-13 February and then decreases.  This change in ice coverage matches view angle changes from the ATMS instrument on Suomi NPP, and those view angle changes affect the spatial resolution of the measurements.  There was a near-nadir afternoon pass on 10 February, and diagnosed ice in Lake Michigan was at a minimum;  ascending pass views of Lake Michigan on 12 and 13 February are near the limb and diagnosed ice over Lake Michigan reached a maximum;  the view was near nadir again on 15 February when, again, diagnosed lake ice was at a minimum.  (Suomi NPP Orbit paths are available here).  It is important when using Ice Coverage data to know the view angle from the satellite!

NPP MIRS Ice Coverage, 4 – 15 February 2021, from Ascending (afternoon) Passes (Click to enlarge)

MIRS algorithms to compute Ice Concentration use information from ATMS channels 1, 2, 3, 16 and 17.  These 5 channels have footprints ranging from 15 to 75 km (at satellite nadir), as shown in the image below, from this paper. Note especially how the footprints increase in size at the limb:  channel 17’s footprint ranges from 15×15 km at nadir to 68×30 km at the limb!   A challenge in using ATMS is that the microwave footprint can easily observe both land and water, in which case the microwave data will not give values representative of the lake coverage.

Scanning geometry for 22 ATMS channels. The figure includes footprint sizes at nadir and at the limb (Click to enlarge)

The figure below shows circles with diameters of 15, 50 and 75 km;  the smaller circle is the approximate nadir footprint of channel 16 and 17 at ATMS;  the larger circle is the approximate nadir footprint of channels 1 and 2.  Lake Ice resolution from MIRS might be considered to be of the order of 50 km.

Circles with diameters of 15, 50 and 75 km in Lakes Michigan, Huron and (inset) Erie (click to enlarge)

NOAA-20 and Suomi-NPP, the two satellites that carry ATMS as part of their payloads, both have 16-day repeats.  That is:  the satellite traces out the same path every 16 days;  in addition, paths are very similar every 5 or 6 days.  See, for example, this toggle of (Suomi NPP) NUCAPS soundings points over the South Pacific ocean, on 25 July and 10 August 2019, 16 days apart.  The same orbit is traced out on these two days.  That is why the ATMS ice concentration plots at the top of this post are from 30 January and 15 February:  16 days apart. The two orbit mappings at the links are identical. The 15 February image of orbits is shown below.

Interpretation of the Ice Concentration imagery at the top of this blog post requires knowledge about the path of Suomi-NPP shown below.  Lake Michigan and western Lake Superior are close to nadir, and there should be some ATMS footprints entirely within those lakes.  Lakes Huron, Erie and Ontario are far enough away that a user might not trust 100% the data being presented.  The ice coverage change between the two days might be useful:  there is a general increase in concentration over coastal Lakes Michigan and Superior.

Predicted Suomi-NPP Orbits for 15 February 2021 (Click to enlarge)

A morning descending pass of Suomi-NPP moved over eastern Lake Ontario, giving the best resolution over that small Great Lake.  The 0659 UTC image from 16 February is shown below.  Notice the difference in Lakes Superior and Michigan between this image (for which Lakes Superior and Michigan are near the limb) and the image at top (for which Lakes Superior and Michigan are near nadir).

Suomi NPP ATMS Estimates of Lake Ice, 0659 UTC on 16 February 2021 (Click to enlarge)

The 15 February 2021 analysis (from this page) from NOAA’s Great Lakes Environmental Research Lab (GLERL) is shown below.  Consider the ATMS imagery as an approximation to the observed field. Care in interpretation of ATMS data is a necessity because of errors that occur when pixels are not entirely over water.  That is a frequent occurrence when the satellite is scanning along the limb.

Ice concentration over the Great Lakes, from GLERL, 15 February 2021 (Click to enlarge)


The toggle below (from this site) highlights resolution differences between ATMS Channel 1 (23.8 GHz), with 75-km resolution at nadir, and ATMS Channel 17 ( 165.5 GHz), with 15-km resolution at nadir.  Note also the differences in the signals between western Lake Erie (ice covered) and eastern Lake Erie (more open water).

ATMS imagery (Channels 1 and 17) derived from Suomi NPP at ~1800 UTC on 17 February 2021 (NPP overflew Buffalo NY on this day) (Click to enlarge)