Using NUCAPS soundings to nowcast convective evolution

August 15th, 2019 |

GOES-16 Visible (Band 2, 0.64 µm) Imagery, 1721 – 1946 UTC on 15 August 2019. NUCAPS Sounding Points — from 1926 UTC — are present over the image at 1946 UTC (Click to animate)

GOES-16 Visible Imagery, above (Click to animate), shows shower/thundershower development over eastern Oklahoma moving into Arkansas. At the end of the animation, 1946 UTC, NUCAPS Sounding profiles from 1926 UTC are shown, and they’re shown below too.

GOES-16 Visible (Band 2, 0.64 µm) Imagery, 1946 UTC on 15 August 2019. (Click to enlarge)

The time 1946 UTC is about the earliest you could hope to have NUCAPS profiles in an AWIPS system — and only if you had access to a Direct Broadcast antenna. The more conventional method of data delivery, the SBN, means NUCAPS will be available about an hour after they are taken, so by 2036 UTC. The visible imagery at 2036 UTC is shown below.

GOES-16 Visible (Band 2, 0.64 µm) Imagery, 1946 UTC on 15 August 2019. (Click to enlarge)

At 2036 UTC, which time is about when in the forecast office the NUCAPS soundings would become available, would you expect the convection in western Arkansas to move southward, or eastward, based solely on Satellite imagery? How could you use NUCAPS profiles to gain confidence in this prediction? Visible imagery alone suggests a moisture boundary; the southern quarter of Arkansas shows markedly less cumulus cloudiness. The animation shows motion mostly to the east, with higher clouds moving more west-northwesterly. The GOES-16 Baseline Total Precipitable Water product, below, shows a maximum in TPW over central Arkansas, with values around 1.5″;  values are around 1.3″ in southern Arkansas, and around 1.2-1.3″ in northwest Arkansas.  A corridor of moisture is indicated.

GOES-16 Baseline Level 2 Total Precipitable Water at 1946 UTC; Visible imagery is shown in cloudy regions. (Click to enlarge)

Baseline Total Precipitable Water, above, part of a suite of products that emerge from Legacy Profiles, is heavily constrained by model fields, however;  the image above could simply show the GFS solution.  In contrast, NUCAPS observations are almost wholly independent of models.  What do NUCAPS profiles show? The animation below steps through vertical profiles east and south of the developing convection.

NUCAPS profiles from the ~1900 UTC overpass at points plotted over the 1946 UTC GOES-16 Band 2 Visible (0.64 µm) image (Click to enlarge)

AWIPS will soon (planned for shortly after Labor Day at the time of this post) include horizontal fields of information derived from NUCAPS vertical profiles. The images below show values computed within the NSharp AWIPS software for a variety of fields: Total Precipitable Water, MU Lifted Index, MU CAPE, MU CINH. All fields suggest that convection more likely to build eastward than to expand southward.

NUCAPS Sounding Points and derived quantities, as indicated, at 1926 UTC 15 August 2019; NUCAPS data are plotted over the 1946 UTC GOES-16 ABI Band 2 Visible 0.64 µm image. (Click to enlarge)

Convection did not move southward; motion and development was to the east. The timing of NUCAPS profiles means that they give a good estimate of atmospheric thermodynamics in mid-afternoon, a key time for assessing convective development.

GOES-16 Visible (Band 2, 0.64 µm) Imagery, 1721 UTC on 15 August 2019 to 0001 UTC on 16 August 2019 (Click to animate).

Use Polar2Grid to create VIIRS True Color imagery over one State (Missouri)

April 11th, 2019 |

VIIRS True-Color Imagery over Missouri, 1942 UTC on 9 April 2019 (Click to enlarge)

Polar2Grid allows users to create true-color imagery from VIIRS (Visible Infrared Imaging Radiometer Suite) data from Suomi-NPP or NOAA-20. This tutorial will take you through the needed steps. Step one is to decide when you want the data; the ways to determine when a Polar Orbiter overflies a particular point are outlined in this blog post, that points to this website. For this blog post I’ve chosen Missouri. The image above shows a True-Color image over Missouri at about 19:42 UTC on 9 April 2019.

To create true-color imagery, Polar2Grid requires VIIRS M-Bands 3, 4 and 5 (Blue (0.48 µm), Green (0.55 µm) and Red (0.67 µm), respectively, all with 750-m resolution); click here for a list of all VIIRS bands). If the VIIRS I-Band 1 (at 0.64 µm) is present in the directory, then that image is used to sharpen the resultant image. Polar2Grid CREFL software also performs a simple atmospheric Rayleigh scattering removal; smoke and haze will still be apparent in the imagery, however.

To create the imagery above, first order the data from NOAA Class. (Steps to follow are shown here). Download the data into a unique directory. We are going to remap these data onto a map centered on Missouri, and for that to happen, Polar2Grid needs mapping parameters. These can be generated automatically with the p2_grid_helper.sh script that comes with Polar2Grid software. From the bin directory, I entered this command to put the grid parameters in a file .

/p2g_grid_helper.sh missouri -93.0 38.0 500 -500 2000 2000 > my_grids.txt

The line of data entered into that file is this:

missouri, proj4, +proj=lcc +datum=WGS84 +ellps=WGS84 +lat_0=38.000 +lat_1=38.000 +lon_0=-93.000 +units=m +no_defs, 2000, 2000, 500.000, -500.000, -99.055deg, 42.352deg

Now I’m ready to generate a true-color image (corrected ceflectance — crefl — imagery) with Polar2Grid, using this command:

./polar2grid.sh crefl gtiff –grid-configs /home/scottl/Polar2Grid/polar2grid_v_2_2_1/bin/my_grids.txt -g missouri -f /data-hdd/storage/Polar2GridData/09April/

The flags “–grid-configs <path to directory where file created by p2g_grid_help sits” and “-g map <name of map inside that file>” instruct to the Polar2Grid software to pull the mapping data for the defined grid out of the file. Otherwise, the data are in satellite projection. This polar2grid.sh invokation created a file named ‘j01_viirs_true_color_20190409_194226_missouri.tif’; I want to put a map on it so it is easier to georeference, and that is done using this shell in the Polar2Grid bin directory:

./add_coastlines.sh –add-borders –borders-resolution=f –borders-level=2 –borders-outline=’black’ j01_viirs_true_color_20190409_194226_missouri.tif

This adds a map to the image, then converts it to the png file (j01_viirs_true_color_20190409_194226_missouri.png) that is shown above.

After doing the same steps for a series of clear days in the midwest (09 March 2019, 15 March 2019, 21 March 2019, 26 March 2019, 31 March 2019), and annotating and concatenating the images in an animation, the greening up of Spring is apparent. See below.

NOAA-20 VIIRS True Color Imagery on select mostly clear days over the mid-Mississippi Valley, dates and times as indicated in the image (Click to enlarge)

Special shout-out to Dave Hoese, SSEC/CIMSS, for crafting software that is so easy to use to produce excellent satellite imagery.

Using Polar2Grid software to create JPSS Imagery

April 9th, 2019 |

NOAA-20 VIIRS Imagery (Image bands I01 (0.64 µm), I03 (1.61 µm) and I05 (11.45 µm)) at 23:25 UTC on 8 April 2019 (Click to enlarge)

Polar2Grid is Python-based software created at CIMSS/SSEC to process data from Polar Orbiting satellites into useful imagery (or into files that can be imported directly into the National Weather Service Advanced Weather Information Processing System (AWIPS) for display). (You can download it for a linux machine at this site.) This tutorial will describe where to find downloadable NOAA-20 (or Suomi-NPP) data, and how to create imagery. Thus, this blog post is similar to earlier blogs posts (here, and here) that described how to use Geo2Grid for Geostationary Satellites. Earlier blog posts showing Polar2Grid output are here and here.

There are data repositories associated with many direct broadcast antennas — but they are not always publically available. NOAA CLASS data is a reliable source for NOAA-20 and Suomi-NPP data. Typically data shows up there within six hours of its being sampled by the satellite. For a post-event analysis, this is a good data source. For near-real time investigation, it’s better to acquire data from a direct broadcast site. (Data from the Direct Broadcast antenna at CIMSS are available here.)

The first thing required at the NOAA CLASS site is to request the data that you want to access. There is a long drop-down menu (‘Please select a product to search’ is its original title) on the CLASS front page; use the drop down to select ‘JPSS Visible Infrared Imaging Radiometer Suite Sensor Data Record (VIIRS_SDR)’. This tells CLASS that you want VIIRS data from either Suomi NPP or NOAA-20. Click on the ‘Go’ button to the right of the menu bar. That takes you to a selection page.

It is necessary to log into the CLASS system. When you order data, an email is sent acknowledging the order, and a follow-on email once the data are available specifies how you can download it. The log-in tells CLASS where to send the emails.

Polar2Grid expects the data to be ‘de-aggregated’, so you must set User Preferences in NOAA CLASS to do this. Click on the ‘User Preferences’ link in the table on the left-hand side of the main NOAA CLASS Page (after you have logged in). The location is shown below in the toggle.

NOAA Class Front Page after logging in as a user. Note the ‘User Preferences’ Tab, circled in red.

Once you have entered the User Preference page, make certain that the ‘Package Geolocation with JPSS Data Products’ button is toggled to ‘No’. (Shown here) You also do want to De-aggregate JPSS Data Products. (That is, separate files for each band). In practice, this blogger finds it easier (that is, Polar2Grid complains less) if I make separate data requests at CLASS for each VIIRS band I want to view. (Update: ‘JPSS Companion Files to deliver with data’ should also be blank, as as shown at the link. This allows for one data request.)

Data requests require you to know something about the orbits of the satellites — so you can narrow down the amount of data you want. This handy website shows where all polar orbiters have been flying. Follow the links to the specific satellite and location; for example, if we’re interested in NOAA-20 orbits over Hawai’i on April 8 2019, this map shows them. The map suggests requesting NOAA-20 from 23:20 to 23:30 on 8 April 2019 will give a good view of the islands.

The data we will request are I01, I03 and I05 (0.64 µm, 1.61 µm and 11.45 µm, respectively). In addition to requesting the Sensor Data Records, it is also necessary to request, near the bottom of the page, the ‘VIIRS Image Bands SDR Ellipsoid Terrain Corrected Geolocation (GITCO) (public)’. These geolocation files are used by Polar2Grid to navigate the imagery.

After submitting the data request, and waiting a period of time (sometimes as short as two or three minutes), you will receive an email telling you exactly where the data sits, and how to access it. It will look something like this. Download the data into a directory, and you’re ready to have Polar2Grid create imagery for you. The directory for I05 (I will put data for each band in a separate directory) should include files named like this:

GIMGO_j01_d20190408_t2320306_e2321551_b07188_c20190409223158545413_noac_ops.h5
GITCO_j01_d20190408_t2320306_e2321551_b07188_c20190409223155222712_noac_ops.h5
SVI03_j01_d20190408_t2320306_e2321551_b07188_c20190409223158545413_noac_ops.h5

‘GIMGO’ and ‘GITCO’ refer to geolocation data; even though you only requested GITCO fields, CLASS will give both. That’s okay. SVI01, SVI03 and SVI05 hold data from bands I01, I03 and I05, respectively. The ‘j01’ tag signifies data from NOAA-20, or JPSS-1. For the 20-minute request I made for each band, I received 8 of each type of file.

The Polar2Grid command that creates imagery is this one:

polar2grid.sh viirs_sdr gtiff -p i03 -f /home/scottl/Polar2Grid/data/viirs/I03/

The polar2grid.sh shell script is within the bin directory of the Polar2Grid software that emerges after you g-unzip and untar the file from the Polar2Grid download site. The command results in a .tif file with highest-resolution grey-scaled imagery. The image below (converted from .tif to .png) shows the scene. It’s quite large: 6876×7616 pixels!

NOAA-20 Imagery (I-3, 1.61 µm) from 23:20 to 23:30 on 8 April 2019 (Click to enlarge)

It’s useful to add maps to the imagery, and that is done in Polar2Grid after the .tif file has been created by using ‘add_coastlines.sh’, also in the bin directory. The command can add coastlines (–add-coastlines), borders (–add-borders), rivers (–add-rivers) and lat/lon lines (–add-grid). Lat/Lon lines can also be labeled. When maps are added to the .tif file, the image is also saved as a .png file so the original scene is not overwritten. You can use the -o flag to rename the new file. The command I used is shown below.

add_coastlines.sh –add-coastlines j01_viirs_i03_20190408_231906_wgs84_fit.tif –coastlines-resolution=f –coastlines-level=6 –coastlines-outline=’magenta’ -o j01_I03_20190408_231906.png

After adding maps to all three files created, I used ImageMagick to crop the large fields to something more manageable in size (convert -crop 1520×900+2112+2409 +repage -quality 100 j01_I05_20190408_231906.png j01_I05_20190408_231906.gif, for example), and added annotation to the imagery:

convert -font helvetica -fill yellow -pointsize 36 -draw “text 70,70 ‘NOAA-20 VIIRS Image Band I01 (0.64 µm) 23:25 UTC 8 April 2019′” j01_I01_20190408_231906.gif j01_I01_20190408_231906.annot.gif

The animation at the top of this blog post shows all three images (I01, I03 and I05) sequentially.

Final note: Infrared imagery by default is not color-enhanced, and built-in enhancements are not (yet) available. Polar2Grid does support creation of colortables, however. That will be the subject of a future blog post.

Adventures with geo2grid: Creating Stereoscopic Imagery in True Color

March 14th, 2019 |

GOES-17 True Color (left) and Himawari-8 True Color (right) at 0330 UTC on 13 March 2019 (Click to enlarge).

Geo2grid is a python-based software package that creates GeoTIFF imagery from native Himawari or GOES-16/GOES-17 imagery, as noted here. This blog post documents how to use the geo2grid software to create stereoscopic imagery, using either a Himawari-8/GOES-17 pairing, or a GOES-16/GOES-17 pairing. This requires first a remapping of the imagery to a fixed domain; when Geostationary Satellites aren’t separated by a great distance — for example when GOES-17 was in the test position and GOES-16 was at 75.2 — native projections can be used. That’s not the case with Satellites separated by 60 degrees of longitude.

Fortunately, geo2grid allows for a way to define a grid onto which the extracted data will be placed. The shell script command to create the map parameters is shown below:

$GEO2GRID_HOME/bin/p2g_grid_helper.sh G17H8Stereo -175.0 0.0 2000 -2000 1000 1000 > $GEO2GRID_HOME/mygrids.conf

I’m creating a map called ‘G17H8Stereo’ that is centered at 175 W and the Equator (Note: if you include a decimal point, you must include a digit afterwards. Some scripting languages fail to interpret ‘-175.’ correctly). The x-direction spacing is 2000 m (i.e., 2 km) and the y-direction spacing is also 2 km (that value is negative because point 1,1 is in the northwest corner). The grid size being created here is 1000×1000. If you were to look in the file created, mygrids.conf, you’d see a line looking like this:

G17H8Stereo, proj4, +proj=eqc +datum=WGS84 +ellps=WGS84 +lat_ts=0.00000 +lon_0=-175.00000 +units=m +no_defs, 1000, 1000, 2000.00000, -2000.00000, 176.01685deg, 8.98315deg

Note that the file name must have that “.conf” extension! The reading software expects it.

Data for both times (Full Disk imagery) has been downloaded and placed in directories.  This is HSD *.DAT files for Himawari-8 and netCDF Radiance files from CLASS for GOES-17.  This is a lot of data to move around.  The geo2grid invocation to create the True Color Imagery will look something like this for Himawari-8:

$GEO2GRID_HOME/bin/geo2grid.sh -r ahi_hsd -w geotiff –grid-configs $GEO2GRID_HOME/mygrids.conf -g G17H8Stereo –method nearest -f /data-ssd/CLASS/CSPPCheck/Stereo/H8/

The GOES-17 call will look like this:

$GEO2GRID_HOME/bin/geo2grid.sh -r abi_l1b -w geotiff –grid-configs $GEO2GRID_HOME/mygrids.conf -g G17H8Stereo –method nearest -f /data-ssd/CLASS/CSPPCheck/Stereo/

In both cases, –grid-configs is used to specify the grid to be used, with the -g tag naming the grid (the same name as used in the p2g_grid_helper.sh call above. The method of interpolation (the –method flag) is nearest neighbor, so a simple interpolation is used. Again, remember that those long dashes are really two short dashes.

Geo2grid does have built-in maps that you can use, and these are listed in the on-line documentation; you would include something like “-g lcc-aus” and that would put the data on a lambert conformal grid centered over Australia (not a useful grid for GOES-17, but very nice for Himawari-8 and for the coming GEOKOMPSAT-2!)

True Color imagery is created by these geo2grid.sh calls — and imagery for all 16 bands is created as well. (You can use the -c flag in geo2grid.sh to limit what is created if you wish). That imagery is shown above. If you cross your eyes and focus on the image that appears in the middle, it will be in three dimensions. Because this region is in the middle of the ocean, geo-location might be important, and the geo2grid script add_coastlines.sh is useful to add latitude/longitude lines.


How will True Color appear in regions with land features as might occur with GOES-16 and GOES-17?  Halfway between GOES-16 (75.2) and GOES-17 (137.2) is 106 degrees W Longitude.  I’ll create a map centered at 35 N, 106 W (near Albuquerque) that is 1200×1200 (also 2 km resolution):

$GEO2GRID_HOME/bin/p2g_grid_helper.sh G16G17Stereo -106.0 35.0 2000 -2000 1200 1200

The output is placed in the same Mygrids.conf file (More than one map definition can appear in that csv file). AFter downloading the GOES16/GOES17 data, I invoked to geo2grid commands:

$GEO2GRID_HOME/bin/geo2grid.sh -r abi_l1b -w geotiff –grid-configs $GEO2GRID_HOME/mygrids.conf -g G16G17Stereo –method nearest -f /data-ssd/CLASS/CSPPCheck/Stereo/G16G17/G17/

$GEO2GRID_HOME/bin/geo2grid.sh -r abi_l1b -w geotiff –grid-configs $GEO2GRID_HOME/mygrids.conf -g G16G17Stereo –method nearest -f /data-ssd/CLASS/CSPPCheck/Stereo/G16G17/G16/

Use ImageMagick to put the images side-by-side

montage GOES-16_ABI_RadF_true_color_20190313_210036_G16G17Stereo.tif GOES-17_ABI_RadF_true_color_20190313_210038_G16G17Stereo.tif -tile 2×1 -geometry +0+0 GOES-16_GOES-17_ABI_RadF_true_color_20190313_210036_G16G17Stereo.png

The beautiful stereoscopic image below is created.

True-Color imagery from GOES-16 (Left) and GOES-17 (Right) over the western United States at 2100 UTC on 13 March 2019 (Click to enlarge)

The mp4 animation below (click here for an animated gif) shows GOES-16 True Color imagery every 15 minutes (GOES-16 was in Mode 3 operations with 15-minute full-disks) from 1500 UTC to 2245 UTC. Imagery was created using geo2grid. The true-color imagery captures the dust that was kicked up by strong winds over Texas and New Mexico.

GOES-16 True Color animation, 1500-2245 UTC on 13 March 2019 (Click to play mp4 animation)

A similar animation made from GOES-17 from geo2grid is below. (Click here for an animated gif).

GOES-17 True Color animation, 1500-2245 UTC on 13 March 2019 (Click to play mp4 animation)

The GOES-16 and GOES-17 animations are combined into a true-color stereoscopic view of the strong cyclone below. The mp4 is below; click here for an animated gif.

True-Color imagery from GOES-16 (Left) and GOES-17 (Right) over the western United States from 1500-2245 UTC on 13 March 2019 (Click to play mp4 animation)