HISTORY.txt changes to PGE02/MOD_PR02 (Level 1B Calibration) This file shows the following: (a) What was changed in the PGE (b) Why was it changed (c) How the output product will be affected by the change (d) The date of the change. v2.4.3 07/25/2000 ============================================================================ 1. Logic for invalid values of "Scan quality array" was changed. Previously, an invalid value of Scan quality array would result in a MOD_PR02 error exit (whole granule was lost). Now, if an invalid value is encountered, the scan is treated as missing and the rest of the scans of the granule are processed normally. 2. A logic error in Preprocess was corrected for saving the value of Band 31 dn_bb values. Previously, the value from the wrong scan and detector was being used for the PC bands cross-talk correction algorithm. Since the values are fairly stable from detector to detector and scan to scan, there will only be a small change in the Emissive bands. In a representative test case, the maximum change was about 0.5%. 3. A function was added to fill in interpolated scaled integer (SI) values for pixels that correspond to a dead detector (as flagged in the QA LUT "Detector Quality Flag Values"). The value is a linear average of valid SI values from the two nearest live detectors within the same scan. If two valid values are not available for the average, then one valid value is used (simply copied). If no valid values are available, then the pixel is set to 65531 (as in previous versions). Users of the product may identify scan lines that are affected by looking at the attribute "Dead Detector List". A side effect is that the value of the ECS core metadata field "QAPERCENTINTERPOLATEDDATA" may now be non-zero. The file specifications were updated to describe this change. 4. The LUT ingest architecture was changed to incorporate data-collection- time-dependent LUTs. The LUT file architectures were changed such that scalar, numerical LUTs implemented as global attributes were changed to be SDSs. There were several such changes in each LUT HDF file. When the values of the LUTs remain the same, there are no changes in the output products. 5. In Granule.c, one instance of function "sqrtf" was changed to "sqrt" because some compilers do not support the sqrtf function. There was no change in the product. v2.4.2 06/09/2000 ============================================================================ 1. A bug in Emissive_Cal was fixed. The indexing of Planck_mir was B_emiss but should have been D_emiss. This was causing erroneous data in all emissive bands radiances. Thus, there will be substantial changes in all emissive bands radiances. v2.4.1 (NOT USED IN OPERATIONS) 05/05/2000 ============================================================================ The macros controlling the maximum number of scans (Granule.h) were raised to allow processing of a MOD01 granule with up to 208 scans. Also, an algorithm was included to detect occurances of "split" scans, which are the root cause of the number of scans in a MOD01 granule exceeding 204. When a "split" scan is detected, the output product scaled integers for that scan will be set to the fill value of 65535. v2.4.0 (NOT USED IN OPERATIONS) 04/24/2000 ============================================================================ 1. New MCFs were delivered by ECS and incorporated into this version. There were no code changes required for the new MCFs. The effect on the product was: (a) a default value of "Not Investigated" was set in the MCF for the core metadata field ScienceQualityFlag, (b) a new archive metadata field, DESCRREVISION, was added and a a value of "1.0" was set in the MCF file. The file specs were changed to reflect these changes. 2. The production rules (MOD_PR02_production_rules.txt) were changed in accordance with CCR 519. This change formally implements the change described below under version "v2.3.2 (emergency makefile modification)". The makefile "MOD_PR02.mk" was modified as described in that section. The possible changes in the product are described in that section. 3. A new MOD_PR02 failure exit code was added as approved in CCR 520. Previously, the only failure exit code was a "1". Now, when the current MOD01 granule is found to have no scans (empty granule), MOD_PR02 will exit with an exit code of 233. This failure exit code is passed out as the PGE02 exit code. 4. A check on the MOD01 Number of Scans was added to Read_L1A_OBCEng to enable detection of empty MOD01 current granules and to produce a clear error message. There were no other changes to the products. 5. Several QA attributes approved in CCR 508 were added. These do not change the product instrument and uncertainty SDSs. The file specs were changed to reflect these new attributes. 6. A bug in the algorithm for determining the moon in the SV port was corrected. Previous instances where the moon was flagged to be in the SV port for thermal bands were erroneous. Other than the periods when the moon was erroneously flagged to be in the SV port, there are no changes to the product due to this change. v2.3.2 (emergency makefile modification) 02/25/2000 ============================================================================ Because of the high number of PGE01 production failures during the checkout phase, some MOD01 data could not be processed under the current production rules of requiring 3 successive MOD01 granules. To try to maximize the data generated, an emergency change to the V2.3.2 makefile was made to allow processing without a previous or following MOD01 granule if the granule is not available at the time of execution for MOD_PR02. The following is the text of the change as sent to the GDAAC: The change is to the MOD_PR02 makefile: MOD_PR02.mk which is not included in the tar file. Only the comparison input and output files are included. GDAAC personnel will modify the MOD_PR02 makefile as follows. Add the following to the "ADD_CFLAGS" macro: -DALLOW_MISSING_GRANULES MOD_PR02 should then be completely recompiled and re-linked. If a previous or following MOD01 granule is not available, then do not define those LUNs in the PCF. This instruction applies to LUNs 500000 (previous MOD01 granule) and 500002 (following MOD01 granule). Set the wait timers to zero for both the previous and following granules. In the comparison files supplied with this patch, the LUNs 500000 (previous MOD01 granule) and 500002 (following MOD01 granule) were not defined in the PCF. If the previous and following granules are not included in the execution, there is no effect on the product for 250m and 1km reflective solar bands. For thermal emissive bands there is a small effect on scans near the granule boundary. To date, using simulated data, typical maximum percentages changes in the "EV_1KM_Emissive" SDS between a product calculated using all three MOD01 granules and a product calculated with the previous and following granules missing are less than 0.1 %. For the 500m SWIR reflective solar bands, there is an effect on the product because the algorithm uses the band 28 radiance. v2.3.2 01/05/2000 ============================================================================ A printf inside a preprocessor #ifdef block was removed from Granule.c. There were no changes to the product from this code change. v2.3.1 (NOT USED IN OPERATIONS) 12/28/1999 ============================================================================ 1. The meaning of _FillValue for scaled-integer, uncertainty and samples- used EV SDSs was changed to include reflective band data at night. This change actually restores the meaning of _FillValue for those SDSs to the meaning prior to V2.3.0. In V2.3.0, implementation of a unique value for RSB at night (as per CCR 505) resulted in the night mode granules becoming the same size as the day mode granules. By restoring the meaning of _FillValue to include reflective band data at night, the night-mode granule sizes prduced by V2.3.1 are restored to their reduced sizes of 21 MB for the 250m and 500m products, and 136 MB for the 1km product. v2.3.0 (NOT USED IN OPERATIONS) 12/14/1999 ============================================================================ 1. The current version reflects changes implemented for the following CCRs which were approved: 470, 491, 498, 501, 502, 505, and part of 508. The following CCRs were rejected but notes were added to the file specs alerting users that certain data items will be removed after launch + 6 months: 485, 488, 504, part of 508. Note that for CCR 508, several new attributes were approved to be added and two bits redefined in the BIT QA flags. Only the redefinition of the two bits was implemented in this release. 2. Both TEB and RSB uncertainty algorithms were substantially revised. Also, bugs in converting uncertainty to uncertainty index were corrected. Thus, there may be substantial changes in the uncertainty index SDSs for this release compared to the last release. 3. Several new LUTs were added, several old LUTs were deleted and several LUTs had name changes. Thus, the LUT files (Emissive_Lookup_Tables_file, Reflective_Lookup_Tables_file and QA_Lookup_Tables_file) for previous releases are not compatible with this release. 4. The reflective calibration algorithm changed, resulting in small changes to the reflective bands scaled integer SDSs and their scaling attributes. However, there should be relatively little change in the reflectance and radiance products (except for SWIR bands, described below). 5. A new SWIR band calibration algorithm was implemented. Thus, there may be substantial changes in the SWIR band reflectance and radiance from previous releases. 6. Error messages were upgraded to include the LUN of the input or output file when trying to open/close files and for nearly all I/O operations. Fatal error messages also write out operator actions. 7. The operator actions (in file "error_messages.txt") were updated. 8. The computation of variance for computing "Noise in ..." metadata was found to be inaccurate at the float32 precision level and was upgraded. Thus, some substantial changes in these metadata items may occur from the previous release. 9. Added logic to allow the L1A sector DN values to be the HDF fill value of -32767 as an alternative missing DN flag. This is due to a defect in the MOD_PR01 code when there are completely missing scans (they do not set the value of DN to -1 as described in file specs). 10.Implemented code which assumes that the SD Sun azimuth and SD Sun zenith angles in the MOD03 product are in radians rather than in degrees. This is because the current MOD03 product writes these angles in radians, which conflicts with the MOD03 file spec. CCR 512 has been filed to change the file spec to describe these angles in radians rather than degrees. Depending on the resolution of CCR 512, the assumption may be incorrect. The only item in the L1B output products that may be affected is bit 11 of the Bit QA Flags. v2.2.0 5/30/1999 ============================================================================ The algorithms for calculate average background DN were changed. The OBC file specification was changed. The uncertainty algorithms was changed. Algorithm about saturation of reflective band detectors was added. Reflectance and radiance calibration scales of reflective bands algorithm was changed. LUTs were changed due to the new algorithms. The scientists in MCST have better understanding of the system and believe these changes will make the calibration more accurate. The output product for EV files is the same. v2.1.5 4/28/1999 ============================================================================ 1. Fixed a bug by changing some variable type from float32 to int32. The output QAPERCENTMISSINGDATA, QAPERCENTINTERPOLATEDDATA and QAPERCENTOUTOFBOUNDSDATA will be within the range[0,100] after this bug was fixed. v2.1.4 4/2/1999 ============================================================================ 1. Changed swir band lookup table values to turn off the swir band correction. v2.1.3 3/30/1999 ============================================================================ 1. Assigned realistic numbers in the LUTs so that the output L1B products have reasonable radiances. Output product changed. (a) Incorporated new R_star and L_sd values in LUTs. (b) DN_to_DN_star: The averaged space-view DN is now being subtracted from DN'. The RVS term involving the space view was removed. (c) Emissive_Cal: There was a bug in initializing dn_ev involving the block for ADC correction. After the "if () { ...}", we added: else { dn_ev = DN_ev; } which resolved the issue. 2. PreprocessP.h: Reset macro "FIRST_SV_1km_FRAME_TO_USE from 0 to 10. (no significant change to product). 3. Get_All_Emiss_Coeff: There was a bug in the computation of average b1 for each scan: it was missing the divide by num_scans. This has been fixed. (no significant change to product). 4. Get_L1B_SD_SRCA_BB_SV: There was an indexing problem in how the arrays of the DNs were being filled. This has been corrected. As a consequence of fixing this, we removed an extremely inefficient array so that our stack size requirement dropped from 161 MB to 114 MB. This changes the OBC product only. 5. MCF Files: Made minor changes to the MCF file in order to clean up some Metadata. 6. Metadata.c: The name of the global attribute: "Post Processing Indicates Bad Data" (the way the code implements) was changed to "Post Processing Indicates Bad data" (the way the file specs read)