Methodology

Results from version 2.2.8

Our focus is on extending the MODIS-heritage cloud detection approach in order to generate appropriate climate data records for clouds. With the previous call, we developed a common cloud mask for MODIS and VIIRS observations that operates only on spectral channels available from both instruments (MVCM). Tables below present a two-month validation of MODIS_MVCM and VIIRS_MVCM results in terms of the agreement with classifications from collocated CALIOP observations. There are differences in the results, although the algorithm is identical for both datasets.

“HR Cloud” is the cloudy hit rate, “HR Clear” is the clear sky hit rate, and “HR Comb” is the combined cloudy plus clear hit rate.  “H-K SS” is the Hansen-Kuiper Skill Score that penalizes one heavily for false positives/negatives.

January

2013

MYD35 vs. CALIOP

MVCM Aqua vs. CALIOP

MVCM NPP vs. CALIOP

(5 minute filter)

Scene

Type

HR Cloud

HR Clear

HR Comb

H-K SS

HR Cloud

HR Clear

HR Comb

H-K SS

HR Cloud

HR Clear

HR Comb

H-K SS

Global

87.8

87.8

87.8

75.6

86.9

85.9

86.5

72.7

87.4

82.5

85.8

69.9

60N-60S

92.0

85.1

89.9

77.1

91.5

84.6

89.4

76.1

91.4

82.1

88.4

73.5

Global

Day

91.1

88.7

90.3

79.8

91.4

87.1

89.9

78.5

89.8

85.3

88.2

75.1

60S-60N

Day

92.6

86.4

90.5

79.8

93.2

84.8

90.3

78.0

92.8

82.4

89.2

75.2

Global

Night

84.8

86.8

85.4

71.6

82.7

84.5

83.3

67.2

85.3

79.4

83.4

64.7

60S-60N

Night

91.4

83.4

89.2

74.7

89.7

84.4

88.3

74.2

90.0

81.8

87.7

71.8

Global

Water

91.0

87.0

89.9

78.0

91.6

84.2

89.6

75.8

91.4

81.7

88.6

73.1

Global

Water Day

94.7

87.8

92.8

82.6

96.2

81.9

92.3

78.1

95.1

81.8

91.2

76.9

60S-60N

Water Day

94.1

88.2

92.3

82.3

96.0

82.4

91.9

78.4

94.9

82.8

90.9

77.7

Global

Water Nt

87.6

86.3

87.2

73.9

87.4

86.4

87.1

73.8

88.1

81.6

86.4

69.7

60S-60N

Water Nt

93.2

82.8

90.8

76.0

92.9

84.0

90.8

76.9

91.4

81.7

89.1

73.1

Global

Land

79.5

88.8

83.7

68.2

74.0

87.9

80.4

61.9

77.0

83.5

79.8

60.5

Global

Land Day

81.9

89.6

85.7

71.5

78.8

92.4

85.5

71.2

76.8

89.4

82.6

66.2

60S-60N

Land Day

87.9

83.1

85.8

71.0

83.8

89.3

86.2

73.1

85.7

81.5

84.0

67.2

Global

Land Nt

77.1

87.6

81.4

64.7

69.2

81.7

74.4

50.9

77.3

76.0

76.8

53.4

60S-60N

Land Nt

85.6

84.2

85.1

69.8

79.8

84.9

81.8

64.7

85.9

81.9

84.3

67.8

Polar

77.1

92.6

83.2

69.6

75.1

88.1

80.2

63.2

78.3

83.1

80.0

61.4

Polar Day

86.7

93.8

89.5

80.5

86.4

92.1

88.7

78.5

82.5

92.0

86.0

74.5

Polar Nt

69.2

91.6

78.0

60.8

65.8

84.8

73.3

50.6

74.8

75.9

75.2

50.7

Arctic Nt

63.2

92.3

75.9

55.5

58.0

89.4

71.7

47.4

67.7

82.8

73.9

50.5

Antarctic

Day

86.2

94.9

89.7

81.1

87.3

92.3

89.3

79.5

83.3

94.3

87.3

77.7

Desert Day

81.7

86.8

84.5

68.5

79.7

88.4

84.8

68.1

82.4

84.8

83.7

67.2

Desert Nt

78.4

86.3

82.5

64.5

73.4

85.4

79.8

58.9

81.1

84.9

83.1

66.0

 

 

July

2013

MYD35 vs. CALIOP

MVCM Aqua vs. CALIOP

MVCM NPP vs. CALIOP

(5 minute filter)

Scene

Type

HR Cloud

HR Clear

HR Comb

H-K SS

HR Cloud

HR Clear

HR Comb

H-K SS

HR Cloud

HR Clear

HR Comb

H-K SS

Global

88.9

85.7

87.9

75.0

88.3

84.8

87.2

73.1

88.7

83.3

87.0

72.0

60N-60S

92.1

86.5

90.3

78.6

92.5

85.4

90.2

77.9

91.5

85.7

89.7

77.2

Global

Day

92.0

85.8

90.0

77.9

92.4

83.3

89.5

75.7

90.5

84.0

88.4

74.6

60S-60N

Day

92.0

88.0

90.6

79.9

93.0

85.0

90.2

78.0

92.3

85.6

90.0

78.0

Global

Night

85.9

85.5

85.8

71.4

84.2

86.4

84.9

70.6

87.1

82.5

85.7

69.6

60S-60N

Night

92.3

84.8

90.1

77.1

92.1

85.9

90.3

78.0

90.8

85.8

89.4

76.7

Global

Water

91.2

81.5

88.8

72.7

91.5

79.3

88.4

70.8

91.1

79.3

88.1

70.4

Global

Water Day

94.1

81.4

90.6

75.5

94.9

77.5

90.0

72.4

92.9

80.3

89.3

73.2

60S-60N

Water Day

93.7

84.3

90.8

77.9

94.8

80.1

90.3

74.9

94.0

82.4

90.4

76.4

Global

Water Nt

88.5

81.6

86.9

70.1

88.2

81.6

86.7

69.7

89.5

78.2

87.0

67.6

60S-60N

Water Nt

93.3

80.4

90.3

73.4

93.0

82.4

90.5

75.4

91.4

81.7

89.1

73.0

Global

Land

82.2

90.8

86.0

73.0

78.6

91.9

84.5

70.4

81.3

88.3

84.4

69.6

Global

Land Day

86.1

92.7

88.8

78.8

85.1

92.7

88.3

77.9

82.6

90.7

86.1

73.4

60S-60N

Land Day

86.3

94.4

90.0

80.7

86.5

94.1

90.0

80.6

85.5

92.2

88.6

77.7

Global

Land Nt

78.4

89.2

83.5

67.6

72.2

91.2

81.2

63.3

80.3

86.7

83.2

66.9

60S-60N

Land Nt

88.2

91.0

89.5

79.2

88.2

91.1

89.5

79.3

88.9

91.9

90.2

80.6

Polar

81.6

83.7

82.3

65.4

78.7

83.4

80.1

62.1

82.5

77.6

81.0

60.2

Polar Day

92.1

79.1

88.8

71.2

91.3

77.8

87.8

69.1

86.9

79.4

84.9

66.3

Polar Nt

70.5

86.9

76.2

57.4

65.2

87.3

72.9

52.5

78.5

76.3

77.8

54.8

Arctic Day

92.1

79.1

88.8

71.2

91.3

77.8

87.8

69.1

86.9

79.5

84.9

66.4

Antarctic

Night

68.8

87.3

75.3

56.1

63.1

87.8

71.8

50.9

77.4

77.8

77.5

55.2

Desert Day

71.5

98.1

90.3

69.6

72.8

97.8

90.4

70.6

72.5

97.5

90.3

69.9

Desert Nt

72.9

95.3

86.6

68.1

73.3

94.9

86.5

68.2

75.5

94.3

86.6

69.8

 

Preliminary results

The table below presents results from the initial proposal and is a two-month validation of MODIS_MVCM and VIIRS_MVCM results in terms of the agreement with classifications from collocated CALIOP observations. There are differences in the results, although the algorithm is identical for both datasets.

VIIRS accuracies are generally lower than MODIS because VIIRS thresholds are not yet optimally tuned. Also, collocated VIIRS pixels can be farther from nadir than Aqua pixels, leading to more pixels being classified as cloudy [Maddux et al., 2010]. In general, MVCM accuracies are not as high as the full MYD35 algorithm because the infrared absorption channels are not included in the MVCM. Finally, we note that January land accuracies are lower than for July because of extensive NH snow/ice cover. Further development/testing of the MVCM is needed.

The MODIS cloud mask includes individual bits that describe the results of various tests. Exploration of the bits for land nighttime scenes indicates that no individual test dominates; however, the brightness temperature difference between the 11 μm and 7.3 μm channels (the latter not available on VIIRS) provides valuable information. Validated with 7-years of collocated CALIOP data, this brightness temperature difference test has a 56% hit rate with only a 1.4% disagreement on detecting clear scenes correctly. Therefore, in addition to MVCM, we will develop a new algorithm for NPP using collocated CrIS and VIIRS observations to better simulate MODIS observations. Referred to as the MOD35-Consistent Cloud Mask (MCCM), we expect this to provide improved continuity with the standard MODIS cloud mask. CrIS will be spectrally integrated to simulate the MODIS absorption channels (water vapor and CO2) and then merged with the VIIRS spectral channels into a common L1B sounder-imager IFF file (referred to as an SI-IFF file, details in Sect. 3.1.3). This continuity cloud mask will be built on the framework of the existing MODIS cloud mask algorithm. The SI-IFF files currently produced at the A-PEATE simulate MODIS CO2 absorption channels; we will add the missing IR water vapor absorption channels to those file structures.We have run the algorithm on 6 years of MVCM lidar data from June 2006, to November 2012 the results are shown below.  The global results are reasonable, for example the boundary layer is higher over the deserts in summer than in the winter, though the SAAL of some deserts seems a bit low in altitude. The retrieved SAAL are reasonable over the oceans, being generally less than 2 km. A publication is being prepared and will be available from this web site that describes the algorithm, the validation apporach and applications.


Table: Agreement (%) in cloud detection between CALIOP and the common MODIS/VIIRS Cloud Mask (MVCM) applied to Aqua MODIS and NPP VIIRS Observations.

The results of the above table represent the type of independent validation we will pursue. Further, there are a number of MODIS cloud detection validation publications [e.g., Frey et al., 2008; Ackerman et al., 2008; Holz et al., 2008; Maddux et al., 2010; and Liu et al, 2010]. The publications demonstrate that cloud detection assessment/improvement requires: (1) comparison with collocated measurements from standard observations (lidars, radars, radiosondes), (2) continued monitoring of instrument calibration and its impact on derived products,
and (3) trend analysis to identify unusual events and artifacts.

References

Ackerman, S. A., K. I. Strabala, W. P. Menzel, R. A. Frey, C. C. Moeller, and L. E. Gumley Discriminating Clear-sky from Clouds with MODIS, 1998: J. Geophys. Res., 103, D24, 32,141.

Ackerman, S. A., R. E. Holz, R. Frey, E. W. Eloranta, B. Maddux, and M. McGill, 2008: Cloud Detection with MODIS: Part II Validation, JTECH.25, 1073-1086.

Baum, B. A., W. P. Menzel, R. A. Frey, D. Tobin, R. E. Holz, Ackerman, S. A., A. K. Heidinger, and P. Yang, 2012: MODIS cloud top property refinements for Collection 6. J. Appl. Meteor. Clim., 51, 1145-1163.

Frey, R. A., S. A. Ackerman, Y. Liu, K. I. Strabala, H. Zhang, J. Key and X. Wang, 2008: Cloud Detection with MODIS, Part I: Recent Improvements in the MODIS Cloud Mask, JTECH 25, 1057-1072.

Maddux, B. C., S. A. Ackerman, and S. Platnick (2010), Viewing Geometry Dependencies in MODIS Cloud Products, J Atmos Oceanic Tech, 27(9), 1519–1528, doi: 10.1175/2010JTECHA1432.1.