2.1.

Steps in the Version-5 and Version-6 Retrieval Algorithms

This section gives an overview of the different steps used in both the AIRS science team version-5 and version-6 retrieval algorithms. A much more detailed description of all the steps used is given by Olsen et al.⁹ Retrievals of all geophysical parameters are physically based and represent states $X_{j,c}$ determined for case $c$ that best matches a set of clear column radiances ${\widehat{R}}_{i,c}$ for the subset of AIRS channels $i$ used in the retrieval process. Retrievals of geophysical parameters are performed sequentially, that is, only a subset of the geophysical parameters within the state $X_j$ is modified from that of the incoming state $X_j^0$ in a given step. A numerical weather prediction model forecast is not used in the retrieval procedure, except for use of the forecasted surface pressure $p_{\text{surf}}$ as the lower pressure boundary when computing radiances $R_i(X)$ expected for a given geophysical state $X_j$. In the case of AIRS only retrievals, a general circulation model forecast is also used in the specification of surface class over potentially frozen ocean.

In version-5, the major steps in the physical retrieval process were done as follows: (1) A start-up procedure, involving use of a cloudy regression followed by a clear regression, was used to generate the initial state $X^0$. (2) Initial clear column radiances ${\widehat{R}}_i^0$ were generated for all channels $i$ using the initial cloud-clearing coefficients, which were generated based on observed radiances in an ensemble of cloud-clearing channels along with the initial state $X^0$. (3) A subsequent physical retrieval procedure was performed, starting with the initial guess $X^0$, in which AIRS/AMSU observations were used to retrieve (a) surface skin temperature $T_s$, surface spectral emissivity ${\epsilon }_{\nu }$, and surface bidirectional reflectance of solar radiation ${\rho }_{\nu }$; (b) atmospheric temperature profile $T(p)$; (c) atmospheric moisture profile $q(p)$; (d) atmospheric ozone profile ${\mathrm{O}}_3(p)$; (e) atmospheric CO profile $\mathrm{CO}(p)$; (f) atmospheric ${\mathrm{CH}}_4$ profile ${\mathrm{CH}}_4(p)$; and (g) retrieve cloud properties and compute OLR. These steps were done sequentially, solving only for the variables to be determined in each retrieval step while using previously determined variables as fixed with an appropriate uncertainty attached to them, which was accounted for in the channel noise covariance matrix used in that step.² The objective in each step [(a) to (f)] was to find solutions for which computed radiances best match ${\widehat{R}}_i$ for the subset of channels selected for use in that step, bearing in mind the channel noise covariance matrix.² Steps (a) to (f) were ordered so as to allow for selection of channels in each step that are primarily sensitive to variables to be determined in that step or in a previous step, and are relatively insensitive to other parameters. Separation of the problem in this manner allowed for the problem in each step to be made as linear as possible. Step (g) was performed after the surface and atmospheric conditions have been determined using a selected set of observed radiances $R_i$, rather than clear column radiances ${\widehat{R}}_i$ as used in the other physical retrieval steps.

In version-6, there are slight modifications to the sequence of steps used in version-5 because there are two new steps performed in the retrieval sequence. In version-5, physical retrieval step (a) used channels in both the longwave and shortwave window regions and simultaneously solved for surface skin temperature $T_s$, shortwave surface spectral emissivity ${\epsilon }_{\mathrm{sw}}(\nu )$, effective shortwave surface spectral bidirectional reflectance ${\rho }_{\mathrm{sw}}(\nu )$, and longwave surface spectral emissivity ${\epsilon }_{\mathrm{lw}}(\nu )$. In version-6, only shortwave window channels are used in this retrieval step to simultaneously determine $T_s$, ${\epsilon }_{\mathrm{sw}}(\nu )$, and ${\rho }_{\mathrm{sw}}(\nu )$. As with regard to the determination of tropospheric temperature profile, use of only shortwave sounding channels is a superior approach to determine surface skin temperature because errors in cloud-clearing coefficients result in smaller errors in shortwave clear column brightness temperatures as compared to errors in longwave brightness temperatures. In addition, shortwave window channel observations are much less sensitive to errors in the assumed water vapor profile than are longwave window observations. The longwave surface spectral emissivity ${\epsilon }_{\mathrm{lw}}(\nu )$ is solved in version-6 in a subsequent step using only channels in the longwave window spectral region. This new step is performed after the humidity profile retrieval step has been performed because longwave window radiances can be very sensitive to the amount of atmospheric water vapor. In addition, version-6 contains a new physical retrieval step, performed before the surface temperature retrieval step, in which ${\rho }_{\mathrm{sw}}(\nu )$ is updated from its initial guess value. This additional step is performed only during the day because reflected solar radiation is not present at night.

The steps used in the version-6 AO (AIRS only) algorithm are identical to those in version-6. In version-6 AO, however, no AMSU-A observations are used in any step of the physical retrieval process; nor are they used in the QC methodology, which is otherwise analogous to that used in version-6. In addition, no AMSU-A observations are used in any way in the generation of the version-6 AO Neural-Net initial state $X^0$, which uses coefficients that are trained separately from those of version-6 and are generated without the benefit of any AMSU observations.

2.2.

Improved Version-6 Surface Parameter Retrieval Methodology

In addition to the separation of the retrieval of surface shortwave spectral emissivity and surface longwave spectral emissivity into two separate steps, Version-6 has also improved other details in the retrieval of surface skin parameters. Version-6 uses an improved form of the equation which modifies the retrieved surface spectral emissivity ${\epsilon }_{\nu }$ from its initial guess ${\epsilon }_{\nu }^0$. In version-6, we treat the variable to be modified as $(1-{\epsilon }_{\nu })$ and solve for ${\epsilon }_{\nu }$ according to

In the shortwave surface parameter retrieval step, in which ${\epsilon }_{\nu }$ is retrieved simultaneously with $T_s$ and $\rho$, $k_{\max }$ is set equal to four. The four shortwave functions $F_k(v)$ have values equal to 1 at the four characteristic frequencies 2439.0, 2500.0, 2564.1, and $2631.6{\mathrm{cm}}^{-1}$, respectively. All $F_k(v)$ are set equal to 0 at the frequency in which an adjacent function is equal to 1, and at all frequencies beyond. The first and last functions are set equal to 1 beyond their characteristic frequencies. A corresponding multiplicative form is also used in version-6 to modify ${\rho }_{\nu }$ during the day in the shortwave surface parameter retrieval step according to

In the longwave surface emissivity step, $k_{\max }$ is set equal to six, with analogous spectral shapes and corresponding characteristic frequencies of 769.23, 819.67, 877.19, 980.39, 1111.10, and $1204.80{\mathrm{cm}}^{-1}$, respectively. Therefore, during the day, nine coefficients, one for $\mathrm{\Delta }{T}_s$ where $\mathrm{\Delta }{T}_s$ is the difference of the retrieved value of $T_s$ from its initial guess $T_s^0$, and four values each of $A_k$ and $B_k$ are solved for in the shortwave surface parameter retrieval step, and five parameters are solved for at night. The basic retrieval algorithm methodology¹ works in terms of the principal components of the information content matrix, on a case-by-case basis, and, thus, uses appropriate linear combinations of the functions $F_k$ used in Eqs. (1) and (2). Adding more functions $F_k$ is not beneficial after a point and becomes detrimental in terms of the computer time needed in the physical retrieval step. The numbers of functions used in version-6 in the shortwave and longwave surface emissivity retrieval steps were determined empirically according to how well the retrievals performed.

The initial guess for surface spectral emissivity in both retrieval steps, ${\epsilon }_{\nu }^0$, is set equal to the AIRS science team ocean emissivity model over nonfrozen ocean, which is based on Wu and Smith.¹⁰ Over land and frozen ocean, we set equal ${\epsilon }_{\nu }^0$ to values interpolated from the $1°\times 1°$ monthly mean MODIS science team aqua MODIS MYD11C3 V4.1 monthly gridded emissivity for the year 2008, interpolated by the method of Seeman et al.¹¹ As in version-5, ${\rho }_v^0$ is initially estimated as being equal to $(1-{\epsilon }_v^0)/\pi$, but is then modified in a subsequent retrieval step in version-6, which is performed immediately prior to the shortwave surface parameter retrieval step. In this step, not performed in version-5, ${\rho }_v^0$ is updated in a one parameter physical retrieval step, using the same channels as in the surface parameter retrieval step, according to

3.1.

Cloud Clearing and Temperature Profile Retrieval

Following cloud-clearing theory,^5,6 coefficients needed to generate clear column radiances for all channels are determined using observations in select longwave channels whose radiances are sensitive to the presence of clouds. Version-6 uses 57 channels to derive the coefficients, which are used to generate clear column radiances for all channels.¹ These channels, which we mark by yellow stars in Fig. 1, range from 701 to $1228{\mathrm{cm}}^{-1}$. The cloud clearing channels are the same channels used in a subsequent cloud parameter retrieval step. The temperature profile retrieval step uses 37 channels between 2358 and $2395{\mathrm{cm}}^{-1}$ that are sensitive to both stratospheric and tropospheric temperatures. During the day, radiances in these channels, which are also used in version-5,³ are sensitive to effects of solar radiation reflected in the direction of the satellite by clouds and by the surface, and also to effects of non-LTE. Effects of non-LTE on AIRS radiances are accounted for in the AIRS RTA,⁴ and effects of solar radiation reflected by clouds and by the surface are well accounted for by different aspects of the retrieval algorithm.Version-6, like version-5, also uses 53 stratospheric sounding channels between 662 and $713{\mathrm{cm}}^{-1}$ that are relatively insensitive to cloud contamination. Longwave channels that are more sensitive to cloud contamination are not used in the temperature profile retrieval step. We indicate the channels used in the determination of temperature profile by red stars in Fig. 1. Version-6 also includes 24 additional channels in the temperature profile retrieval step between 2396 and $2418{\mathrm{cm}}^{-1}$, also shown in red, which are used in both the temperature profile step and the surface skin temperature retrieval step. Version-6 uses AMSU-A channels 3, 6, and 8 to 14 in the temperature profile retrieval step as well, while version-6 AO does not use these or any AMSU channels. AMSU-A channel 7 was noisy at launch and was never used in any step of the retrieval process. Version-5 included AMSU-A channels 4 and 5 in the temperature profile retrieval step, but those channels subsequently became noisy and neither is used in version-6. In addition, version-5 included 12 AIRS channels between 2198 and $2252{\mathrm{cm}}^{-1}$ in the temperature profile retrieval step that are no longer used in version-6. These channels are sensitive to absorption by ${\mathrm{N}}_2\mathrm{O}$ and were found to contribute to the spurious negative mid-tropospheric temperature trend found in version-5 because increases in ${\mathrm{N}}_2\mathrm{O}$ concentration over time are not accounted for in either the version-5 or version-6 retrieval algorithms.

3.2.

Surface Skin Temperature and Longwave Spectral Emissivity Retrievals

Unlike in version-5, the surface skin temperature retrieval and longwave spectral emissivity retrieval are done in separate steps in version-6. The surface skin temperature retrieval step uses 36 channels between 2420 and $2664{\mathrm{cm}}^{-1}$, which we show by light blue stars in Fig. 1, along with the 24 highest frequency (red stars) channels, which are also used in the temperature profile retrieval step. These 60 channels are used to determine $T_s$ simultaneously with four independent pieces of information about surface shortwave spectral emissivity and, during the day, four additional independent pieces of information about shortwave surface bidirectional reflectance as shown in Eqs. (1) and (2). Surface longwave spectral emissivity is determined using 77 channels between 758 and $1250{\mathrm{cm}}^{-1}$, which we indicate by purple stars in Fig. 1. In this step, coefficients of six longwave emissivity perturbation functions are solved for, with $T_s$ being held fixed at the value determined from the previously performed skin temperature retrieval step.

3.3.

Constituent Profile Retrievals

As in version-5, constituent profile retrievals are performed in separate subsequent steps, each having its own set of channels and functions. Figure 1 indicates, by stars of different colors, the version-6 channels used in each of these retrieval steps. The $q(p)$ retrieval (pink stars) uses 41 channels in the spectral ranges from 1310 to $1605{\mathrm{cm}}^{-1}$ and 2608 to $2656{\mathrm{cm}}^{-1}$; the ${\mathrm{O}}_3(p)$ retrieval (green stars) uses 41 channels between 997 and $1069{\mathrm{cm}}^{-1}$; the $\mathrm{CO}(p)$ retrieval (gray stars) uses 36 channels between 2181 and $2221{\mathrm{cm}}^{-1}$; and the ${\mathrm{CH}}_4(p)$ retrieval (brown stars) uses 58 channels between 1220 and $1356{\mathrm{cm}}^{-1}$. The version-6 $q(p)$ retrieval step, including the channels used, is essentially unchanged from that used in version-5 other than the use of the Neural-Net first guess $q^0(p)$. Some small modifications have been made to the details of the trace gas retrieval steps. Version-6 trace gas retrieval methodology and results are not treated in this paper.

4.1.

Ocean Surface Skin Temperature T_s and Surface Spectral Emissivity ${\epsilon }_{\nu }$

The term $T_s$ refers to surface skin temperature over all surfaces. We also refer to values of $T_s$ over nonfrozen ocean as sea surface temperature (SST). Figure 2 shows counts of QC’d values of SSTs over the latitude range of 50°N to 50°, as a function of the difference between $T_s$ and truth for the nine-day evaluation period, where truth for $T_s$, and for most other geophysical parameters, is taken from the ECWMF 3-h forecast field. We show the counts of version-5 retrievals in red and pink, version-6 retrievals in dark blue and light blue, and version-6 AO retrievals in black and gray. The lighter shade of each color shows counts of the best quality $T_s$ retrievals, with $\mathrm{QC}=0$, that pass the DA error estimate thresholds. The darker shade of each color shows counts of combined best and good quality $T_s$ retrievals, including cases with $\mathrm{QC}=0$ and also cases with $\mathrm{QC}=1$ that pass the looser climate error estimate thresholds but do not pass the tighter DA thresholds. Ocean $T_s$ retrievals with $\mathrm{QC}=0$ or 1 are the ensemble used over ocean in the generation of the level-3 surface skin temperature product used for climate studies. Figure 2 contains statistics for each set of retrievals showing the mean difference from ECMWF, the standard deviation (STD) of the ensemble differences, the percentage of all possible cases included in the QC’d ensemble, and the percentage of all accepted cases with absolute differences from ECMWF of more than 3 K from the mean difference, which we refer to as outliers. Version-6 QC’d retrievals accept considerably more cases than version-5 and have much lower STDs of the errors as well. In both ensembles, the percentage of outliers grows with loosening the QC thresholds, as expected. The percentage of version-6 outliers with $\mathrm{QC}=0$, 1 is somewhat larger than that in version-5, but the version-6 yield with $\mathrm{QC}=0$, 1 is more than twice as large as that of version-5. It is noteworthy that version-6 retrievals with $\mathrm{QC}=0$ have a much smaller percentage of outliers than do version-5 retrievals with $\mathrm{QC}=0$, 1, along with substantially higher yield. Statistics of QC’d version-6 AO retrievals are very similar to those of version-6.

Fig. 2

Statistics of quality controlled (QC’d) sea surface temperature (SST) differences from ECMWF truth for version-5, version-6, and version-6 AIRS only (AO) using each data assimilation QC and climate QC thresholds.

Figure 3 shows the spatial distribution, over the latitude range from 60°N to 60°S, of the nine-day mean differences of the level-3 oceanic SST products from collocated ECMWF values for both version-6 and version-5. The values shown in a given grid box are the average values for that grid box of all cases in which the SST retrieval was accepted using climate QC either at 1:30 a.m. or 1:30 p.m. The oceanic grid boxes shown in gray indicate grid boxes in which not a single value of climate QC’d SST occurred for all 18 possible cases (nine days, twice daily). Figure 3 represents the spatial coverage and accuracy of a pseudo nine-day mean level-3 product. The results shown in Fig. 3 are referred to as a pseudo nine-day mean product because they do not represent those of a typical nine-day level-3 product in which the nine days used are consecutive. Figure 3 provides valuable information nonetheless. The version-6 pseudo nine-day mean level-3 product is significantly improved over the version-5 product in terms of accuracy as compared to ECMWF and also has almost complete spatial coverage, with 99.55% of possible oceanic grid points covered, while version-5 has only 91.28% oceanic spatial coverage, and is marked by gaps in areas that had significant cloud cover in each of the 18 time periods included in the nine-day mean field.

Fig. 3

Nine-day mean difference of version-6 and version-5 level-3 SST products from collocated ECMWF truth for ocean grid points between 50°N and 50°S.

Figures 4(a) and 4(b) show the mean difference of the retrieved ocean surface emissivity ${\epsilon }_{\nu }$ from that of the AIRS science team ocean surface emissivity model as a function of satellite zenith angle for $\nu =950{\mathrm{cm}}^{-1}$ and $\nu =2400{\mathrm{cm}}^{-1}$, and Figs. 4(c) and 4(d) show the STDs of the retrieved values at a given zenith angle. The two channels shown are in the longwave and shortwave window regions, respectively. In these figures, we show statistics separately for a.m. orbits in dark colors and p.m. orbits in light colors. In both the longwave and shortwave window regions, version-6 (as well as version-6 AO) retrieved ocean spectral emissivities as a function of satellite zenith angle are very close to the values expected using the AIRS science team ocean surface emissivity model. Differences of version-6 retrieved values of ${\epsilon }_{\nu }$ from the ocean emissivity model are much smaller than those of version-5. Version-5 retrieved values of ${\epsilon }_{\nu }$ also showed a large spurious feature during the day in the vicinity of satellite zenith angle $-18.24\mathrm{deg}$ at both frequencies. This spurious feature occurs at the viewing angle at which maximum sunglint appears in the FOR. In addition to being more accurate in the mean sense, the retrieved values of ${\epsilon }_{\nu }$ are much more stable in version-6 compared to those of version-5, as evidenced by the much lower standard deviations of their values as shown in Figs. 4(c) and 4(d). There is no appreciable difference between version-6 and version-6 AO results related to retrieved ocean values of ${\epsilon }_{\nu }$.

Fig. 4

Statistics related to ocean surface emissivity as a function of satellite zenith angle.

Figures 4(a) and 4(b) show that daytime and nighttime version-6 retrieved values of ocean surface emissivity are not only close to those of the ocean emissivity model, which is a good measure of truth, but also very close to each other, as expected. Over land, surface spectral emissivity values change rapidly in space, and time as well, as a result of variations in ground cover, such as vegetation, rock, and soil types, and even snow cover. At a given location and day, these values should not change appreciably from day to night, however. Figure 5 shows the nine-day mean 1:30 a.m./1:30 p.m. differences of retrieved values of ${\epsilon }_v$ at 950 and $2400{\mathrm{cm}}^{-1}$ over land obtained using the version-6 and version-5 retrieval systems. As in the case of ocean, day/night differences of version-6 retrieved land surface emissivity are much smaller than those of version-5, as they should be.

Fig. 5

Difference of 1:30 a.m. and 1:30 p.m. nine-day mean land level-3 emissivity products shown at 950 and $2400{\mathrm{cm}}^{-1}$ for each of version-6 and version-5.

4.2.

T(p) Retrieval Accuracy as a Function of Yield

Unlike the major improvements made in version-6 methodology to retrieve surface skin parameters, the fundamentals of the methodology used in version-6 to retrieve temperature profile $T(p)$ from AIRS cloud cleared radiances $\widehat{R_i}$ are basically the same as those used in version-5. Neverthless, version-6 temperature profile accuracy as a function of cloud cover is significantly improved over that of version-5 for a number of reasons. Foremost among these is the use of the Neural-Net temperature profile first guess, which is more accurate than the regression-based version-5 first guess, especially under more stressing cloud conditions. Version-6 retrieved temperature profiles also benefit from the improvement in version-6 surface skin parameters as well as improved version-6 QC methodology.

Figure 6 shows statistics of the differences of QC’d version-5 and version-6 $T(p)$ retrievals from collocated ECMWF truth for a global ensemble of cases taken over the nine focus days. Figure 6(a) shows the percentage of QC’d cases accepted as a function of height, Fig. 6(b) shows RMS differences of 1 km layer mean temperatures from collocated ECMWF truth, and Fig. 6(c) shows biases of QC’d 1 km layer mean differences from ECMWF. Statistics are shown for seven sets of results. We show in red the results for version-5 retrievals using three different QC procedures, in blue the results for version-6 retrievals using two different QC procedures, and in black the results for version-6 AO retrievals using QC procedures analogous to those of version-6.

Fig. 6

Global mean statistics of QC’d version-5, version-6, and version-6 AO temperature profiles, compared to ECMWF truth, using different QC thresholds.

QC procedures used in both version-5 and version-6 designate two characteristic pressures for each temperature profile, $p_{\text{best}}$ and $p_{\text{good}}$. These pressures are computed using thresholds of temperature profile error estimates $\delta T(p)$. The Appendix describes the manner in which $\delta T(p)$ is computed and how it is used for QC purposes. Version-5 had only one set of $T(p)$ QC error estimate thresholds, called standard thresholds, which were used to define version-5 values of $p_{\text{best}}$, down to which $T(p)$ retrievals were considered to be of highest quality. The version-5 standard thresholds were chosen such that if one utilized only $T(p)$ retrievals down to $p_{\text{best}}$ for each case, this procedure would provide a middle ground of keeping retrievals with highest accuracy, which would be optimal for DA purposes on one hand, and keeping retrievals with the highest yield (best spatial coverage), optimal for climate purposes on the other hand. Experience using version-5 products showed that standard QC thresholds were optimal for neither purpose. For example, DA experiments assimilating version-5 retrievals down to a value of $p_{\text{best}}$ defined using a tighter set of thresholds than those of the official version-5 system, referred to as tight thresholds,¹² resulted in significantly improved forecasts compared to assimilation of $T(p)$ retrievals down to values of $p_{\text{best}}$ computed using the looser standard QC thresholds. The dotted red lines in Fig. 6 show acceptance yield and accuracy of version-5 retrievals down to $p_{\text{best}}$ as defined using the tight QC thresholds (not officially part of version-5). The solid red lines in Fig. 6 show equivalent statistics for the ensemble of version-5 retrievals down to $p_{\text{best}}$ as computed using the standard thresholds. The global yield of cases in which $p_{\text{best}}$ is equal to the surface pressure $p_{\text{surf}}$, as defined using standard thresholds, is shown in Fig. 6(a) to be $\sim 35\%$. Utilization of an ensemble of retrievals with such a low yield would not be adequate for the generation of level-3 $T(p)$ products with reasonable spatial coverage near the surface. The spatial coverage near the surface of such an ensemble of cases is particularly poor over land and sea-ice. In order to be able to generate level-3 products with reasonable spatial coverage in version-5, an additional case-by-case characteristic pressure, $p_{\text{good}}$, was defined in an ad hoc manner over land and sea-ice for use in the generation of level-3 products. If $p_{\text{best}}$ was at least 300 mb over these domains, $p_{\text{good}}$ was set to be equal to the surface pressure $p_{\text{surf}}$. Otherwise, $p_{\text{good}}$ was set equal to $p_{\text{best}}$. Version-5 level-3 products for $T(p)$ at a given pressure $p$ were generated using all cases for which $p$ was $\le p_{\text{good}}$. Over nonfrozen ocean, there was no need to include additional cases in the generation of a level-3 product with good spatial coverage, and $p_{\text{good}}$ over nonfrozen ocean was always set equal to $p_{\text{best}}$. The dashed red lines in Fig. 6 show statistics for version-5 retrievals, which are included down to $p_{\text{good}}$, i.e., statistics for the ensemble of cases used in the generation of the version-5 level-3 $T(p)$ products. Global yield of version-5 cases down to $p_{\text{good}}$ at the surface has increased to $\sim 60\%$, and the global mean RMS error of version-5 cases down to $p_{\text{good}}$ has increased to 2.7 K near the surface.

Having learned from the experience with version-5 QC methodology based on the use of a single set of $T(p)$ thresholds for both DA and climate applications, version-6 defines $p_{\text{best}}$ and $p_{\text{good}}$ independent of each other based on the use of two different sets of QC thresholds. A tight set of DA $T(p)$ thresholds, optimized for DA purposes (cases with $\mathrm{QC}=0$), was used to derive $p_{\text{best}}$, and a substantially looser set of climate $T(p)$ thresholds, optimal for climate purposes (cases with $\mathrm{QC}=1$), was used to derive $p_{\text{good}}$. The solid blue and black lines in Fig. 6 show statistics for version-6 and version-6 AO results, respectively, using their appropriate sets of DA QC thresholds, including all cases down to $p_{\text{best}}$, and the blue and black dashed lines show results using the appropriate climate thresholds, including all cases down to $p_{\text{good}}$. As in version-5, version-6, and version-6 AO, level-3 gridded products utilize all cases passing climate QC, that is, all cases down to $p_{\text{good}}$. The Appendix provides detailed information about the $T(p)$ QC methodologies used in version-5 and version-6, as well as the different thresholds used in each version.

In version-5, all retrievals were either accepted or rejected above 70 mb based on the use of different types of tests, even before applying the error estimate based QC procedures.³ One of the tests that disqualified the entire temperature profile, and flagged the entire profile with $\mathrm{QC}=2$ (do not use), was that the retrieved cloud fraction is $>90\%$. Roughly 83% of version-5 retrievals passed the initial screening procedure, with none of them occurring under near overcast conditions. Version-5 retrievals with tight QC have considerably lower yield than those with standard QC below 200 mb, with correspondingly smaller RMS errors on the order of 1K beneath 300 mb. The ensemble of version-5 retrievals used to generate level-3 $T(p)$ products (dashed red line) differs from that of those accepted down to $p_{\text{best}}<300\mathrm{mb}$. The yield near the surface is $\sim 60\%$, which is better for the generation of level-3 products, but the RMS error for this larger ensemble of cases with $\mathrm{QC}=0$ or $\mathrm{QC}=1$ is much larger near the surface than those with $\mathrm{QC}=0$. While RMS errors of retrievals increased with increasing yield, there is no appreciable difference in version-5 bias errors, compared to ECMWF, found using any of the three version-5 ensembles of cases shown in Fig. 6.

Version-6 does not apply any test that eliminates the entire temperature profile, other than the requirement that the retrieval runs to completion. Version-6 retrievals using DA thresholds ($\mathrm{QC}=0$) have a yield much higher than those passing version-5 tight thresholds down to $\sim 700\mathrm{mb}$ and have RMS errors $<1\mathrm{K}$ at all levels, which has been found to be optimal for DA purposes.¹² Among other benefits from the perspective of DA is that version-6 will allow for the assimilation of AIRS temperature products above the clouds, both in storms, as well as under overcast conditions in general. The yield of version-6 retrievals with climate QC ($\mathrm{QC}=0$, 1) is extremely high throughout the atmosphere, with a value of $\sim 80\%$ at the surface. Achievement of this very high yield is extremely valuable in the generation of more representative version-6 level-3 products used for climate studies. RMS errors of version-6 retrievals with climate QC are better than, or comparable to, those of version-5 with standard QC down to the surface, and significantly better than that of the ensemble of version-5 retrievals used to generate level-3 products. Results for version-6 AO using either QC procedure are roughly comparable to those of version-6, but with slightly lower yields near the surface.

The results shown in Fig. 6 are for all accepted retrievals, whether they were obtained during the night, when non-LTE and reflected solar radiation do not affect shortwave radiances, or during the day, when these effects must be well accounted for in order to produce accurate retrievals. Figure 7 is analogous to Fig. 6, but breaks down the version-6 results shown into those obtained during the night (1:30 a.m.) and those obtained during the day (1:30 p.m.). Figure 7 shows that there is essentially no difference in the quality of the results obtained at night and those during the day. This implies that effects of non-LTE on shortwave radiances during the day are well accounted for by the AIRS RTA. This result also implies that cloud cleared radiances obtained during the day are as accurate as those obtained at night and, indeed, that the procedure used to generate cloud cleared radiances also accounts for the effects of solar radiation reflected by clouds in the AIRS FOVs.

Fig. 7

Global nine-day version-6 temperature profile statistics shown separately for daytime and nighttime cases.

Figures 8(a) and 8(b) compare RMS errors of QC’d version-6 and version-5 retrievals with those of their first guesses. The solid and dashed blue and red lines shown in Fig. 8 are identical to those in Fig. 6(b). The RMS errors of the first guesses are shown by light blue lines for version-6 and pink lines for version-5. Figure 8(a) shows that the version-6 retrievals improve on the Neural-Net guess at all pressures $>\sim 150\mathrm{mb}$, especially for the easier ensemble of cases accepted using DA thresholds. Version-6 retrievals are slightly poorer than their first guess above 60 mb, which is at least in part due to the fact that the Neural-Net guess above 60 mb is extremely accurate. Figure 8(b) shows that essentially the same relative result holds for version-5, though in version-5 the retrievals always improve on their first guess, which is less accurate than the Neural-Net, at all levels. In addition, unlike for version-6, the improvement over the first guess is greatest in the mid-troposphere.

Fig. 8

Comparison of the accuracies of QC’d version-6 and version-5 retrieved temperature profiles with those of their initial guesses.

4.2.1.

T(p) retrieved accuracy as a function of cloud fraction

Figure 9(a) shows % yields of version-5 and version-6 retrievals, accepted using version-5 standard QC and version-6 climate QC, respectively, as a function of retrieved cloud fraction at three mid-lower tropospheric pressures, and Fig. 9(b) shows the RMS $T(p)$ errors over three corresponding 1-km layers. Version-6 climate QC yields are much higher than those of version-5 at all cloud fractions, especially at larger cloud fractions. Version-6 RMS errors over these larger ensembles of cases for all cloud fractions are also considerably better than those of the smaller version-5 ensembles. The fact that version-6 retrievals remain accurate and improve over the Neural-Net first guess at larger cloud fractions indicates that the version-6 cloud cleared radiances are accurate as well under more difficult cloud conditions.

Fig. 9

(a) Percent acceptance, using climate QC, of global version-5 and version-6 temperature profiles as a function of retrieved fractional cloud cover at three select pressure levels. (b) RMS difference of version-5 and version-6 1 km layer mean temperatures from colocated truth in three select layers.

4.3.

Retrieval Accuracy of q(p)

The details of the $q(p)$ retrieval step are essentially unchanged from what was done in the $q(p)$ retrieval step both in version-5 and in version-4. Version-7 will address further improvements to be made to the $q(p)$ retrieval algorithm. Nevertheless, version-6 retrieved values of $q(p)$ are improved over those of version-5 as a result of the same factors that led to improved version-6 values of $T(p)$ as compared to version-5: (1) improved surface skin temperatures and spectral emissivities, (2) an improved first guess $q^0(p)$ provided by the Neural-Net start-up system, and (3) improved clear column radiances ${\widehat{R}}_i$. Version-6 retrieved values of $q(p)$ also benefit from improved values of $T(p)$ that are used as input to the $q(p)$ retrieval step.

Figure 10 shows results analogous to those of Fig. 6 comparing QC’d 1-km layer precipitable water to that of collocated values of ECMWF. We show results only up to 200 mb, above which water vapor retrievals are considered to be of minimal validity and are not included in the AIRS science team standard product data set. The relative results comparing version-5 and version-6 $q(p)$ retrievals are analogous to those found for $T(p)$. Version-6 $q(p)$ retrievals with both DA and climate QC are considerably improved over those of version-5 in the lower troposphere. This improvement in the lower troposphere is at least partially a result of the improved values of $T_s$ and ${\epsilon }_{\nu }$ in version-6 compared to version-5. As with $T(p)$, version-6 $q(p)$ retrievals with climate QC are unbiased, have high accuracy, and contain almost complete spatial coverage. Globally, version-6 AO $q(p)$ retrievals are slightly less accurate than those of version-6 near the surface. This difference between results of version-6 and version-6 AO occurs primarily over the ocean and is a result of the benefit over ocean of the 22- and 31-GHz channels of AMSU-A, which are not included in the AIRS only retrieval procedure.

Fig. 10

Global mean statistics of QC’d version-5, version-6, and version-6 AO water vapor profiles, compared to ECMWF truth, using different QC thresholds.

In version-6, $W_{\mathrm{TOT}}$ is flagged to be of highest quality ($\mathrm{QC}=0$) if the water vapor profile has best quality ($\mathrm{QC}=0$) down to the surface and $W_{\mathrm{TOT}}$ is flagged to be of good quality ($\mathrm{QC}=1$) if the water vapor profile has good quality ($\mathrm{QC}=1$) at the surface. This same test is also applied to generate QC flags for (1) surface air temperature, (2) clear sky OLR, (3) ${\mathrm{O}}_3$, ${\mathrm{CH}}_4$, and CO profiles, and (4) surface skin temperatures over land and nonfrozen ocean.

Figure 11 shows the spatial distribution of the pseudo-level-3 nine-day mean field of accepted cases of total precipitable water, $W_{\mathrm{TOT}}$, flagged to be of climate quality ($\mathrm{QC}=0$, 1). The statistics shown for version-6 and version-5 represent the % of grid points containing values, the area weighted global mean difference of the gridded level-3 values of $W_{\mathrm{TOT}}$ from the collocated ECMWF values of $W_{\mathrm{TOT}}$, and the area weighted spatial standard deviation of those values from ECMWF. Statistically, the version-6 pseudo nine-day mean level-3 values of $W_{\mathrm{TOT}}$ are considerably more accurate than those of version-5, especially over land. This improvement of $W_{\mathrm{TOT}}$ is partially a result of improved version-6 surface skin parameters, especially over land. Version-5 used a different procedure to accept those values of $W_{\mathrm{TOT}}$ to be used in the generation of the level-3 product than that used in version-6.

Fig. 11

Difference of climate QC’d nine-day mean total precipitable water (cm) from collocated ECMWF for version-6 and version-5.

4.4.

Yield Trends and Spurious Bias Trends of T(p)

Our research using version-5 retrieved products indicated that QC’d version-5 values of $T(p)$ had a large negative yield trend, as well as spurious bias trends when compared to collocated ECMWF values of $T(p)$. A prime consideration in the finalization of version-6 was to alleviate these negative yield trends and spurious bias trends as much as possible. Figure 12 shows yield trends and temperature bias trends of version-5 retrievals using standard QC, and both version-6 and version-6 AO retrievals using climate QC, as evaluated over the nine days used in all other figures. Figure 12(a) shows that the % yield of accepted version-5 retrievals was decreasing over time (negative yield trend), and Fig. 12(b) shows that version-5 retrievals had substantial negative spurious temperature bias trends in the troposphere, which were in part due to the fact that the regression first guess used in version-5 had a negative tropospheric temperature bias trend itself. A substantial part of the negative yield trend was due to a significant degradation of the noise characteristics of AMSU-5. Version-6 contains modifications that alleviated these problems, one of which is that version-6 no longer uses AMSU-5 at all. Other factors also contributed to the spurious temperature bias trends found in version-5, and these were also corrected in version-6.

Fig. 12

Global mean yield and spurious layer mean temperature bias trends of QC’d version-5, version-6, and version-6 AO retrievals as a function of pressure.

Figure 12 shows that version-6 has eliminated the substantial negative tropospheric temperature profile yield trends, on the order of 2% per year, which were found in version-5. In addition, the version-6 negative $T(p)$ bias trends beneath 500 mb are much smaller than those of version-5, which were as large as $-0.08\mathrm{K}/\mathrm{yr}$. Part of this improvement is due to the fact that the Neural-Net first guess used in version-6 does not appear to have any spurious trends associated with it. In addition, version-6 does not use any channels sensitive to ${\mathrm{N}}_2\mathrm{O}$ in the $T(p)$ retrieval step, which were used in version-5. These channels are no longer used in the temperature profile retrieval step because the concentration of ${\mathrm{N}}_2\mathrm{O}$ has been changing over time and this is not accounted for in the AIRS RTA. There is no appreciable difference between the yield or bias trend results obtained for version-6 and version-6 AO.

4.5.

Comparison of Version-6 and Version-5 Retrieved Values of Cloud Fraction and Cloud Top

4.5.1.

Pressure

The procedure used to derive cloud fraction and cloud top pressure in version-6 is similar to that used in version-5,^1,3 but version-6 has a number of significant improvements. The radiatively effective cloud fraction at frequency $\nu$, $\alpha {\epsilon }_{\nu },$ is given by the product of $\alpha$, the geometric fractional cloud cover of an AIRS FOV as seen from above, and ${\epsilon }_{\nu }$, the cloud spectral emissivity. The AIRS science team cloud parameter retrieval methodology determines only the product of these two terms, $\alpha {\epsilon }_{\nu }$, along with a corresponding cloud top pressure $p_c$, for each of up to two layers of clouds in a given scene.^1,3 A basic simplifying assumption of the cloud retrieval methodology used in both version-5 and version-6 is that the clouds are gray, that is, $\alpha {\epsilon }_{\nu }$ is independent of frequency. Version-5 simultaneously derived 20 parameters for each AIRS FOR, nine effective cloud fractions $\alpha {\epsilon }_1$ and $\alpha {\epsilon }_2$, one pair for each AIRS FOV $\ell$ contained within the AMSU FOR, along with two cloud top pressures $p_{c1}$, and $p_{c2}$ considered to be representative of the pressures of each of the two layers of clouds covering the entire AIRS FOR. In version-6, the cloud parameter retrieval step is performed separately for each AIRS FOV $\ell$ to determine the four parameters, $\alpha {\epsilon }_{1,\ell }$, $\alpha {\epsilon }_{2,\ell }$, $p_{c1,\ell }$, and $p_{c2,\ell }$, in each FOV. A total radiatively effective cloud fraction for the entire FOR, $\alpha \epsilon$, is computed as the average cloud fraction according to

Eq. (4)

$$\alpha \epsilon =\sum _{\ell =1}^9(\alpha {\epsilon }_{1,\ell }+\alpha {\epsilon }_{2,\ell })/9,$$

and an effective cloud top pressure for the entire FOR is computed as the weighted average of all nine values of $p_{c1}$ and $p_{c2}$ in the FOR

Eq. (5)

$$p_c=\sum _{\ell =1}^9(\alpha {\epsilon }_{1,\ell }{p}_{c1,\ell }+\alpha {\epsilon }_{2,\ell }{p}_{c2,\ell })/\sum _{\ell =1}^9(\alpha {\epsilon }_{1,\ell }+\alpha {\epsilon }_{2,\ell })$$

as was also done in version-5. The version-6 level-2 product contains individual values of $\alpha {\epsilon }_{1,\ell }$, $\alpha {\epsilon }_{2,\ell }$, $p_{c1,\ell }$, $p_{c2,\ell }$ for each AIRS FOV, as well as the single FOR heritage values $\alpha \epsilon$ and $p_c$ defined according to Eqs. (4) and (5).

Cloud parameters in an AIRS FOV are derived such that channel radiances computed using these cloud parameters $R_i(\alpha {\epsilon }_1,\alpha {\epsilon }_2,p_{c1},p_{c2},X)$, where $X$ is a state vector for the FOV, best match the observed radiances $R_i$ in that FOV for the ensemble of cloud retrieval channels $i$. The $i$ channels used to determine cloud fraction and cloud top pressure are the same as those used in the cloud clearing step and are shown by yellow stars in Fig. 1. The state vector $X$ used to derive cloud parameters in an AIRS FOV is the geophysical state retrieved for the entire AIRS FOR containing the nine FOVs.

In version-5, the state vector $X$ used to derive values of $\alpha \epsilon$ and $p_c$ in an FOR was the retrieved state used in the final cloud clearing step for those cases in which a successful combined AIRS/AMSU retrieval was performed. In the roughly 27% of the cases in which the AIRS/AMSU retrieval was rejected (see Fig. 6), the state $X$ used to derive cloud parameters was the so-called fallback state that was obtained from a previously performed AMSU only retrieval step.³ Cloud parameters retrieved using the fallback state vector $X$ were flagged as $\mathrm{QC}=1$, and those retrieved using the final retrieval state vector $X$ were flagged as $\mathrm{QC}=0$. Under some conditions, the cloud parameter retrieval step was not able to complete successfully, and clouds retrieved for those cases were flagged as $\mathrm{QC}=2$ in version-5.

In version-6 and version 6 AO, successful retrievals are performed under essentially all conditions, and there is no need to use $X$ derived from a microwave fallback state. Nevertheless, version-6 does resort to the use of $X$ derived from a partial fallback state under some circumstances in which part of the retrieved state $X$ is known to be of poor quality and a better alternative is available. In particular, values of $T_s$ retrieved under either near overcast or overcast conditions over ocean can be spuriously very low. These values of $T_s$ will, in general, be flagged as bad, with $\mathrm{QC}=2$, meaning they are not used in the generation of the level-3 $T_s$ product. Associated values of ${\epsilon }_i$ retrieved under these conditions will also be poor and are also flagged to be of poor quality. Nevertheless, some value for $T_s$ and ${\epsilon }_i$ must be included in the state vector $X$ used to derive the cloud product. We have found that the initial guess $T_s^0$ coming from the Neural-Net start-up procedure gives reasonable values over nonfrozen ocean even for very cloudy cases. Therefore, over ocean, if $|T_s-T_s^0|>5\mathrm{K}$, we assume the retrieved values ${\epsilon }_i$ and $T_s$ are in error and replace $T_s$ and ${\epsilon }_i$ in the retrieved state vector $X$ by $T_s^0$ and ${\epsilon }_i^o$ while retaining the remainder of the retrieved state vector $X$ when computing cloud parameters. We have found that Neural-Net values of $T_s^0$ over land or ice are not of sufficiently good quality for use in the generation of cloud parameters, so this test and replacement procedure is done only over open ocean. As in version-5, cloud parameters retrieved in such fallback cases are flagged as $\mathrm{QC}=1$. Cloud parameters retrieved under almost all other cases, which represent the vast majority of the cases, are flagged as $\mathrm{QC}=0$. Under the extremely rare conditions in which the final cloud parameter retrieval step does not complete successfully, cloud parameters are flagged as $\mathrm{QC}=2$ as was done in version-5 and are not used in the generation of the level-3 cloud products.

A complication in the cloud parameter retrieval methodology is that the best least squares fit may result from a cloud parameter solution that lies in a region that is unphysical. To avoid an unphysical result, we do not allow retrieved cloud fractions to be $<0$ or $>100\%$; nor do we allow cloud top pressures to be very close to the surface or above the tropopause. Because of the way these constraints were handled in version-5, many cloud retrievals in version-5 failed to converge properly. We made numerous enhancements in version-6, which stabilized the cloud parameter retrieval step and also allowed for cloud top pressures to lie closer to the surface than was allowed in version-5.

Figure 13(a) shows the number of cases in which a nonzero cloud fraction $\alpha \epsilon$ was retrieved as a function of cloud top pressure $p_c$ for version-5, version-6, and also for version-6 AO. Two features are readily apparent from Fig. 13(a): the distributions of the number of cases obtained as a function of retrieved cloud top pressure are essentially identical in version-6 and version-6 AO, and both are substantially different from that of version-5. Version-5 has spikes in the number of cases retrieved at select pressures, such as 200, 300, 350, 750, 850, and 950 mb, which resulted from the cloud retrieval algorithm’s inability to converge properly in those cases. Such features are not observed in either version-6 or version-6 AO. Even more significant is the shift to higher pressures in the peak of the occurrence of low clouds in version-6 as compared to version-5. This difference near 1000 mb is, in part, due to the constraint used in version-5 that $p_c$ must be at least 50 mb above the surface, while in version-6, $p_c$ was allowed to go down to 10 mb above the surface. The large shift in the peak in the number of clouds retrieved in version-5 as a function of cloud top pressure, from $\sim 650$ to $\sim 750\mathrm{mb}$ in version-6, is a combined result of changes not only in the cloud parameter retrieval step, but also in the state vector $X$ used in version-5 compared to that used in version-6, which does not use a microwave only fallback retrieval state.

Fig. 13

Global statistics of version-5, version-6, and version-6 AO cloud parameter retrievals as a function of retrieved cloud top pressure: (a) Number of retrieved cases for a given cloud top pressure. (b) Average retrieved cloud fraction of a function of cloud top pressure.

Figure 13(b) shows plots analogous to those shown in Fig. 13(a), but shows the average cloud fraction $\alpha \epsilon$ found for each cloud top pressure $p_c$. Cloud fractions in version-6 and version-6 AO are again very close to each other and differ significantly at some cloud top pressures from those of version-5. Version-6 has more clouds than version-5, between 130 and 400 mb. On the other hand, version-6 has fewer clouds than version-5, between $\sim 600$ and 750 mb, which corresponds to the pressure interval in which the maximum numbers of cloud parameter retrievals occurred in version-5. Figure 13(b) shows spikes in the retrieved cloud fraction in version-5 at the same pressures in which they occurred in Fig. 13(a). These version-5 spikes in Fig. 13(b) are negative at pressures $<500\mathrm{mb}$, indicative of the fact that the spurious cloud retrievals occurring at these discrete pressures had low, probably $\sim 0$, cloud fractions. On the other hand, these spikes in Fig. 13(b) for version-5 were positive at pressures 700 mb and greater, indicative that these spurious cases had large cloud fractions, most likely close to 100%. Version-5 also had a somewhat disconcerting peak near 90 mb in Fig. 13(b), but Fig. 13(a) shows that there were very few such cases.

Figure 14 shows the spatial distributions of values of cloud fraction $\alpha \epsilon$ and cloud top pressure $p_c$ for the daytime and nighttime orbits on August 10, 2007 as retrieved using version-5 and version-6. These plots depict both $\alpha \epsilon$ and $p_c$ at the same time. There are seven different color scales used for different intervals of $p_c$, as indicated on the figures. Reds, violets, and purples indicate high (low pressure) clouds, blues and greens indicate mid-level clouds, and oranges and yellows indicate low clouds. Within each color scale, darker colors indicate greater fractional cloud cover, and paler colors indicate lower fractional cloud cover. While the basic cloud patterns are the same in version-6 and version-5, the cloud features are much more coherent, and the colors are darker, in version-6. Of particular significance are the coherent areas of dark orange, depicting extensive cloud cover with cloud top pressures between 680 and 800 mb, found in version-6 that are at best muted in version-5. This finding is indicative of the better ability to derive the existence of stratus clouds over ocean in version-6 as compared to version-5. Particularly noteworthy is the region in the vicinity of 30°N, 120°W, off the West Coast of North America, in which version-6 depicts extensive stratus cloud cover at both 1:30 a.m. and 1:30 p.m., while version-5 shows very little cloud cover at all. The results shown in Fig. 13(a) are suggestive of this result because many more cases with $p_c>700\mathrm{mb}$ exist in version-6 as compared to version-5. Another noteworthy improvement in version-6 clouds compared to version-5 is that the spatial distribution of clouds in version-5 has many missing grid points in which no successful cloud retrieval was performed.

Fig. 14

Version-6 and version-5 retrieved cloud fractions and cloud top pressures for 1:30 a.m. and 1:30 p.m. orbits on August 10, 2007.

There are very few missing grid points (other than orbit gaps) found in version-6. The percent of grid boxes in which data exist, indicated beneath each figure, shows that version-6 has retrieved cloud parameter values in $\sim 6\%$ more of the grid boxes than does version-5, both at 1:30 a.m. and 1:30 p.m.

Kahn et al.¹³ give more details about the updates to the cloud parameter retrieval algorithm in version-6 as compared to version-5 and show that the higher spatial resolution cloud top pressures, and corresponding cloud top temperatures found in version-6, have coherent spatial structure and contain a larger range of values than is found in version-5. Kahn et al.¹³ also show a much better agreement of retrieved version-6 cloud parameters, as compared to version-5, with those found in CloudSat and CALIOP data. Kahn et al.¹³ also introduce the new version-6 products of cloud thermodynamic phase, ice cloud effective diameter, and ice cloud optical thickness, which are derived on an AIRS FOV basis using other AIRS retrieved products as a starting point for their radiative transfer calculations.

4.6.

Outgoing Longwave Radiation

AIRS OLR is computed for each AIRS FOV in which a successful cloud parameter retrieval is performed. QC flags used for OLR are identical to those used for cloud parameters. OLR is computed via a radiative transfer calculation, which generates the total longwave flux to space expected for the final retrieved state vector $X$, including the retrieved cloud parameters. More details of the methodology used to calculate AIRS version-5 and version-6 OLR are given in Susskind et al.¹⁴ Susskind et al.¹⁴ show that version-5 OLR gives good agreement with CERES OLR, in terms of both absolute values and anomaly time series on a $1°\times 1°$ latitude-longitude spatial scale. Susskind et al.¹⁴ also show that version-6 OLR gives even better agreement with CERES than does version-5 OLR. Part of this improvement of OLR in version-6 is a result of the improved accuracy of version-6 retrieved products as compared to version-5. In addition, version-6 uses an improved OLR radiative transfer parameterization¹⁵ compared to what was used in version-5.¹⁶