Analysis of the NASA-POWER system for estimating reference evapotranspiration in the Comarca Lagunera, Mexico

Introduction

Actual crop evapotranspiration is commonly estimated by considering reference evapotranspiration (ET_o) and crop coefficient. The former can be evaluated using the Food and Agriculture Organization of the United Nations (FAO) manual 56, where the FAO-56 Penman Monteith (FAO-56 PM) method is the most used for accuracy under different climatic conditions and regions (Allen et al., 1998).

The FAO-56 PM method requires several meteorological input variables, which are usually not available at the required frequency and quality. Therefore, variants of that methodology have been proposed to estimate ET_o, such as the use of alternative equations to estimate it with limited meteorological data (Jabloun & Sahli, 2008), the use of empirical models based on machine learning algorithms (Huang et al., 2019; Mattar, 2018), among others. However, an emerging alternative that eliminates the need for a robust network of weather stations is the use of reanalysis data or gridded weather data. These data are commonly available for free on web platforms or published in regular gridded formats (with a time-lag of days from the present) and can be used to obtain continuous data from a site or to fill geospatial gaps in weather data (Bai et al., 2010).

Reanalysis data are produced from the combination of measurement and simulation data using data assimilation techniques to obtain the most realistic description of climate occurrences. Reanalysis data have been used in applications where their performance, advantages and disadvantages are analyzed, such as in crop simulation models to estimate yields (Monteiro et al., 2018; Camargo-Rodriguez & Ober, 2019; Starr et al., 2020; Tyagi et al., 2019), ET_o estimation (Martins et al., 2017; Paredes et al., 2018; Pelosi et al., 2020), irrigation scheduling, global warming assessment (Marzouk, 2021), among others.

Reanalysis data used to estimate ET_o are mainly from NASA-POWER (NP), Global Land Data Assimilation System (GLDAS), Climate Forecast System ver. 2 (CFSv2), North American Land Data Assimilation System (NLDAS), Real-Time Mesoscale Analysis (RTMA), National Digital Forecast Database (NDFD) and gridMET. Of these, some have global coverage and others regional; the latter provide data with finer spatial and temporal resolution, although it should be noted that high spatial resolution does not necessarily indicate high accuracy (Blankenau et al., 2020).

The NP system is one of the most used systems for estimating ET_o (Ndiaye et al., 2020; Negm et al., 2017; Srivastava et al., 2020) due to its frequent updating and near real-time data availability. Moreover, the data can be accessed automatically through the mobile or web applications (Maldonado et al., 2019).

In several studies, NP system data are used without prior local assessment (e.g. Ndiaye et al., 2020), because insufficient information is available. However, these data contain uncertainty; therefore, they must be evaluated and validated locally with available in situ measurements. Regarding the above, the objective of this study was to analyze, by means of four error parameters, the errors that can be generated by meteorological data and variables of the NP system to estimate ET_o, using as reference data measured at seven meteorological stations.

Materials and methods

Study area

The study area was the area comprised by irrigation district 017 (DR 017) Comarca Lagunera and neighboring irrigation units (Figure 1). The predominant climate is classified as BWhw, corresponding to very arid, semiarid (Garcia, 2004). The Comarca Lagunera is one of the most important agricultural areas in Mexico, but with great water pressure.

Meteorological stations

To evaluate ET_o estimation from climate data obtained from the NP system, daily data recorded at seven agrometeorological stations (Table 1) of the National Network of Automated Agrometeorological Stations of Instituto Nacional de Investigaciones Forestales, Agrícolas y Pecuarias (the INIFAP, 2020 ) standardized to 2 m were used. This network is operated by the National Laboratory of Modeling and Remote Sensing. Selected meteorological stations have daily records from 2005 to 2020 of the variables mean (T_mean), minimum (T_min) and maximum (T_max) temperature, mean wind speed (u ₂ ), mean relative humidity (RH) and cumulative solar radiation (Rs).

Table 1. List of weather stations used in the study.

The accuracy of ET_o estimates depends on the quality of the weather data used; therefore, a data quality control procedure is required. Allen (1996) describes different procedures to ensure the quality of weather input variables of ET_o. This study analyzed the data used according to this author. Table 2 shows a summary of the quality control, and sensors measuring Rs and u ₂ require continuous maintenance.

Table 2. Data quality control.

NASA-POWER (NP) system

The climate geoportal of the NP system (https://power.larc.nasa.gov) allows the extraction of values (from 1981 to date) of agroclimatological variables with global coverage. Table 3 shows the main features of the NP system database.

Table 3. Characteristics of the NASA-POWER (NP) system.

The NP system collects information from several sources: directly measured data, satellite data, wind soundings and data derived from assimilated data systems. Daily temperature and RH data are derived from the Goddard Earth Observing System (GEOS) assimilation model version 4, 5.01 and 5.1, wind speed from the Modern Era Retrospective-Analysis for Research and Applications (MERRA-2) model, and solar radiation from satellite observations (White et al., 2008).

To conduct the study, data were downloaded from the NP system (Table 4) for the same period from the weather stations listed in Table 1. Cells with at least one agrometeorological station (Figure 1) were selected, i.e., data from 5 cells of 0.5 x 0.5° were downloaded.

Table 4. Agrometeorological variables from the NASA-POWER system used to estimate reference evapotranspiration.

Reference evapotranspiration (ETo)

FAO 56 Penman-Monteith (PM) and Hargreaves and Samani (HS)

The estimate of ET_o (mm∙d^-1) with the FAO-56 PM method (Allen et al., 1998) is based on Equation 1. This method is recommended by the FAO for calculating ET_o and is used to evaluate the accuracy of other methods.

where Rn is the net radiation at the reference crop surface (MJ∙m^-2∙d^-1), G is the soil heat flux (MJ∙m^-2∙d^-1), e _s is the saturated vapor pressure (kPa), e _a is the actual vapor pressure (kPa), e _s - e _a is the vapor pressure deficit (kPa), ∆ is the slope of the saturated vapor pressure curve (kPa∙°C^-1) and γ is the psychrometric constant (kPa∙°C^-1). For daily time intervals, the values of G are relatively small and, therefore, the term is neglected (Allen et al., 1998).

The HS equation (Hargreaves & Samani, 1985) is used to estimate ET_o when only temperature data are available. This equation was developed for semi-arid areas and has good accuracy for the study area (e.g. Chávez-Ramírez et al., 2013).

where ET _o-HS is the ET_o estimated by the HS equation (mm·day^-1), Ra is the cumulative extraterrestrial radiation (MJ∙m^-2∙d^-1), K _H and K _T are the empirical calibration parameters and A _H is an empirical Hargreaves exponent. This study used the original values proposed by Hargreaves and Samani (1985): K _H = 0.0023, K _T = 17.8 and A _H = 0.5.

Estimate of reference evapotranspiration (ETo)

ET_o (daily and decadal average) was estimated in five different ways; each case used a method for estimating ET_o (FAO-56 PM or HS) and a different meteorological data source (measured at agrometeorological stations, NP data or combination of both) (Table 5). The FAO-56 PM method with data measured from agrometeorological stations (MEDEST) was used to calculate the observed ET_o. This ET_o was used to evaluate the performance of the five proposed cases. Evaluation is based on the FAO-56 PM method being the most reliable for different regions (Allen et al., 2000).

Table 5. Cases analyzed to estimate reference evapotranspiration (ET_o).

Cases C1 and C3 were proposed to know the accuracy of ET_o with NP reanalysis data. C2 arises to compare the yield of the HS equation in the area using MEDEST data. In C4 and C5, different data sources (MEDEST and NP) are mixed with the FAO-56 PM method. C4 was proposed because u ₂ of NP is the variable with the lowest R² in validation performed by developers (Stackhouse et al., 2019), while C5 was proposed because it is common to have poor quality and quantity of Rs data in meteorological stations (Sayago et al., 2020).

Accuracy evaluation

First, the precision of the variables (T_max, T_min, T_mean, u ₂ , RH and Rs) was evaluated by comparing the estimated data from the NP system to the measured data. The accuracy assessment was performed with four statistical parameters: root mean square error (RMSE, Equation 3), mean error (ME, Equation 4), standard deviation of errors (SDE, Equation 5) and coefficient of determination (R², Equation 6) (Cobaner et al., 2017; Jabloun & Sahli, 2008).

where V _o is the measured or observed value, V _e is the estimated value (in this case those extracted from the NP system), i is the day analyzed and n is the number of values or days analyzed.

Subsequently, the precision of the ET_o of cases C1-C5 was evaluated (Table 4) using the same four statistical parameters, where V _o is the observed ET_o and V _e is the value of the ET_o estimated with cases C1 to C5.

Sensitivity analysis of variables

A sensitivity analysis is essential to evaluate the impact of input variables on ET_o estimate with the FAO-56 PM equation. This analysis was performed according to the following approach: a systematic error was introduced to the value of a given climate parameter (temperature, RH, u ₂ or Rs) and the other values were kept constant; then, ET_o was estimated with the FAO-56 PM method and ME and RMSE values of change in ET_o were calculated. Induced errors were within a range (Table 6) and were defined considering the maximum value of RMSE found in the evaluation of the precision of each variable, plus an increment of 5 %; that is, if a variable had a maximum RMSE of 5, then the range established would be ±5 with an increment of 0.5 in that range.

Table 6. Ranges of increase in variables to estimate their sensitivity

Results and discussion

Assessment of variables

Temperature

The temperature data (T_min, T_mean and T_max) of the NP system, compared to those measured, had an R² of 0.84 to 0.95 and RMSE values lower than 4 °C. The lowest R² (0.84 to 0.90) and highest RMSE values (1.95 to 3.67 °C) were found for T_min, while the highest R² (0.92 to 0.95) and lowest RMSE values (1.29 to 2.16 °C) were for T_mean (Figure 2). These results agree with those found by other authors, where higher R² and lower RMSE are reported with T_mean. Negm et al. (2017) compared data from the NP system against 42 agrometeorological stations in Sicily, Italy, located from 10 to 1 470 m a. s. l., and found lower average RMSE values with T_mean (RMSE = 3.2 °C) and higher with T_min (RMSE = 5 °C). In the United States, White et al. (2008) had higher correlations with T_mean (R² = 0.91) and reported an average R² of 0.88 for T_min and T_max; thereby concluding that if mountainous and coastal regions are excluded, the NP system is a reliable data source for daily temperature. In Egypt, Aboelkhair et al. (2019) found higher R² for T_mean.

Regarding SDE, average values of 2.32, 1.46 and 1.76 °C were found for T_min, T_mean and T_max, respectively (Figure 2a). This indicates greater variation between T_min from NP and measured data.

T_mean and T_max from NP data tend to underestimate the measured data, because ME values < 0 °C were found (Figure 2a). This agrees with that reported by Stackhouse et al. (2019), who validated measured data in several countries and found ME values < 0 for T_mean and T_max, and positive values for T_min. For T_mean, higher underestimates were found in the winter-spring months, while for T_max lower underestimates were observed in the spring months (Figure 2b).

Regarding T_min, the ME found are in the range of -1.1 to 2.34 °C. In general, at low values (T_min < 10 °C), the NP system data overestimated measurements, and at higher values it was underestimated (Figure 3). This indicates that the maximum overestimation occurs mainly in the autumn-winter months (Figure 2b), where temperatures drop in the region. Bai et al. (2010) reported an underestimation of NP data for air temperature (T_max and T_min) in China, where the underestimation was higher for T_max.

According to the results, there are differences between the measured data and those estimated by NP, which are not constant throughout the year. Among the factors that may account for variability of the data are topography, changes in land use or localized effects that may be sources of errors in assimilation models (White et al., 2008). In this sense, Negm et al. (2018) describe a methodology to improve accuracy of NP data considering the Julian day and site topography.

Relative humidity

R² values from 0.60 to 0.76, ME from -0.32 to -7.52%, RMSE from 8.83 to 12.16 % and SDE from 8.18 to 10.38 were found (Figure 4 a). Negm et al. (2017) had RMSE from 8 to 18 and indicate that coastal areas have higher RMSE values than inland areas. RMSE values found in this study are < 13 % because stations are in inland areas. In Egypt, Aboelkhair et al. (2019) had RMSE values from 4.2 to 31.8 %; moreover, they found that NP data underestimated the measured RH (ME < 0) at all stations, as found in this study and in that conducted by Rodrigues and Braga (2021). This underestimation is greater in the winter-spring months (Figure 4b), and according to Figure 5, the higher the RH, the greater the underestimation.

Wind speed (u 2 )

The u ₂ of the NP system was the variable with the lowest R² values (0.19 to 0.52). This behavior is also reported by Duarte and Sentelhas (2020), Najmaddin et al. (2017), and Rodrigues and Braga (2021). De Pondeca et al. (2011) indicate there are many challenges in u ₂ determination, such as quality control of the measured data, so improving the quality could provide more accurate estimates.

NP system data tend to overestimate the measured u ₂ values at all stations, as positive ME values of up to 1.43 m∙s^-1 were found (Figure 6a). These overestimates are higher in the months of June to September (Figure 6b). RMSE values of 0.92 to 1.63 m∙s^-1 and SDE of 0.67 to 1.01 m∙s^-1 were found (Figure 6a). Stackhouse et al. (2019), in a global-level evaluation of the NP system, report average RMSE values of 1.65 m∙s^-1 and positive ME values. Higher RMSE values (1.3 to 3.5 m∙s^-1) were reported by Negm et al. (2017). RMSE values found in the present study are within the range reported by these authors.

A relationship was found between error parameters and latitude of stations. The highest values of ME (>1.19) and RMSE (>1.40) were found at the southernmost stations (#1, #4 and #7), and the lowest values (ME = 0.53 and RMSE = 0.92) were found at the station located at the highest latitude (#2) (Figure 1). Stations #1, #4 and #7 had more noticeable overestimation of u ₂ (Figure 7), while in the rest of the stations, at high u ₂ (> 3 m∙s^-1) a tendency to underestimation is observed.

Solar radiation

R² values from 0.72 to 0.88 were found, where five stations were in the range of 0.81 to 0.84. Station #3 had an R² lower than 0.80, in addition to having the highest RMSE value (Figure 8a). R² values found coincide with those reported by other authors. In the United States, R² values from 0.76 to 0.97 have been reported (White et al., 2011), and in Spain, R² values from 0.85 to 0.96 (Sayago et al., 2020).

Regarding RMSE, values from 2.73 to 3.41 MJ∙m^-2∙d^-1 were found (Figure 8). These values agree with those of Negm et al. (2017), who had values from 2.0 to 4.8 MJ∙m^-2∙d^-1, and those of Bai et al. (2010), with RMSE values of 3.1 MJ∙m^-2∙d^-1. NP Rs data tend to underestimate the measured values. Only station #2 had a ME > 0 MJ∙m^-2∙d^-1. According to Figure 8b, mean differences are higher in the months from May to October, which corresponds with the rainy season (Comisión Nacional del Agua [CONAGUA], 2020). According to Sayago et al. (2020), the accuracy of NP Rs data depends on atmospheric conditions, because they found on clear days an R² of 0.96 and an RMSE of 1.78 MJ∙m^-2∙d^-1, concluding that these estimators are deficient on cloudy days.

Figure 9 shows that the larger the Rs value, the NP data tend to underestimate, and the smaller the Rs (< 20 MJ∙m^-2∙d^-1), NP data tend to overestimate.

Sensitivity analysis of variables

Figure 10 shows the values of ME and RMSE of ET_o when errors were introduced in the values of climatic variables; each variable shows a different impact on ET_o accuracy. Negative errors induced in T_mean and RH show an overestimation of observed ET_o (ME > 0 mm∙d^-1), and positive errors in T_max, T_min, u ₂ and Rs resulted in an overestimation of ET_o (Figure 10a). Similar behaviors to those shown in Figure 10 have been found in studies conducted in different regions when sensitivity analysis is performed (Biazar et al., 2019; Debnath et al., 2015; Ndiaye et al., 2017).

Induced errors to u ₂ indicate that this variable produces the largest values of ME and RMSE associated with ET_o (Figure 10). A difference, compared to the measured value, of +0.5 m∙s^-1 can generate an overestimation in ET_o of up to 0.4 mm∙d^-1, while a difference of +2 m∙s^-1 can generate an overestimation of 1.5 mm∙d^-1. These results show high sensitivity of ET_o to u ₂ , which has been reported by Debnath et al. (2015), and Tabari and Hosseinzadeh (2014) in arid and semiarid regions. Debnath et al. (2015) indicate that in such regions wind flow probably replaces moist air with dry air very rapidly, which causes an increase in ET_o compared to other regions.

Considering the RMSE values (Figure 10b), T_max has, in general, the second largest influence on ET_o estimate, followed by Rs, T_mean, T_min and RH. Table 7 shows RMSE values derived from ET_o estimate when values with some errors are introduced. These errors are considering the maximum RMSE values found in the evaluation of precision of NP variables.

Table 7. Root mean square error (RMSE) when estimating reference evapotranspiration (ET_o) with modified climate values from the NASA-POWER (NP) system.

According to Table 7, even though the maximum RMSE of RH of the NP system is 12.16 %, this variable can generate a difference in ET_o of 0.22 mm∙d^-1, which is less than that produced by the maximum RMSE of T_max, Rs and u ₂ . Of these variables, the only one that can generate a difference greater than 0.5 mm∙d^-1 compared to the ET_o observed is u ₂ ; therefore, it is suggested that this be measured directly in situ.

Performance for reference evapotranspiration (ETo) estimation

Figure 11 shows mean difference (mm∙d^-1) between estimated ET_o and observed ET_o, and C1 overestimates the observed ET_o over the entire year. The magnitude of overestimation is related to the precision of each variable and has been reported when only NP data and the FAO-56 PM method are employed (Maldonado et al., 2019; Negm et al., 2017; Srivastava et al., 2020). The largest mean C1 differences occur in the spring months because this is when NP data underestimate T_mean and RH by the greatest magnitude, coupled with the overestimation of u ₂ (Figure 11). Summer months show the smallest differences, and coincide with the highest u ₂ , overestimates, so we would expect to find the highest ET_o overestimates; however, they also coincide with T_max underestimates and the highest Rs underestimates.

C2 shows underestimates in summer, while overestimates occur in spring and autumn. C3 is influenced by the behavior of C2 and accuracy of NP in temperature variables. C3 tends to underestimate ET_o from winter to summer and overestimates in autumn. C4 shows overestimates in the first four months of the year, which may be due to the underestimation of T_mean and RH data; however, the rest of the year shows a tendency to overestimation, which is where the impact of Rs and T _max data of the NP system is manifested. As expected, C5 shows the same behavior as Figure 8b, because only the NP Rs value was used.

Concerning decadal average ET_o (Figure 11b), the same behaviors of Figure 11a are observed; however, mean differences decrease when ET_o is analyzed on a decadal basis. In other studies, it has been observed that when using reanalysis data, accuracy improves when using 10-day average values compared to daily data (Monteiro et al., 2018).

When decadal mean ET_o was analyzed (Figure 12), higher R² values were found (greater than 0.84) than with daily data; SDE and RMSE values decreased. This indicates a better accuracy of ET_o when using decadal mean values of the NP system.

For daily ET_o with C1, RMSE values from 0.99 to 1.36 were found, and for decadal mean ET_o, from 0.63 to 1.02 mm∙d^-1. Errors of daily ET_o of 0.68 to 1.27 mm∙d^-1 have been reported by Negm et al. (2017), who indicate that for most stations the observed ETo is overestimated. Maldonado et al. (2019), with measured data from agrometeorological stations located in different countries at different altitudes, reported RMSE values from 0.31 to 1.45 mm∙d^-1. Although NP data overestimate ET_o with values close to 1 mm∙d^-1, they had R² values higher than 0.73, indicating that bias correction strategies can be employed as proposed by Rodrigues and Braga (2021).

C2 tends to underestimate ET_o for values > 5 mm∙d^-1, while, for lower values, it overestimates it. With daily ET_o values, an RMSE of 0.75 to 1.06 mm∙d^-1 was obtained, while with decadal averages, values of 0.46 to 0.69 mm∙d^-1 were found (Table 8). These values are lower than those found in C1; however, with daily values C1 shows higher R².

Table 8. Values of estimators associated with reference evapotranspiration (ET_o) in five case studies.

C3 shows similar values of error parameters as C2 due to high precision of NP temperature data. RMSE and SDE values in C3 are lower than in C1 for daily and decadal ET_o. Najmaddin et al. (2017) note that when using reanalysis data in an arid zone, lower RMSE values are found with the HS method than with the FAO-56 PM method; this may be due to the precision of NP u ₂ and the high sensitivity of ET_o to this variable. Therefore, for the study area, when there are no measured data, it is recommended to use the NP data with the HS method (C3), and not with the PM method (C1), and preferably to estimate values of decadal mean ET_o.

The highest R² values and lowest RMSEs were found in C4 and C5 for both daily and decadal ET_o. This indicates that NP data, except for u ₂ , can be used as an option to fill geospatial gaps in meteorological data to estimate ET_o. Comparing both cases, in the case of C5, we found R² > 0.95, and RMSE values < 0.43 mm∙d^-1 for daily ET_o and < 0.38 mm∙d^-1 for decadal mean ET_o. On the other hand, in the case of C4, we observed R² > 0.91 and RMSE < 0.53 mm∙d^-1 for daily ET_o, and R² > 0.93 and RMSE < 0.40 mm∙d^-1 for decadal mean ET_o (Table 8). The above indicates greater accuracy is obtained with C5, where only NP Rs data were used. This makes sense, since more NP variables are used in C4, including Rs, which show some error compared to measured data. The difference in RMSE of C4, compared to C1, shows the effect that u ₂ has on that of ET_o.

Conclusions

Reanalysis data can be an important tool for areas with no information to estimate ET_o or when this information is incomplete because it facilitates access to values of various meteorological variables of interest, such as temperature, relative humidity, wind speed and solar radiation. However, before using the data, it should be validated using values measured in the area, so that accuracy provided by data set can be quantified and alternatives for bias correction can be sought. Often, measured data are not available to make an evaluation; therefore, an alternative is to compare several reanalysis data sets and select those with the same behaviors, although there are not many well-documented studies available in the literature on this subject.

Of the five cases analyzed, those that combine measured data and NP system data with the FAO-56 PM method (C4 and C5) had the lowest RMSE and highest R² values. This indicates that both cases can be used for the study area and estimate ET_o with high accuracy. These cases are valid when measured variables are available in the area of interest. When measured data are not available, it is convenient to estimate ET_o with HS equation and temperature data from the NP system, as discussed in C3.

Results indicate that the NP system data (except for wind speed) can be used as an option to complete missing data for ET_o estimation, or in other applications, such as estimating growing degree days, due to high R² values obtained in the evaluation of temperature.