BGBiogeosciencesBGBiogeosciences1726-4189Copernicus PublicationsGöttingen, Germany10.5194/bg-14-1457-2017Technical note: Dynamic INtegrated Gap-filling and partitioning for OzFlux
(DINGO)BeringerJasonjason.beringer@uwa.edu.auhttps://orcid.org/0000-0002-4619-8361McHughIanHutleyLindsay B.https://orcid.org/0000-0001-5533-9886IsaacPeterKljunNataschahttps://orcid.org/0000-0001-9650-2184School of Earth and Environment (SEE), The University of Western
Australia, Crawley WA, 6009, AustraliaSchool of Earth, Atmosphere and Environment, Monash University,
Clayton, 3800, AustraliaSchool of Environment, Research Institute for the Environment and
Livelihoods, Charles Darwin University, NT 0909, AustraliaSchool of Earth, Atmosphere and Environment, Monash University,
Clayton, 3800, AustraliaDepartment of Geography, Swansea University, Singleton Park, Swansea,
Wales SA2 8PP, UKJason Beringer (jason.beringer@uwa.edu.au)23March2017146145714605May201610May20167February20177February2017This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/This article is available from https://bg.copernicus.org/articles/14/1457/2017/bg-14-1457-2017.htmlThe full text article is available as a PDF file from https://bg.copernicus.org/articles/14/1457/2017/bg-14-1457-2017.pdf
Standardised, quality-controlled and robust data from flux networks underpin
the understanding of ecosystem processes and tools necessary to support the
management of natural resources, including water, carbon and nutrients for
environmental and production benefits. The Australian regional flux network
(OzFlux) currently has 23 active sites and aims to provide a
continental-scale national research facility to monitor and assess
Australia's terrestrial biosphere and climate for improved predictions. Given
the need for standardised and effective data processing of flux data, we have
developed a software suite, called the Dynamic INtegrated Gap-filling and
partitioning for OzFlux (DINGO), that enables gap-filling and partitioning of
the primary fluxes into ecosystem respiration (Fre) and gross primary
productivity (GPP) and subsequently provides diagnostics and results. We
outline the processing pathways and methodologies that are applied in DINGO
(v13) to OzFlux data, including (1) gap-filling of meteorological and other
drivers; (2) gap-filling of fluxes using artificial neural networks; (3) the
u* threshold determination; (4) partitioning into ecosystem respiration
and gross primary productivity; (5) random, model and u* uncertainties;
and (6) diagnostic, footprint calculation, summary and results outputs. DINGO
was developed for Australian data, but the framework is applicable to any
flux data or regional network. Quality data from robust systems like DINGO
ensure the utility and uptake of the flux data and facilitates synergies
between flux, remote sensing and modelling.
Introduction
OzFlux is the regional Australian and New Zealand flux tower network that
aims to provide a continental-scale national research facility to monitor
and assess Australia's terrestrial biosphere and climate for improved
predictions (Beringer et al., 2016). High-quality and
reliable data are a crucial foundation in achieving the objectives of the
OzFlux network (Beringer et al., 2016) and underpin the
process understanding needed to (1) support sound management of natural
resources including water, carbon and nutrient resources for environmental
and production benefits; (2) monitor, assess, predict and respond to climate
change and variability; (3) improve weather and environmental information and
prediction; (4) support disaster management and early warning systems needed
to meet Australia's priorities in national security; and (5) ensure that
Earth system models used to underpin Australia's policies and commitments to
international treaties adequately represent Australian terrestrial ecosystem
processes (Beringer et al., 2016).
Beringer et al. (2016) provide an overview of the evolution, design and
current status of OzFlux as well as a brief summary of the instrumentation
and data collection that forms the backbone of the network. A detailed
description of the quality control and post-processing of the eddy covariance
data using OzFluxQC and the data pathway to curation is given by Isaac et
al. (2016). In summary, from Beringer et al. (2016), most sites have data
loggers that provide the average (usually over 30 min) covariances that are
processed through 6 levels using the OzFluxQC standard software processing
scripts. Levels 1, 2 and 3 are as follows: L1 is the raw data as received
from the flux tower, L2 it the quality-controlled data, and L3 is the
post-processed, corrected, but not gap-filled data. Quality control measures
by OzFluxQC are applied at L2 and comprise checks for plausible value ranges,
spike detection and removal, manual exclusion of date and time ranges and
diagnostic checks for all quantities used in the flux correction
calculations. The quality checks make use of the diagnostic information from
the sonic anemometer and the infrared gas analyser. For sites calculating
fluxes from the averaged covariances, post-processing includes 2-D coordinate
rotation, low- and high-pass frequency correction, conversion of virtual heat
flux to sensible heat flux and application of the Webb–Pearman–Leuning (WPL) correction
to the latent heat and CO2 fluxes (see Burba, 2013 for a general
description of the data processing pathways). Steps performed at L3 include
the correction of the ground heat flux for storage in the layer above the
heat flux plates (Mayocchi and Bristow, 1995) and correction of the CO2
flux data for storage in the canopy (where available). OzFlux data are
available at http://data.ozflux.org.au/. OzFlux sites submit their data
to FLUXNET at L3.
Given the international need by the community for standardised data
processing to enable effective comparison across biomes and to understand
inter-annual variability (Papale et al., 2006), we
have developed a software tool to address this need. In this paper, we
describe the development and testing of the Dynamic INtegrated Gap-filling
and partitioning for the Ozflux (DINGO) system that utilises the L3 data from
OzFluxQC to gap-fill and partition the fluxes in ecosystem respiration (Fre)
and gross primary productivity (GPP) and subsequently provides diagnostics
and results. This paper is not intended to be a thorough review of the data
processing, but the application of standard techniques in DINGO for the flux
community. DINGO is a research version for OzFlux data, whereas the OzFluxQC
system (which has many similar features to DINGO) is considered an
operational version. Quality data from robust systems like DINGO ensure the
utility and uptake of the flux data and facilitates synergies between flux,
remote sensing, modelling and canopy physiological studies. We conclude by
looking ahead at the future direction of the DINGO system.
Approach
The overall approach used in DINGO is to take the L3 OzFluxQC data – which
has gaps from data processing (data excluded due to values out of range,
spike detection or manual exclusion of date and time ranges) or from site
issues (instrument or power failure, herbivores, fire, eagles nests, cows,
lightning, PI on sabbatical, etc.) – and gap-fill and partition the data using
a variety of data sources (Fig. S1). DINGO is programmed in python 2.7 and
is currently at version 13 and publically available on GitHub (https://github.com/jberinge/DINGO13). It is designed to work with OzFlux
data produced in NetCDF format by the OzFluxQC (Isaac et
al., 2016) and draws on Australian automatic weather station
(AWS) data, but it could be adapted for other
data sources across other flux sites. The primary interface for the user is
through a text-based control file that has information on site
characteristics (name, latitude and longitude, the frequency of the flux
measurements, i.e. 30 or 60 min, and elevation, file paths (to the OzFluxQC
NetCDF files and other ancillary data inputs), data processing options, and
data plotting and output formats. In general, prior to the processing steps
below, any gaps in fluxes or meteorological quantities of less than 2 h
are filled by DINGO using linear interpolation. The pathway for
processing is shown in Fig. S1,
and each step is outlined in the Supplement.
Conclusions
The OzFlux network has been highly successful in generating standardised
measurements and protocols that provide robust primary data. Only via
transparent, advanced and consistent QA/QC will we ensure compatibility
within the OzFlux network (Beringer et al., 2016) and with international
databases (FLUXNET) (Papale et al., 2006), ensuring uptake by the broader
scientific community. Through robust software systems such as OzFluxQC (Isaac
et al., 2016) and DINGO, we are able to ensure timely and quality gap-filling
and partitioning of fluxes that in turn enable a significant uptake of the
eddy covariance data for application to a range of research questions as
exemplified in Beringer et al. (2016). This includes integration of the eddy
covariance and remote sensing datasets for the validation of satellite
products (e.g. GPP and ET, Kanniah et al., 2009; Restrepo-Coupe et al.,
2016) and to aid the parameterisation of models that rely on
remotely sensed data (e.g. GPP, ET, canopy conductance, and light use
efficiency (LUE), Barraza et al., 2014, 2015; Glenn et al., 2011; Goerner et
al., 2011). In addition, OzFlux data have been instrumental in constraining a
continent-wide assessment of terrestrial carbon and water cycles (Haverd et
al., 2013) and featured in the development of new models (Haverd et al.,
2007, 2009). There is utility of the data to support carbon accounting
activities (Hutley et al., 2005), as demonstrated in research focussed on the conversion of savanna to
pasture (Bristow et al., 2016). Ultimately, flux data are required to address the key ecosystem
science questions of OzFlux (Beringer et al., 2016) that are focused on the
improved understanding of the responses of carbon and water cycles of
Australian ecosystems to current climate and disturbance regimes as well as
on impacts of projected future changes to precipitation, temperature and
atmospheric CO2 concentration. Key questions include the following: (1) what are the
key drivers of ecosystem productivity (carbon sinks) and greenhouse gas
emissions; (2) how resilient is the Australian ecosystem productivity to an
increasingly variable and changing climate; and, (3) what is the current water
budget of the dominant Australian ecosystems and how will it change in the
future?
The data used to create the example figures in the manuscripts
were from two sites in the OzFlux network. Site data for Whroo can be found
in Beringer (2013) “Whroo OzFlux tower site OzFlux: Australian and
New Zealand Flux Research and Monitoring”
http://hdl.handle.net/102.100.100/14232, whilst the Calperum data can be found
in Calperum Tech (2013) “Calperum Chowilla OzFlux tower site OzFlux:
Australian and New Zealand Flux Research and Monitoring”
http://hdl.handle.net/102.100.100/14236.
The Supplement related to this article is available online at doi:10.5194/bg-14-1457-2017-supplement.
The authors declare that they have no conflict of interest.
Acknowledgements
This work utilised data collected by grants funded by the Australian
Research Council DP130101566. Beringer is funded under an ARC Future
Fellowship (FT110100602).
Edited by: D. Papale
Reviewed by: three anonymous referees
ReferencesBarraza, V., Grings, F., Ferrazzoli, P., Huete, A., Restrepo-Coupe, N.,
Beringer, J., Van Gorsel, E., and Karszenbaum, H.: Behavior of multitemporal
and multisensor passive microwave indices in Southern Hemisphere ecosystems,
J. Geophys. Res.-Biogeosci., 119, 2231–2244,
10.1002/2014JG002626, 2014.Barraza, V., Restrepo-Coupe, N., Huete, A., Grings, F., and Van Gorsel, E.:
Passive microwave and optical index approaches for estimating surface
conductance and evapotranspiration in forest ecosystems, Agric. For.
Meteorol., 213, 126–137, 10.1016/j.agrformet.2015.06.020, 2015.Beringer, J.: Whroo OzFlux tower site, available at:
http://hdl.handle.net/102.100.100/14232 (last access: March 2017),
2013.Beringer, J., Hutley, L. B., McHugh, I., Arndt, S. K., Campbell, D., Cleugh,
H. A., Cleverly, J., Resco de Dios, V., Eamus, D., Evans, B., Ewenz, C.,
Grace, P., Griebel, A., Haverd, V., Hinko-Najera, N., Huete, A., Isaac, P.,
Kanniah, K., Leuning, R., Liddell, M. J., Macfarlane, C., Meyer, W., Moore,
C., Pendall, E., Phillips, A., Phillips, R. L., Prober, S. M.,
Restrepo-Coupe, N., Rutledge, S., Schroder, I., Silberstein, R., Southall,
P., Yee, M. S., Tapper, N. J., van Gorsel, E., Vote, C., Walker, J., and
Wardlaw, T.: An introduction to the Australian and New Zealand flux tower
network – OzFlux, Biogeosciences, 13, 5895–5916,
10.5194/bg-13-5895-2016, 2016.Bristow, M., Hutley, L. B., Beringer, J., Livesley, S. J., Edwards, A. C.,
and Arndt, S. K.: Quantifying the relative importance of greenhouse gas emissions
from current and future savanna land use change across northern Australia, Biogeosciences, 13, 6285–6303, 10.5194/bg-13-6285-2016, 2016.
Burba, G.: Eddy Covariance Method for Scientific, Industrial, Agricultural,
and Regulatory Applications: A Field Book on Measuring Ecosystem Gas Exchange
and Areal Emission Rates, LI-COR Biosciences, Lincoln, NE, USA, 2013.Calperum Tech: Calperum Chowilla OzFlux tower site, available at:
http://hdl.handle.net/102.100.100/14236 (last access: March 2017), 2013.Glenn, E. P., Doody, T. M., Guerschman, J. P., Huete, A. R., King, E. A.,
McVicar, T. R., Van Dijk, A. I. J. M., Van Niel, T. G., Yebra, M., and Zhang,
Y.: Actual evapotranspiration estimation by ground and remote sensing
methods: the Australian experience, Hydrol. Process., 25, 4103–4116,
10.1002/hyp.8391, 2011.Goerner, A., Reichstein, M., Tomelleri, E., Hanan, N., Rambal, S., Papale,
D., Dragoni, D., and Schmullius, C.: Remote sensing of ecosystem light use
efficiency with MODIS-based PRI, Biogeosciences, 8, 189–202,
10.5194/bg-8-189-2011, 2011.
Haverd, V., Cuntz, M., Leuning, R., and Keith, H.: Air and biomass heat
storage fluxes in a forest canopy: Calculation within a soil vegetation
atmosphere transfer model, Agric. For. Meteorol., 147, 125–139, 2007.Haverd, V., Leuning, R., Griffith, D., van Gorsel, E., and Cuntz, M.: The
Turbulent Lagrangian Time Scale in Forest Canopies Constrained by Fluxes,
Concentrations and Source Distributions, Boundary-Layer Meteorol., 130,
209–228, 10.1007/s10546-008-9344-4, 2009.Haverd, V., Raupach, M. R., Briggs, P. R., Canadell, J. G., Isaac, P.,
Pickett-Heaps, C., Roxburgh, S. H., van Gorsel, E., Viscarra Rossel, R. A.,
and Wang, Z.: Multiple observation types reduce uncertainty in Australia's
terrestrial carbon and water cycles, Biogeosciences, 10, 2011–2040,
10.5194/bg-10-2011-2013, 2013.Hutley, L. B., Leuning, R., Beringer, J., and Cleugh, H. A.: The utility of the eddy covariance
techniques as a tool in carbon accounting: tropical savanna as a case study, Aust. J. Bot., 53, 663–675, 10.1071/BT04147, 2005.
Isaac, P. R., Cleverly, J., Beringer, J., and McHugh, I.: The OzFlux network
data path: from collection to curation, to be submitted to Biogeosciences
Discuss., 2016.Kanniah, K. D., Beringer, J., Hutley, L. B., Tapper, N. J., and Zhu, X.:
Evaluation of Collections 4 and 5 of the MODIS Gross Primary Productivity
product and algorithm improvement at a tropical savanna site in northern
Australia, Remote Sens. Environ., 113, 1808–1822,
10.1016/j.rse.2009.04.013, 2009.
Mayocchi, C. L. and Bristow, K. L.: Soil surface heat flux: some general
questions and comments on measurements, Agric. For. Meteorol., 75, 43–50,
1995.Papale, D., Reichstein, M., Aubinet, M., Canfora, E., Bernhofer, C., Kutsch,
W., Longdoz, B., Rambal, S., Valentini, R., Vesala, T., and Yakir, D.:
Towards a standardized processing of Net Ecosystem Exchange measured with
eddy covariance technique: algorithms and uncertainty estimation,
Biogeosciences, 3, 571–583, 10.5194/bg-3-571-2006, 2006.Restrepo-Coupe, N., Huete, A., Davies, K., Cleverly, J., Beringer, J., Eamus,
D., van Gorsel, E., Hutley, L. B., and Meyer, W. S.: MODIS vegetation
products as proxies of photosynthetic potential along a gradient of
meteorologically and biologically driven ecosystem productivity,
Biogeosciences, 13, 5587–5608, 10.5194/bg-13-5587-2016, 2016.
Schmidt, A., Hanson, C., Chan, W. S., and Law, B. E.: Empirical assessment of
uncertainties of meteorological parameters and turbulent fluxes in the
AmeriFlux network, J. Geophys. Res., 117, G04014, 10.1029/2012JG002100,
2012.