Technical note : Dynamic INtegrated Gap-filling and partitioning for OzFlux ( DINGO )

Standardised, quality-controlled and robust data from flux networks underpin the understanding of ecosystem processes and tools necessary to support the management of natural resources, including water, carbon and nutrients for environmental and production benefits. The Australian regional flux network (OzFlux) currently has 23 active sites and aims to provide a continental-scale national research facility to monitor and assess Australia’s terrestrial biosphere and climate for improved predictions. Given the need for standardised and effective data processing of flux data, we have developed a software suite, called the Dynamic INtegrated Gap-filling and partitioning for OzFlux (DINGO), that enables gap-filling and partitioning of the primary fluxes into ecosystem respiration (Fre) and gross primary productivity (GPP) and subsequently provides diagnostics and results. We outline the processing pathways and methodologies that are applied in DINGO (v13) to OzFlux data, including (1) gap-filling of meteorological and other drivers; (2) gapfilling of fluxes using artificial neural networks; (3) the u threshold determination; (4) partitioning into ecosystem respiration and gross primary productivity; (5) random, model and u uncertainties; and (6) diagnostic, footprint calculation, summary and results outputs. DINGO was developed for Australian data, but the framework is applicable to any flux data or regional network. Quality data from robust systems like DINGO ensure the utility and uptake of the flux data and facilitates synergies between flux, remote sensing and modelling.


Introduction
OzFlux is the regional Australian and New Zealand flux tower network that aims to provide a continental-scale national research facility to monitor and assess Australia's terrestrial biosphere and climate for improved predictions (Beringer et al., 2016).High-quality and reliable data are a crucial foundation in achieving the objectives of the OzFlux network (Beringer et al., 2016) and underpin the process understanding needed to (1) support sound management of natural resources including water, carbon and nutrient resources for environmental and production benefits; (2) monitor, assess, predict and respond to climate change and variability; (3) improve weather and environmental information and prediction; (4) support disaster management and early warning systems needed to meet Australia's priorities in national security; and (5) ensure that Earth system models used to underpin Australia's policies and commitments to international treaties adequately represent Australian terrestrial ecosystem processes (Beringer et al., 2016).Beringer et al. (2016) provide an overview of the evolution, design and current status of OzFlux as well as a brief summary of the instrumentation and data collection that forms the backbone of the network.A detailed description of the quality control and post-processing of the eddy covariance data using OzFluxQC and the data pathway to curation is given by Isaac et al. (2016).In summary, from Beringer et al. (2016), most sites have data loggers that provide the average (usually over 30 min) covariances that are processed through 6 levels using the OzFluxQC standard software processing scripts.Levels 1, 2 and 3 are as follows: L1 is the raw data as received from the flux tower, L2 it the qualitycontrolled data, and L3 is the post-processed, corrected, but not gap-filled data.Quality control measures by OzFluxQC are applied at L2 and comprise checks for plausible value ranges, spike detection and removal, manual exclusion of date and time ranges and diagnostic checks for all quantities used in the flux correction calculations.The quality checks make use of the diagnostic information from the sonic anemometer and the infrared gas analyser.For sites calculating fluxes from the averaged covariances, post-processing includes 2-D coordinate rotation, low-and high-pass frequency correction, conversion of virtual heat flux to sensible heat flux and application of the Webb-Pearman-Leuning (WPL) correction to the latent heat and CO 2 fluxes (see Burba, 2013 for a general description of the data processing pathways).
Steps performed at L3 include the correction of the ground heat flux for storage in the layer above the heat flux plates (Mayocchi and Bristow, 1995) and correction of the CO 2 flux data for storage in the canopy (where available).OzFlux data are available at http://data.ozflux.org.au/.OzFlux sites submit their data to FLUXNET at L3.
Given the international need by the community for standardised data processing to enable effective comparison across biomes and to understand inter-annual variability (Papale et al., 2006), we have developed a software tool to address this need.In this paper, we describe the development and testing of the Dynamic INtegrated Gap-filling and partitioning for the Ozflux (DINGO) system that utilises the L3 data from OzFluxQC to gap-fill and partition the fluxes in ecosystem respiration (Fre) and gross primary productivity (GPP) and subsequently provides diagnostics and results.This paper is not intended to be a thorough review of the data processing, but the application of standard techniques in DINGO for the flux community.DINGO is a research version for OzFlux data, whereas the OzFluxQC system (which has many similar features to DINGO) is considered an operational version.Quality data from robust systems like DINGO ensure the utility and uptake of the flux data and facilitates synergies between flux, remote sensing, modelling and canopy physiological studies.We conclude by looking ahead at the future direction of the DINGO system.

Approach
The overall approach used in DINGO is to take the L3 OzFluxQC data -which has gaps from data processing (data excluded due to values out of range, spike detection or manual exclusion of date and time ranges) or from site issues (instrument or power failure, herbivores, fire, eagles nests, cows, lightning, PI on sabbatical, etc.) -and gap-fill and partition the data using a variety of data sources (Fig. S1).DINGO is programmed in python 2.7 and is currently at ver-sion 13 and publically available on GitHub (https://github.com/jberinge/DINGO13).It is designed to work with OzFlux data produced in NetCDF format by the OzFluxQC (Isaac et al., 2016) and draws on Australian automatic weather station (AWS) data, but it could be adapted for other data sources across other flux sites.The primary interface for the user is through a text-based control file that has information on site characteristics (name, latitude and longitude, the frequency of the flux measurements, i.e. 30 or 60 min, and elevation, file paths (to the OzFluxQC NetCDF files and other ancillary data inputs), data processing options, and data plotting and output formats.In general, prior to the processing steps below, any gaps in fluxes or meteorological quantities of less than 2 h are filled by DINGO using linear interpolation.The pathway for processing is shown in Fig. S1, and each step is outlined in the Supplement.

Conclusions
The OzFlux network has been highly successful in generating standardised measurements and protocols that provide robust primary data.Only via transparent, advanced and consistent QA/QC will we ensure compatibility within the OzFlux network (Beringer et al., 2016) and with international databases (FLUXNET) (Papale et al., 2006), ensuring uptake by the broader scientific community.Through robust software systems such as OzFluxQC (Isaac et al., 2016) and DINGO, we are able to ensure timely and quality gap-filling and partitioning of fluxes that in turn enable a significant uptake of the eddy covariance data for application to a range of research questions as exemplified in Beringer et al. (2016).This includes integration of the eddy covariance and remote sensing datasets for the validation of satellite products (e.g.GPP and ET, Kanniah et al., 2009;Restrepo-Coupe et al., 2016) and to aid the parameterisation of models that rely on remotely sensed data (e.g.GPP, ET, canopy conductance, and light use efficiency (LUE), Barraza et al., 2014Barraza et al., , 2015;;Glenn et al., 2011;Goerner et al., 2011).In addition, OzFlux data have been instrumental in constraining a continent-wide assessment of terrestrial carbon and water cycles (Haverd et al., 2013) and featured in the development of new models (Haverd et al., 2007(Haverd et al., , 2009)).There is utility of the data to support carbon accounting activities (Hutley et al., 2005), as demonstrated in research focussed on the conversion of savanna to pasture (Bristow et al., 2016).Ultimately, flux data are required to address the key ecosystem science questions of OzFlux (Beringer et al., 2016) that are focused on the improved understanding of the responses of carbon and water cycles of Australian ecosystems to current climate and disturbance regimes as well as on impacts of projected future changes to precipitation, temperature and atmospheric CO 2 concentration.Key questions include the following: (1) what are the key drivers of ecosystem productivity (carbon sinks) and greenhouse gas emissions; (2) how resilient is the Aus-tralian ecosystem productivity to an increasingly variable and changing climate; and, (3) what is the current water budget of the dominant Australian ecosystems and how will it change in the future?Data availability.The data used to create the example figures in the manuscripts were from two sites in the OzFlux network.Site data for Whroo can be found in Beringer (2013)  The Supplement related to this article is available online at doi:10.5194/bg-14-1457-2017-supplement.
Competing interests.The authors declare that they have no conflict of interest.