Journal cover Journal topic
Biogeosciences An interactive open-access journal of the European Geosciences Union
Biogeosciences, 10, 3313-3340, 2013
© Author(s) 2013. This work is distributed under
the Creative Commons Attribution 3.0 License.
Research article
17 May 2013
A comprehensive benchmarking system for evaluating global vegetation models
D. I. Kelley1, I. C. Prentice1,2, S. P. Harrison1,3, H. Wang1,4, M. Simard5, J. B. Fisher5, and K. O. Willis1 1Department of Biological Sciences, Macquarie University, North Ryde, NSW 2109, Australia
2Grantham Institute for Climate Change, and Department of Life Sciences, Imperial College, Silwood Park Campus, Ascot SL5 7PY, UK
3Geography & Environmental Sciences, School of Human and Environmental Sciences, Reading University, Whiteknights, Reading, RG6 6AB, UK
4State Key Laboratory of Vegetation and Environmental Change, Institute of Botany, Chinese Academy of Science, Xiangshan Nanxincun 20, 100093 Beijing, China
5Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA
Abstract. We present a benchmark system for global vegetation models. This system provides a quantitative evaluation of multiple simulated vegetation properties, including primary production; seasonal net ecosystem production; vegetation cover; composition and height; fire regime; and runoff. The benchmarks are derived from remotely sensed gridded datasets and site-based observations. The datasets allow comparisons of annual average conditions and seasonal and inter-annual variability, and they allow the impact of spatial and temporal biases in means and variability to be assessed separately. Specifically designed metrics quantify model performance for each process, and are compared to scores based on the temporal or spatial mean value of the observations and a "random" model produced by bootstrap resampling of the observations. The benchmark system is applied to three models: a simple light-use efficiency and water-balance model (the Simple Diagnostic Biosphere Model: SDBM), the Lund-Potsdam-Jena (LPJ) and Land Processes and eXchanges (LPX) dynamic global vegetation models (DGVMs). In general, the SDBM performs better than either of the DGVMs. It reproduces independent measurements of net primary production (NPP) but underestimates the amplitude of the observed CO2 seasonal cycle. The two DGVMs show little difference for most benchmarks (including the inter-annual variability in the growth rate and seasonal cycle of atmospheric CO2), but LPX represents burnt fraction demonstrably more accurately. Benchmarking also identified several weaknesses common to both DGVMs. The benchmarking system provides a quantitative approach for evaluating how adequately processes are represented in a model, identifying errors and biases, tracking improvements in performance through model development, and discriminating among models. Adoption of such a system would do much to improve confidence in terrestrial model predictions of climate change impacts and feedbacks.

Citation: Kelley, D. I., Prentice, I. C., Harrison, S. P., Wang, H., Simard, M., Fisher, J. B., and Willis, K. O.: A comprehensive benchmarking system for evaluating global vegetation models, Biogeosciences, 10, 3313-3340,, 2013.
Publications Copernicus