Biogeosciences, 9, 3857-3874, 2012
© Author(s) 2012. This work is distributed under
the Creative Commons Attribution 3.0 License.
Reviews and syntheses
09 Oct 2012
A framework for benchmarking land models
Y. Q. Luo1, J. T. Randerson2, G. Abramowitz3, C. Bacour4, E. Blyth5, N. Carvalhais6,7, P. Ciais8, D. Dalmonech6, J. B. Fisher9, R. Fisher10, P. Friedlingstein11, K. Hibbard12, F. Hoffman13, D. Huntzinger14, C. D. Jones15, C. Koven16, D. Lawrence10, D. J. Li1, M. Mahecha6, S. L. Niu1, R. Norby13, S. L. Piao17, X. Qi1, P. Peylin8, I. C. Prentice18, W. Riley16, M. Reichstein6, C. Schwalm14, Y. P. Wang19, J. Y. Xia1, S. Zaehle6, and X. H. Zhou20 1Department of Microbiology and Plant Biology, University of Oklahoma, Norman, OK 73019, USA
2Department of Earth System Science, University of California, Irvine, CA 92697, USA
3Climate Change Research Centre, University of New South Wales, Sydney, Australia
4Laboratory of Climate Sciences and the Environment, Joint Unit of CEA-CNRS, Gif-sur-Yvette, France
5Centre for Ecology and Hydrology, Wallingford, Oxfordshire, OX10 8BB, UK
6Max-Planck-Institute for Biogeochemistry, Jena, Germany
7Departamento de Ciências e Engenharia do Ambiente, DCEA, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
8Laboratoire des Sciences du Climat et de l'Environnement, CEA-CNRS-UVSQ, CE Orme des Merisiers, 91191 Gif sur Yvette, France
9Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109 USA
10National Centre for Atmospheric Research, Boulder, CO, USA
11College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, UK
12Pacific Northwest National Laboratory, Richland, WA, USA
13Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA
14School of Earth Science and Environmental Sustainability, Northern Arizona University, Flagstaff, AZ 86011, USA
15Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB, UK
16Earth Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA
17Department of Ecology, Peking University, Beijing 100871, China
18Department of Biological Sciences, Macquarie University, NSW 2109 Sydney, Australia
19CSIRO Marine and Atmospheric Research PMB #1and Centre for Australian Weather and Climate Research, Aspendale, Victoria 3195, Australia
20Research Institute of the Changing Global Environment, Fudan University, 220 Handan Road, Shanghai 200433, China
Abstract. Land models, which have been developed by the modeling community in the past few decades to predict future states of ecosystems and climate, have to be critically evaluated for their performance skills of simulating ecosystem responses and feedback to climate change. Benchmarking is an emerging procedure to measure performance of models against a set of defined standards. This paper proposes a benchmarking framework for evaluation of land model performances and, meanwhile, highlights major challenges at this infant stage of benchmark analysis. The framework includes (1) targeted aspects of model performance to be evaluated, (2) a set of benchmarks as defined references to test model performance, (3) metrics to measure and compare performance skills among models so as to identify model strengths and deficiencies, and (4) model improvement. Land models are required to simulate exchange of water, energy, carbon and sometimes other trace gases between the atmosphere and land surface, and should be evaluated for their simulations of biophysical processes, biogeochemical cycles, and vegetation dynamics in response to climate change across broad temporal and spatial scales. Thus, one major challenge is to select and define a limited number of benchmarks to effectively evaluate land model performance. The second challenge is to develop metrics of measuring mismatches between models and benchmarks. The metrics may include (1) a priori thresholds of acceptable model performance and (2) a scoring system to combine data–model mismatches for various processes at different temporal and spatial scales. The benchmark analyses should identify clues of weak model performance to guide future development, thus enabling improved predictions of future states of ecosystems and climate. The near-future research effort should be on development of a set of widely acceptable benchmarks that can be used to objectively, effectively, and reliably evaluate fundamental properties of land models to improve their prediction performance skills.

Citation: Luo, Y. Q., Randerson, J. T., Abramowitz, G., Bacour, C., Blyth, E., Carvalhais, N., Ciais, P., Dalmonech, D., Fisher, J. B., Fisher, R., Friedlingstein, P., Hibbard, K., Hoffman, F., Huntzinger, D., Jones, C. D., Koven, C., Lawrence, D., Li, D. J., Mahecha, M., Niu, S. L., Norby, R., Piao, S. L., Qi, X., Peylin, P., Prentice, I. C., Riley, W., Reichstein, M., Schwalm, C., Wang, Y. P., Xia, J. Y., Zaehle, S., and Zhou, X. H.: A framework for benchmarking land models, Biogeosciences, 9, 3857-3874,, 2012.
Publications Copernicus