A model inter-comparison study to examine limiting factors in modelling  Australian tropical savannas

Whitley, Rhys; Beringer, Jason; Hutley, Lindsay B.; Abramowitz, Gab; De Kauwe, Martin G.; Duursma, Remko; Evans, Bradley; Haverd, Vanessa; Li, Longhui; Ryu, Youngryel; Smith, Benjamin; Wang, Ying-Ping; Williams, Mathew; Yu, Qiang

doi:https://doi.org/10.5194/bg-13-3245-2016

Articles | Volume 13, issue 11

https://doi.org/10.5194/bg-13-3245-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Special issue:

OzFlux: a network for the study of ecosystem carbon and water...

https://doi.org/10.5194/bg-13-3245-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 13, issue 11

Research article

|

03 Jun 2016

Research article |

| 03 Jun 2016

A model inter-comparison study to examine limiting factors in modelling Australian tropical savannas

Rhys Whitley, Jason Beringer, Lindsay B. Hutley, Gab Abramowitz, Martin G. De Kauwe, Remko Duursma, Bradley Evans, Vanessa Haverd, Longhui Li, Youngryel Ryu, Benjamin Smith, Ying-Ping Wang, Mathew Williams, and Qiang Yu

Download

Final revised paper (published on 03 Jun 2016)
Supplement to the final revised paper
Preprint (discussion started on 02 Dec 2015)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC C9409: 'Referee comment', Anonymous Referee #1, 28 Jan 2016
RC C9410: 'Review: A model inter-comparison study to examine limiting factors in modeling Australian tropical savannas', Anonymous Referee #2, 29 Jan 2016
AC C9699: 'Response to Reviewer Comments', Rhys Whitley, 12 Feb 2016

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (15 Feb 2016) by Natascha Kljun

AR by Rhys Whitley on behalf of the Authors (02 Mar 2016) Author's response Manuscript

ED: Referee Nomination & Report Request started (02 Mar 2016) by Natascha Kljun

RR by Anonymous Referee #2 (12 Apr 2016)

Suggestions for revision or reasons for rejection

On a strictly factual basis, the authors have responded to my initial comments, and made changes to the paper that address them. It still seems to me that the discussion is fairly clinical, especially for Figures 2-4. I think that there is more information here than what the authors are telling us, that what the reader is provided is more basic reportage than insight into savanna behavior and models’ representation of it. I’m sympathetic to the fact that there is a massive amount of information on these graphs, and not every detail can be described. But it seems to me that there are causal explanations, having to do with model structure and ecosystem behavior, beyond a recitation of which models did what.

But I appreciate that the authors are showing us model results in Section 3, and then talking about implications and interpretation in the Discussion and conclusions (Sections 4 and 5).

I do still have a few comments and questions about figures 3 and 4:

Figure 3:
There is a paragraph about the empirical benchmarks (lines 356-363), but not really in the context of what they show in this figure. There are places where emp2/emp3 perform quite well (dry season LE Howard Spgs), places where none of them seem to have a handle (dry season GPP, Dry River). The authors briefly mention the lack of internal modes and storage capability in emp1-3, but we’re not given much more than that. For example, when emp1 has a peak at a similar value to what is observed, the density is generally much too high. When the peak density is reasonable, there is generally an offset in value. What does this mean? Anything? Does emp1 have value at all other than a low bar for models to clear?

Figure 4:
Do the bias vs. flux plots add information beyond what is in the time-course plots? The other reviewer touched on this when asking about the regression lines-do they give us pertinent information or not? There is no discussion about what the bias vs. flux plots mean, beyond what the reader can see for him/herself. CABLE has positive slope at all sites for LE, and crosses zero. To me, this says that during the dry season, at all sites, CABLE underestimates LE. In the wet season, LE is overestimated. But we can see that from the time-course plot. In this case the bias/flux plot seems superfluous: Are there other cases where they are important to the analysis, or can they be discarded?

The positive residuals for BIOS2 at Howard Springs and Adelaide River in Apr-Jun are interesting. That says to me that at these wetter sites, the ‘real world’ dries out quicker than the model does, and water is available for ET in the model. This doesn’t happen in this model at the drier sites. Also, BIOS; Howard/Adelaide are negatively biased in the wet season, more than other sites. With a 10m deep soil for water storage, and 5m root depth for access, I might think that BIOS would do a better job in these wetter sites, but instead the comparison is better in dry sites. Some commentary about some of these types of behaviors in the context of model architecture would be welcome, but I don’t demand it.
The bias vs. flux for LPJGUESS/GPP show a negative slope for all sites but the driest two, which have a positive slope. Does this mean anything, or are these slopes indicating trends that really aren’t there (it appears that the bias at Sturt Plains is always positive). If the latter, is it possible that these regression lines are suggesting facts not in evidence?

If there is actually ‘a larger tendency towards underestimation in the wet than in the dry season’ then we should see a negative bias at larger GPP/LE at all sites. I see this for CABLE;GPP, LPJGUESS;LE/GPP, MAESPA;LE. But I don’t see it for other metrics (CABLE; LE, BESS;LE/GPP, MAESPA; GPP). Am I misinterpreting something?

Typos and errata
Line 100: change ‘savannas’ to ‘savanna’
Line 126: change ‘their ability at simulating’ to ‘their ability to simulate’
Line 143: change ‘savannas’ to ‘savanna’
Line 169: change ‘Table 1, and’ to ‘Table 1, which’
Line 174. Period is on the subscript
Line 212: change ‘simulate water; carbon’ to ‘simulate water and carbon’
Line 220: change ‘BIOS is fine-spatial-resolution’ to ‘BIOS is a fine spatial resolution’
Line 223: change ‘similarities reduces’ to ‘similarities reduce’
Line 223: remove comma after ‘truly’
Line 260: delete ‘then’
Line 261: PALS is given as acronym before it is described in lines 266-267
Line 308: should be ‘Table A1’, not ‘Table B1’

Line 319: delete ‘in the determination of these metrics’
Line 329: add ‘each’ after ‘flux and’
Line 339: delete ‘savanna’. All sites are savanna sites
Line 379: delete ‘vegetation’. Transpiration is from vegetation by definition.
Line 382: delete ‘and’ (first word of line)
Line 457: insert ‘it’ between ‘although’ and ‘performed’
Line 495: should be ‘shrubs are established’ or ‘shrubs have been established’
Lines 530-533: once you say that up to 75% of total LE is from understory herbaceous transpiration, mentioning that C4 understorey will be a large contributor is redundant.
Line 560: change ‘of TBMs’ to ‘of the TBMs’
Line 561: delete ‘that we discuss below’
Line 592: change semi-colon at the end of the line to a comma
Lines 622, 623: change semi-colons to commas
Line 654: delete ‘on’
Line 659. Remove dash before ‘The’

Hide

ED: Publish subject to minor revisions (Editor review) (13 Apr 2016) by Natascha Kljun

AR by Rhys Whitley on behalf of the Authors (16 May 2016) Author's response Manuscript

ED: Publish as is (17 May 2016) by Natascha Kljun

AR by Rhys Whitley on behalf of the Authors (23 May 2016)

Download

Article (1814 KB)
Full-text XML

Short summary

In this study we assess how well terrestrial biosphere models perform at predicting water and carbon cycling for savanna ecosystems. We apply our models to five savanna sites in Northern Australia and highlight key causes for model failure. Our assessment of model performance uses a novel benchmarking system that scores a model’s predictive ability based on how well it is utilizing its driving information. On average, we found the models as a group display only moderate levels of performance.