Five problems with ‘integrated assesment’ models, and what to do about them

farmers in dryland landscape

What are the most appropriate ways of understanding changes in natural resource change in rural areas, particularly in the context of climate change? How can we make use of data that is patchy and uncertain? How can models help decision-making about future management?

These questions are at the heart of three recently published journal articles on Zimbabwe. The three papers focus respectively on climate impacts on livestock feed (in Nkayi), land use intensity patterns (in Wedza) and the prevalence of grass fires (in Mazowe). What connects them is the use of remotely-sensed data on land use with an integrated modelling approach, aimed at policy prescriptions for resource management.

This style of research on natural resource use has become more and more common in recent years, as increasingly detailed data derived from satellite systems has become freely available. Integrated assessment models, modelling everything from climate impacts to crop production to land-use to water scarcity, can be linked to geo-referenced spatial data and parameterised with field-based data collection.

As a style of inquiry, integrated modelling approaches have a number of advantages. Diverse data sources can be combined, and predictions made around key policy issues. But there are also problems – and, in different ways, these three papers illuminate some of them.

Five problems with integrated resource assessment modelling

First, models are always framed by assumptions around problems and solutions. Each of these studies adopts a particular stance, resulting in recommendations for interventions to address the highlighted problem. So, climate change results in feed gaps for livestock, which can be solved by ‘climate smart’ adaptation measures in Nkayi. High land use intensity – excessive extraction of primary production – means that ‘hot spots’ of land degradation ‘externalities’ can be identified for intervention measures in Wedza. Increasing fire frequencies are assumed to be universally a bad thing, not a necessary consequence land clearance or a reflection of natural cycles in savannah dynamics, as fuel load builds up. Instead, recommendations, including the deployment of fire teams, creating fire-breaks and developing monitoring systems, are put forward for Mazowe.

Second, the uncertainties embedded in complex models are legion, meaning that any predictions have to be heavily qualified. These papers all acknowledge important uncertainties. In the assessment of land use intensity against a baseline of net primary production in Wedza, these arise, for example, from problems of estimating primary production in the baseline case, especially below-ground. Linking biomass harvesting to specific areas when livestock move is also recognised as a source of uncertainty. In the analysis of climate impacts on fodder management options in Nkayi, the uncertainties surrounding climate predictions across scenarios is acknowledged, and the model in turn is developed with parameters that are constrained within a ‘reasonable range of uncertainty’. Yet, by the end of the papers, important uncertainties are seemingly put aside in the desire to reach a definitive conclusion for the way forward. The apparent need for prediction, directions for ‘decision-making’ and control-oriented intervention are all-consuming.

Third, the style of argument too often leads to a closing down of discussion of more diverse options. All three papers are structured in the standard way of scientific papers, with propositions tested according to a set of methods, leading to results and conclusions. In the methods section, the qualifications, imperfections and uncertainties are duly noted. But, by the time the results are presented, around a particular quantitative model, such difficult issues are quietly put to the background. By the time of the conclusions, they have all but disappeared, and much stronger causal, predictive statements offer a definitive way forward, frequently hinted at by the original framing. For example, a model of land use intensity Wedza, focused on the extraction of net primary productivity, inevitably side-steps questions of how landscapes are understood, and how future resource use is seen by different groups of people. The social and political dynamics of change are not part of the storyline, despite the attempt to link resource use with different wealth groups.

Fourth, models are only models – simplified ways of thinking about the world – and they certainly can be helpful in thinking through options. But sometimes the assumptions just don’t make sense. Models to have any purchase need some ground-truthing, and some stress-testing with reality. The paper on grass fires shows clearly that there are no statistically significant differences across tenure types in fire frequency and extent. In other words, land reform farmers cannot be blamed, but without field based data, the paper is unable to explain the patterns, and instead uses a model that extrapolates future patterns from the past. In respect of fire, this is rather unlikely – fires due to land clearing will decline as farms and fields are established, while hunting will decline as game animal populations are eliminated. As a result, the regression-based models become detached from likely future realities. Instead, the regressions play a political role: by extrapolating increases in fires, they justify a set of externally-defined interventions.

Finally, the rush to a definitive recommendation for policy too often results in missing out on complex system dynamics, histories and contexts. The paper in this trio on livestock fodder systems, for example, assumes that the ‘feed gap’ will be filled by improved fodder quantity and quality, including the growing of fodder crops and the application of fertiliser to crops to improve stover. And this in dryland Nkayi? Surely not. The paper acknowledges that past attempts at improved fodder management have consistently failed, but does not probe why in the rush to provide an intervention-friendly recommendation aligned with a ‘climate-smart’ intervention narrative.

Styles of science: how to broaden out inquiry and open up debate

All three of these papers make important arguments and present significant data. They all have been peer-reviewed in respectable journals (Agricultural Systems, Ecological Economics and Geocarto International). The data is (mostly) of high quality, the models are consistent (if problematic) and the arguments are clearly made (although open to challenge). But reading these (and these are only exemplars of many, many others, perhaps rather unfairly singled out), the five wider concerns raised above kept coming back.

It makes me uneasy when a style of science closes down debate. Uncertainties are not embraced and alternative interpretations are not given space. An assumption that the end-point must be a science-based ‘smart’ intervention means other possibilities – more social, political for example – are not countenanced. This is less a critique of the particular methods and models, but more the style of policy-oriented science, centred on integrated assessment modelling, now central to a huge industry of ‘global change’ research.

What might an alternative approach look like? Modelling that takes uncertainty seriously would not close down to definitive solutions, but would aim to open debates up. Models that are interrogated with deep, field-based data, thus triangulating between modelling approaches, result in greater robustness and wider interpretation. When reading the papers, I had to ask: are there alternatives to new fodder regimes and crop fertilisation to address the consequences of climate change on livestock production in Nkayi? Of course there are! Does fire management have to be focused always on fire prevention; are fires always bad? Of course not! But such alternatives were not debated.

Suggesting diverse, alternative options for the future – different interpretations and solutions from an open approach to data, evidence and integrated assessment modelling – allows for an engaged, inevitably political debate, about what makes sense for whom. This would make for papers that are less neat, but perhaps ultimately more useful.


This is the fourth of a short series of blogs profiling recent papers on Zimbabwe.

This post was written by Ian Scoones and first appeared on Zimbabweland. Photo credit: Ian Scoones

Leave a Reply

Your email address will not be published. Required fields are marked *