At a time when discussions in public health and epidemiology in general increasingly grapple with ‘complexity’, as well as focusing tightly on specific areas of interest (such of transport and health), it is likewise useful to step back to more abstract considerations. A general shift from the dominance of statistical-based models, which are arguably not well suited to represent change, towards dynamic modelling (Agent Based Modelling and System Dynamic Modelling have become prominent in the literature), and the possibility of including clearly specified local contextual details represent a major advance; however, such attention to technical and contextual issues may potentially move questions about modelling itself into the background. In any case, as someone who doesn’t specialise in work on transport and health, this is where my attention is drawn…
Many years ago (decades in fact), biologist Richard Levins considered the trade-offs involved when developing a modelling strategy. Unless you’re doing exceptionally badly to begin with, trade-offs are almost inevitable. As ‘new’ modelling methods come into our view, including methods which may appear to offer only benefits, I think Levins’ principles potentially offer useful guidance.
Models can be thought of in terms of three axes: reality, generality, and precision (this typology is not intended to be comprehensive, but rather to capture some critical aspects). In general, gains in any one area are accompanied by losses elsewhere. Briefly, the three situations described by Levins are:
i. Sacrifice realism for generality and precision
This strategy is very familiar to me as it’s typically used in global scale climate and health impact assessments. The aim to quantify impacts as precisely as possible using general relationships that are applicable to all world regions necessitates a loss of realism; in this case, what is happening nationally or below this level is lost. An analogous case may be modelling the general impacts of transport policies at UK-level rather than specific impacts in a particular place.
But despite the apparent limitations of this approach compared to adopting a tighter, local focus, it is not without benefits. For instance, the system dynamics operating at a ‘high’ level are likely to play a role in shaping local conditions and possibilities; viewing the system only at the local level may well miss these dynamics. From this perspective, a loss of realism in terms of the local is an advantage.
ii. Sacrifice generality for realism and precision
‘New’ modelling methods may naturally lead to this strategy as they potentially allow the inclusion of detailed local dynamics. But the degree of detail means generality is lost; the model is only useful for the very specific circumstances being considered. Viewed temporally, the lack of generality may limit the useful lifespan of such a model: as local conditions change, or as conditions at a higher level (e.g. national) change and potentially alter local dynamics, the model may no longer represent reality well.
Another limitation is the amount of data and knowledge required to construct and parameterise such a model: to make precise estimates of the outcome of interest, precise representation of relations and variables is necessary. Often the data or knowledge, particularly for a specific local setting, are not available. In the absence of the required inputs, the model, despite looking ‘fancy’, may tell us little of use about the situation of interest. This leads on to the third strategy…
iii. “Sacrifice precision for generality and realism”
At least in my experience, the goal of producing precise quantitative estimates is beyond discussion: of course precise quantification is the (or at least, a major) goal!
But is this sensible? In any case, it comes at a cost. If no data or quantification of a relation is available, it is either left out or crudely represented. In many cases, so much is left out or represented by ‘best guesses’ (maybe named as ‘expert opinion’) that the resulting quantifications are accompanied by the qualification that they should only be interpreted in terms of magnitude; that is, in a sense, qualitatively. The very sacrifices made to achieve ‘precision’ undermine the value of that precision.
Why not then – in some models, but of course not all – relax the relentless drive for precision and instead aim simply to understand system behaviour generally? Instead of asking, ‘How much will this increase if we do that’, you can ask, for example, ‘If we do this, will this increase of decrease?’. At the very least this type of strategy can be used to get a general understanding of the system of interest, and guide attention to specific aspects that appear to be critical to behaviour.
Because of the inevitable trade-offs, a mixture of strategies seems appropriate, with which is used at any moment being guided by what we know and want to know.
The final strategy is of particular interest to me. We’ll be discussing and developing its potential for transport and health during the second event of the seminar series. It will no doubt require some imaginative input from those who come along…