Choosing a model: If all you have is a hammer…

By Jen Badham

As a modeller, I often get requests from research or policy colleagues along the lines of ‘we want a model of the health system’. It’s relatively easy to recognise that ‘health system’ is too vague and needs explicit discussion about the specific issue to be modelled. It is much less obvious that the term ‘model’ also needs to be refined. In practice, different modelling methods are more or less appropriate for different questions. So how is the modelling method chosen?

Sometimes the colleague will actually name a technique, for example ‘we want an agent-based model of an intervention to encourage physical activity’. However, this choice may not be informed. It might reflect something that he or she saw in a paper, at a conference, or whatever some previous modeller used. On the other hand, the colleague may have sufficient knowledge to have consciously selected an appropriate method.

From the other direction, many modellers work with only system dynamics or agent-based modelling or some other specific technique. As with other fields, it takes a great deal of experience to be really good at any one approach and there is insufficient time for any one person to be good at everything. Furthermore, people are typically employed for their skills, not for skills they wish to learn, so the usual path is to gain deeper skills rather than broader skills. Consequently, it is easy for the modeller to slip into using the term ‘model’ for whatever technique they use and automatically make that association without considering that others may mean something different.

However, there are many factors that contribute to whether a particular technique is appropriate for a particular project. For example, agent-based modelling is good when modelling humans who make decisions based on what’s happening around them because the model can simulate each person’s behaviour individually. On the other hand, this detail is irrelevant when what’s important is the amount of something and how quickly that changes – such as how much rainfall reaches a river. In the latter case, system dynamics would be a better tool.

As a modeller, I must overcome my natural inclination to go with the tool I know best. So I must resist the tendency to treat everything as a nail because I happen to have a hammer. But it also helps if my model-seeking colleague is aware that there are screwdrivers, spanners and drills and describes the problem in enough detail to work out which is best. Good, early discussions about a modelling problem are essential to get a good model. It would be embarrassing and costly to get to the first version of a model before discovering that the modeller is building something that is completely different from what was expected.

Some key questions for this discussion are:

• If the research or policy colleague request a specific technique: What is it about that technique that is attractive?
• What is the model going to be used for (decisions, understanding …)?
• What is the key driver in the system to be modelled?

Of course, many other questions will need to be discussed in later meetings, including data, funding and timing. However, I use this initial discussion to work out whether I am actually the right modeller to work on the project, or to identify somebody more appropriate.

More detail on different modelling techniques can be found in:

• Badham., J. (2010). A Compendium of modelling techniques, Integration Insights #12, Integration and Implementation Sciences, Australian National University, Canberra, Australia. (Online): https://i2s.anu.edu.au/wp-content/uploads/2009/10/integration-insight_12.pdf
• Badham., J. (2015). Functionality, accuracy, and feasibility: Talking with modelers. Journal on Policy and Complex Systems, 1, 2: 60-87. (Online) (Joom article viewer): https://joom.ag/h6Db/p62
• Kelly (Letcher), R. A., Jakeman, A. J., Barreteau, O., Borsuk, M. E., El Sawaha, S., Hamiltona, S. H., Henriksene, H. J., Kuikkaf, S., Maierg, H. R., Rizzolih, A. E., van Deldeng, H., Voinov, A. A. (2013). Selecting among five common modelling approaches for integrated environmental assessment and management. Environmental Modelling & Software, 47: 159-181. (Online) (DOI): 10.1016/j.envsoft.2013.05.005

Biography: Jen Badham is a Research Fellow at the Centre for Public Health, Queen’s University Belfast. She has worked as both a modeller and policy advisor and is currently interested in the way in which social networks influence changes in behaviour. She is member of the Core Modeling Pursuit funded by the National Socio-Environmental Synthesis Center (SESYNC).

This blog post is one of a series resulting from the first meeting in March 2016 of the Core Modelling Pursuit. This pursuit is part of the theme Building Resources for Action-Oriented Team Science through Syntheses of Practices and Theories funded by the National Socio-Environmental Synthesis Center (SESYNC).