Community member post by Sondoss Elsawah

How can we capture the highly qualitative, subjective and rich nature of people’s thinking – their mental models – and translate it into formal quantitative data to be used in numerical models?

This cannot be addressed by a single method or software tool. We need multi-method approaches that have the capacity to take us through the learning journey of eliciting and representing people’s mental models, analysing them, and generating algorithms that can be incorporated into numerical models.

More importantly, this methodology should allow us to see in a transparent way the progression on this learning journey. This transparency is important to build stakeholders’ confidence in the modelling process, and promote reflection and learning.

The ICTAM method described below was developed in the context of integrated water assessment, but it is more widely applicable. Integrated water assessment is a field which integrates knowledge from various scientific disciplines (eg., hydrology, economics, and social science) in order to build an understanding of the complex water problems that arise from the interactions between humans and the environment.

Before describing the method, it is worth addressing the question: Why do we need to bring mental models – the subjective, and often incomplete and flawed assumptions about the surrounding world, biased by personal (eg., past) experience and external factors (eg., media) – to integrated water assessment and other models? The answer is that people’s decisions and actions influence water (and other resource) use directly and indirectly. To change people’s resource use, policies need to understand and target factors that influence how people make decisions, as well as how their decisions affect the biophysical environment, and the feedback effects on future decisions.

The ICTAM method

ICTAM is a step-wise method for bringing qualitative mental models into formal quantitative simulation models. The ICTAM acronym stands for the key methods used throughout the process: Interviews, Cognitive mapping, Time-sequence Unified Modelling Language (UML), All-encompassing framework, and numerical agent-based Models. The figure below shows the steps and outputs.

The process starts by conducting semi-structured interviews with stakeholders. The purpose is to collect data about how people think, interpret information, and make judgements, with minimal intrusion from the researcher.

In the second step, the researcher develops cognitive maps for individuals based on the data collected through interviews. The structure and content of cognitive maps are validated by sharing them with interviewees and seeking their feedback.

Using results from mapping structure and content analysis techniques, the researcher merges individual cognitive maps into a collective map as a unifying view that encompasses individual views. This is step 3.

In step 4, the collective map is used to develop a sequence of conceptual decision models. The conceptual models are transition objects between conceptual and numerical modelling. They provide more formal implementation-based descriptions of the decision making process. This step includes three activities: (1) using UML time sequence diagramming technique to abstract all functions required to represent decisions identified in the collective map, (2) identifying possible models and data required to implement decision functions, and (3) developing pseudo-code representation of those parts of the conceptual model to be implemented.

In the final step, the researcher uses the conceptual decision making models to create a detailed agent-based model that can be executed. The pseudo-code is translated into an actual code implementation. For the inner working of the model, this step involves using additional quantitative data (eg., from literature reviews) to specify thresholds and functional forms of certain functions used by ‘agents’ in the agent-based model.

ICTAM accommodates the complexities of human decision making and behaviour, moving beyond simple treatments of human response as a single parameter and simplistic rational assumptions about human cognition and behaviour.

The process is cyclic. At any step, the researcher can revisit past data analysis, examining any inconsistencies and omissions. Depending on the project’s objective and degree of stakeholder participation in the process, the researcher can share outputs from each step with them. This can serve multiple purposes, such as data validation, engaging participants in the modelling process, and using a particular output from the process to achieve a learning and communication outcome (eg., using cognitive maps to communicate to the group about individual mental models, information gaps, inconsistencies).

What does ICTAM offer modellers?

1. It leverages the strengths of mixing methods by bringing together two well-established methods: cognitive mapping and agent-based modelling. Cognitive mapping taps into the richness and diversity of subjective mental models and decision making processes. However, the conceptual nature of cognitive mapping limits its capacity to simulate and visualise the effects of decisions over time. Simulation based approaches such as agent-based modelling overcome this limitation.
2. It is easy to explain modelling artefacts. The graphical format of the cognitive maps, and the fact that they are built using natural everyday language that stakeholders use, make them easy-to-explain tools to communicate about mental models and complex systems between stakeholders, modellers and software developers.
3. It aggregates individual mental models into collective views. The network structure allows for capturing and visualising complex interactions between system processes. Analysing the structural properties of the maps, along with the content analysis, allows the researcher to integrate different views into composite maps.
4. It provides modelling clarity and transparency. The progression from qualitative subjective data, to formal UML decision models, and agent simulation provides transparency and clarity. At any point in the modelling process, the modeller can share outputs with decision makers and revisit previous steps.

I am very interested to hear of other examples of how people deal with the challenges of eliciting and analysing mentals models, especially when the objective is to develop numerical policy assessment models. I will also be excited to know if people have ideas of case studies where ICTAM can be applied and further developed.

To find out more, see:

Elsawah, S., Guillaume, J. H. A., Filatova, T., Rook, J., and Jakeman, A. J. (2015). A methodology for eliciting, representing, and analysing stakeholder knowledge for decision making on complex socio-ecological systems: From cognitive maps to agent-based models. Journal of Environmental Management, 151, 500-516. Online (DOI): doi:10.1016/j.jenvman.2014.11.028.

Biography: Sondoss Elsawah is a senior lecturer at the University of New South Wales, Canberra. She comes from an operations research background. Her research focuses on the development and use of multi-method approaches to support learning and decision making in complex socio-ecological and socio-technical decision problems. Application areas include natural resource management and defence capability management. Her recent work focuses on designing and conducting laboratory experiments to examine the effectiveness of simulation models in understanding how people make decisions about dynamic decision making problems.

This blog post is one of a series resulting from the first meeting in March 2016 of the Core Modelling Pursuit. This pursuit is part of the theme Building Resources for Complex, Action-Oriented Team Science funded by the National Socio-Environmental Synthesis Center (SESYNC).