Managing deep uncertainty: Exploratory modeling, adaptive plans and joint sense making

Community member post by Jan Kwakkel

jan-kwakkel
Jan Kwakkel (biography)

How can decision making on complex systems come to grips with irreducible, or deep, uncertainty? Such uncertainty has three sources:

  1. Intrinsic limits to predictability in complex systems.
  2. A variety of stakeholders with different perspectives on what the system is and what problem needs to be solved.
  3. Complex systems are generally subject to dynamic change, and can never be completely understood.

Deep uncertainty means that the various parties to a decision do not know or cannot agree on how the system works, how likely various possible future states of the world are, and how important the various outcomes of interest are. This implies that, under deep uncertainty, it is possible to enumerate possible representations of the system, plausible futures, and relevant outcomes of interest, without being able to rank order them in terms of likelihood or importance.

There is an emerging consensus that effort needs to be devoted to making any decision regarding a complex system robust with respect to such uncertainties. A plan is robust if its expected performance is only weakly affected by deep uncertainty. Alternatively, a plan can be understood as being robust if no matter how the future turns out, there is little cause for regret (the so-called “no regrets” approach to decision making).

Over the last decade a new paradigm, known as ‘decision-making under deep uncertainty’, has emerged to support the development of robust plans. This paradigm rests on three key ideas: (i) exploratory modeling; (ii) adaptive planning; and, (iii) joint sense-making.

Exploratory modelling

Exploratory modeling allows examination of the consequences of the various irreducible uncertainties for decision-making. Typically, in the case of complex systems this involves the use of computational scenario approaches (see also the blog post by Laura Schmitt-Olabisi on Dealing with deep uncertainty: Scenarios).

A set of models that is plausible or interesting in a given context is generated by the uncertainties associated with the problem of interest, and is constrained by available data and knowledge. A single model drawn from the set is not a prediction. Rather, it is a computational ‘what-if’ experiment that reveals how the real world system would behave if the various assumptions this particular model makes about the various uncertainties were correct.

A single ‘what-if’ experiment is typically not that informative, other than suggesting the plausibility of its outcomes. Instead, exploratory modeling aims to support reasoning and decision-making on the basis of the set of models. Thus exploratory modeling involves searching through the set of models using (many-objective) optimization algorithms, and sampling over the set of models using computational design of experiments and global sensitivity analysis techniques. By searching through the set of models, one can identify which (combination of) uncertainties negatively affects the outcomes of interest. In light of this, actions can be iteratively refined to be robust with respect to these uncertainties.

Adaptive planning

Adaptive planning means that plans are designed from the outset to be altered over time in response to how the future actually unfolds. In this way, modifications are planned for, rather than taking place in an ad hoc manner. The flexibility of adaptive plans is a key means of achieving decision robustness.

This means that a wide variety of futures has to be explored. Insight is needed into which actions are best suited to which futures, as well as what signals from the unfolding future can be monitored in order to ensure the timely implementation of the appropriate actions. Adaptive planning thus involves a paradigm shift from planning in time, to planning conditional on observed developments.

Joint sense-making

Decision making on uncertain complex systems generally involves multiple actors who have to come to agreement. In such a situation, planning and decision-making require an iterative approach that facilitates learning across alternative framings of the problem, and learning about stakeholder preferences and trade-offs, in pursuit of a collaborative process of discovering what is possible.

Various decision analytic techniques can be used to enable a constructive learning process amongst the stakeholders and analysts. Decision analysis in this conceptualization must shift away from the a priori agreement on (or imposition of assumptions on) the probability of alternative states of the world and the way in which competing objectives are to be aggregated, with the aim of producing a preference ranking of decision alternatives. Instead decision analysis must shift to an a posteriori exploration of trade-offs amongst objectives and their robustness across possible futures. Decision analysis should move away from trying to dictate the right choice, and instead aim at enabling deliberation and joint sense-making amongst the various parties to decision. (For more on sense-making, see Bethany Laursen’s blog post on Making sense of wicked problems).

Closing remarks

Exploratory modeling, adaptive planning, and joint sense-making are the three key ideas that underpin the emerging paradigm of decision making under deep uncertainty. Various specific approaches that exemplify these ideas include (many-objective) robust decision-making, dynamic adaptive policy pathways, decision scaling, info-gap decision theory, adaptive policy making and assumption based planning. Notwithstanding the many technical differences that exist between these approaches, there is an increasing emphasis on what is shared. In practice also, increasingly people are adopting aspects from multiple approaches in order to offer context-specific support for making decisions under deep uncertainty.

What has your experience been with decision making under deep uncertainty? What methods have you found to be useful?

Further reading:
Bankes, S. C. (1993). Exploratory Modeling for Policy Analysis. Operations Research, 4, 3: 435-449.

Haasnoot, M., Kwakkel, J. H., Walker, W. E. and ter Maat, J. (2013). Dynamic adaptive policy pathways: A method for crafting robust decisions for a deeply uncertain world. Global Environmental Change, 23: 485-498. Online (DOI): 10.1016/j.gloenvcha.2012.12.006

Herman, J. D., Reed, P. M., Zeff, H. B. and Characklis, G. W. (2015). How should robustness be defined for water systems planning under change. Journal of Water Resources Planning and Management, 141, 10. Online (DOI): 10.1061/(ASCE)WR.1943-5452.0000509

Kwakkel, J. H., Walker, W. E. and Haasnoot, M. (2016). Coping with the Wickedness of Public Policy Problems: Approaches for Decision Making under Deep Uncertainty. Journal of Water Resources Planning and Management. Online (DOI): 10.1061/(ASCE)WR.1943-5452.0000626

Lempert, R. J., Groves, D. G., Popper, S. W. and Bankes, S. C. (2006). A General, Analytic Method for Generating Robust Strategies and Narrative Scenarios. Management Science, 52: 514-528. Online (DOI): 10.1287/mnsc.1050.0472

Walker, W. E., Haasnoot, M. and Kwakkel, J. H. (2013). Adapt or Perish: A Review of Planning Approaches for Adaptation under Deep Uncertainty. Sustainability, 5: 955-979. Online (DOI): 10.3390/su5030955

Biography: Jan Kwakkel is an associate professor at Delft University of Technology in the faculty of Technology, Policy and Management. He has a background in systems engineering and policy analysis for transport systems. His current research focuses on supporting decision making under deep uncertainty. This involves the development of taxonomies and frameworks for uncertainty analysis and adaptive planning, as well as research on model-based scenario approaches for designing adaptive plans. He has applied his research in various domains, including transportation, energy systems, and health. His primary application domain is climate adaptation in the water sector. A secondary research interest is in text mining of science and patent databases. His research is currently funded for four years through a personal development grant of the Dutch National Science Foundation.

Good practices in system dynamics modelling

Community member post by Sondoss Elsawah and Serena Hamilton

sondoss-elsawah
Sondoss Elsawah (biography)

Too often, lessons about modelling practices are left out of papers, including the ad-hoc decisions, serendipities, and failures incurred through the modelling process. The lack of attention to these details can lead to misperceptions about how the modelling process unfolds.

serena-hamilton
Serena Hamilton (biography)

We are part of a small team that examined five case studies where system dynamics was used to model socio-ecological systems. We had direct and intimate knowledge of the modelling process and outcomes in each case. Based on the lessons from the case studies as well as the collective experience of the team, we compiled the following set of good practices for systems dynamics modelling of complex systems. Continue reading

What’s in a name? The role of storytelling in participatory modeling

Community member post by Alison Singer

singer
Alison Singer (biography)

That which we call a rose,
by any other name would smell as sweet.

That Shakespeare guy really knew what he was talking about. A rose is what it is, no matter what we call it. A word is simply a cultural agreement about what we call something. And because language is a common thread that binds cultures together, participatory modeling – as a pursuit that strives to incorporate knowledge and perspectives from diverse stakeholders – is prime for integrating stories into its practice.

To an extent, that’s what every modeling activity does, whether it’s through translating an individual’s story into a fuzzy cognitive map, or into an agent-based model. But I would argue that the drive to quantify everything can sometimes make us lose the richness that a story can provide. Continue reading

Sharing integrated modelling practices – Part 2: How to use “patterns”?

Community member post by Sondoss Elsawah and Joseph Guillaume

sondoss-elsawah
Sondoss Elsawah (biography)

In part 1 of our blog posts on why use patterns, we argued for making unstated, tacit knowledge about integrated modelling practices explicit by identifying patterns, which link solutions to specific problems and their context. We emphasised the importance of differentiating the underlying concept of a pattern and a pattern artefact – the specific form in which the pattern is explicitly described. Continue reading

Sharing integrated modelling practices – Part 1: Why use “patterns”?

Community member post by Sondoss Elsawah and Joseph Guillaume

sondoss-elsawah
Sondoss Elsawah (biography)

How can modellers share the tacit knowledge that accumulates over years of practice?

In this blog post we introduce the concept of patterns and make the case for why patterns are a good candidate for transmitting the ‘know-how’ knowledge about modelling practices. We address the question of how to use patterns in a second blog post. Continue reading

Looking for patterns: An approach for tackling tough problems

Community member post by Scott D. Peckham

Scott D. Peckham (biography)

What does the word ‘pattern’ mean to you? And how do you use patterns in addressing complex problems?

Patterns are repetitions. These can be in space, such as patterns in textiles and wallpaper, which include houndstooth, herringbone, paisley, plaid, argyle, checkered, striped and polka-dotted.

The pattern concept can also be applied to repetitions in time, as occur in music. Those who know the temporal patterns can classify a piece of music as a blues, waltz or salsa. For each of these types of music, there are also classic dance steps, that usually go by the same name; these are patterns of movement in space and time.

These examples get to the idea that patterns can be viewed more generally as any type of repetitive structure or recurring theme that we can look for and potentially recognize or discover and then assign a memorable name to, such as “houndstooth” or “waltz”. Recognizing the pattern may then indicate a particular course of action, such as “perform dance moves that go with a waltz”.

The ability to recognize a pattern and then take appropriate action is something that we associate with intelligence. Continue reading