Blackboxing unknown unknowns through vulnerability analysis

By Joseph Guillaume

Author - Joseph Guillaume
Joseph Guillaume (biography)

What’s a productive way to think about undesirable outcomes and how to avoid them, especially in an unpredictable future full of unknown unknowns? Here I describe the technique of vulnerability analysis, which essentially has three steps:

  • Step 1: Identify undesirable outcomes, to be avoided
  • Step 2: Look for conditions that can lead to such outcomes, ie. vulnerabilities
  • Step 3: Manage the system to mitigate or adapt to vulnerable conditions.

The power of vulnerability analysis is that, by starting from outcomes, it avoids making assumptions about what led to the vulnerabilities. The causes of the vulnerabilities are effectively a ‘black box’, in other words, they do not need to be understood in order to take effective action. The vulnerability itself is either a known known or a known unknown. The causes of the vulnerability, on the other hand, can be unknown unknowns.

We can of course partially open the black box and try to construct an understanding of those causes – turning them into known unknowns. Mitigating a vulnerability relies on having sufficient knowledge to anticipate and counter it. We can, however, also recognise that the black box remains partially unopened – that the vulnerability might occur for reasons we have not anticipated, but we can still monitor to check whether the vulnerability is occurring and adapt accordingly.

Let’s take an example investment decision. We want to store water from wet times to prepare for drought, and we are considering two options: “surface storage” in a dam and “managed aquifer recharge”, involving storing water underground, as groundwater. We want to make our decision based on outcomes – we want to choose the option that provides the greatest net benefit.

There’s a lot we know about costs and benefits of each option. They both have capital costs – to build the dam and infrastructure to infiltrate or inject water underground. They both have maintenance costs. Benefits come from having water available when needed, and a key advantage of storing water underground is that it reduces evaporation – we expect that this means there will be more water available for dry years, which translates to better socio-economic outcomes.

These costs and benefits are uncertain but vulnerability analysis gives us a way of thinking through them. Suppose we had decided to invest in managed aquifer recharge. It would be undesirable if surface storage then turned out to be better value – there would be a “crossover” in our preferred option. What are then our vulnerabilities?

If we look at each of our costs and benefits in turn, surface storage would have the advantage if its costs were lower than expected and benefits higher, and vice versa for managed aquifer recharge. If the cost of infiltration infrastructure rises, or the price of an irrigated crop falls, managed aquifer recharge may no longer be worthwhile. We can investigate how much of a change would cause this crossover to occur. If we look at both uncertainties at once, even small changes in infrastructure cost and crop price may cause a crossover – managed aquifer recharge is even less viable. These vulnerabilities become scenarios we can discuss within our investment planning process (for a description of scenarios, see Bonnie McBain’s blog post). (Info-gap theory as described in Yakov Ben-Haim’s blog post does something similar to vulnerability analysis.)

At this stage, we have not needed to know why the infrastructure cost would rise or crop price would fall. Both remain unknown unknown black boxes. But we can add information if we have it: we might be able to get a fixed price contract for the infrastructure, and we might be able to use price forecasts to evaluate how worried we should be about that vulnerability. And importantly, we can do this while only partially opening the black box, by identifying the vulnerabilities introduced by our new information. A fixed price contract can be associated with a black box probability that the contractor will not finish the job. Our price forecast can be accompanied by an error or a probability distribution with unknown unknown drivers, used for example to maximise expected utility (for a description of expected utility see the blog post by Siobhan Bourke and Emily Lancsar).

Using vulnerability analysis to work backwards from outcomes provides a powerful way of working with unknown unknowns, gradually identifying known unknowns as we come across them, while making the best use of what we consider known knowns.

What has your experience been with vulnerability analyses? Have you seen them used to blackbox unknown unknowns in practice?

To find out more:

Arshad, M., Guillaume, JHA. and Ross, A. (2014). Assessing the Feasibility of Managed Aquifer Recharge for Irrigation under Uncertainty. Water, 6 (9): 2748–69. (Online) (DOI):

Guillaume JHA, Arshad M, Jakeman AJ, Jalava M, Kummu M (2016) Robust Discrimination between Uncertain Management Alternatives by Iterative Reflection on Crossover Point Scenarios: Principles, Design and Implementations. Environmental Modelling & Software, 83: 326–43. (Online) (DOI):

Biography: Joseph Guillaume PhD is a DECRA (Discovery Early Career Researcher Award) Research Fellow in the Fenner School of Environment & Society at the Australian National University in Canberra, Australia. He is an integrated modeller with a particular interest in uncertainty and decision support. Application areas have focussed primarily on water resources. Ongoing work involves providing a synthesis of the many ways we communicate about uncertainty, and their implications for modelling and decision support.

This blog post is part of a series on unknown unknowns as part of a collaboration between the Australian National University and Defence Science and Technology.

For the six other blog posts already published in this series, see:

Scheduled blog posts in this series:
December 3: Yin-yang thinking – A solution to dealing with unknown unknowns? by Christiane Prange and Alicia Hennig
January 14, 2020: Detecting non-linear change ‘inside-the-system’ and ‘out-of-the-blue’ by Susan van ‘t Klooster and Marjolijn Haasnoot
January 28, 2020: How can resilience benefit from planning? by Pedro Ferreira
February 11, 2020: Why do we protect ourselves from unknown unknowns? by Bem Le Hunte

Using a cartoon video to achieve research impact

By Darren Gray, Yuesheng Li and Don McManus

Darren Gray
Darren Gray (biography)

In the right circumstances, a cartoon video can be an effective way to communicate research information. But what’s involved in developing a cartoon video?

This blog post is based on our experience as a Chinese-Australian partnership in developing an educational cartoon video (The Magic Glasses, link at end of post) which aimed to prevent soil-transmitted helminths (parasitic worm) infections in Chinese schoolchildren. We believe that the principles we applied are more broadly applicable and share them here. Continue reading

Fourteen knowledge translation competencies and how to improve yours

By Genevieve Creighton and Gayle Scarrow

Genevieve Creighton
Genevieve Creighton (biography)

Knowledge translation encompasses all of the activities that aim to close the gap between research and implementation.

What knowledge, skills and attitudes (ie., competencies) are required to do knowledge translation? What do researchers need to know? How about those who are using evidence in their practice?

As the knowledge translation team at the Michael Smith Foundation for Health Research, we conducted a scoping review of the skills, knowledge and attitudes required for effective knowledge translation (Mallidou et al., 2018). We also gathered tools and resources to support knowledge translation learning. Continue reading

Designing scenarios to guide robust decisions

Community member post by Bonnie McBain

Bonnie McBain (biography)

What makes scenarios useful to decision makers in effectively planning for the future? Here I discuss three aspects of scenarios:

  • goals;
  • design; and,
  • use and defensibility.

Goals of scenarios

Since predicting the future is not possible, it’s important to know that scenarios are not predictions. Instead, scenarios stimulate thinking and conversations about possible futures. Continue reading

Agent-based modelling for knowledge synthesis and decision support

Community member post by Jen Badham

Jen Badham (biography)

The most familiar models are predictive, such as those used to forecast the weather or plan the economy. However, models have many different uses and different modelling techniques are more or less suitable for specific purposes.

Here I present an example of how a game and a computerised agent-based model have been used for knowledge synthesis and decision support.

The game and model were developed by a team from the Centre de Coopération Internationale en Recherche Agronomique pour le Développement (CIRAD), a French agricultural research organisation with an international development focus. The issue of interest was land use conflict between crop and cattle farming in the Gnith community in Senegal (D’Aquino et al. 2003).

Agent-based modelling is particularly effective where understanding is more important than prediction. This is because agent-based models can represent the real world in a very natural way, making them more accessible than some other types of models. Continue reading

Managing uncertainty in decision making: What can we learn from economics?

Community member post by Siobhan Bourke and Emily Lancsar

Siobhan Bourke (biography)

How can researchers interested in complex societal and environmental problems best understand and deal with uncertainty, which is an inherent part of the world in which we live? Accidents happen, governments change, technological innovation occurs making some products and services obsolete, markets boom and inevitably go bust. How can uncertainty be managed when all possible outcomes of an action or decision cannot be known? In particular, are there lessons from the discipline of economics which have broader applicability? Continue reading