Blackboxing unknown unknowns through vulnerability analysis

By Joseph Guillaume

Author - Joseph Guillaume
Joseph Guillaume (biography)

What’s a productive way to think about undesirable outcomes and how to avoid them, especially in an unpredictable future full of unknown unknowns? Here I describe the technique of vulnerability analysis, which essentially has three steps:

  • Step 1: Identify undesirable outcomes, to be avoided
  • Step 2: Look for conditions that can lead to such outcomes, ie. vulnerabilities
  • Step 3: Manage the system to mitigate or adapt to vulnerable conditions.

The power of vulnerability analysis is that, by starting from outcomes, it avoids making assumptions about what led to the vulnerabilities. The causes of the vulnerabilities are effectively a ‘black box’, in other words, they do not need to be understood in order to take effective action. The vulnerability itself is either a known known or a known unknown. The causes of the vulnerability, on the other hand, can be unknown unknowns.

We can of course partially open the black box and try to construct an understanding of those causes – turning them into known unknowns. Mitigating a vulnerability relies on having sufficient knowledge to anticipate and counter it. We can, however, also recognise that the black box remains partially unopened – that the vulnerability might occur for reasons we have not anticipated, but we can still monitor to check whether the vulnerability is occurring and adapt accordingly.

Let’s take an example investment decision. We want to store water from wet times to prepare for drought, and we are considering two options: “surface storage” in a dam and “managed aquifer recharge”, involving storing water underground, as groundwater. We want to make our decision based on outcomes – we want to choose the option that provides the greatest net benefit.

There’s a lot we know about costs and benefits of each option. They both have capital costs – to build the dam and infrastructure to infiltrate or inject water underground. They both have maintenance costs. Benefits come from having water available when needed, and a key advantage of storing water underground is that it reduces evaporation – we expect that this means there will be more water available for dry years, which translates to better socio-economic outcomes.

These costs and benefits are uncertain but vulnerability analysis gives us a way of thinking through them. Suppose we had decided to invest in managed aquifer recharge. It would be undesirable if surface storage then turned out to be better value – there would be a “crossover” in our preferred option. What are then our vulnerabilities?

If we look at each of our costs and benefits in turn, surface storage would have the advantage if its costs were lower than expected and benefits higher, and vice versa for managed aquifer recharge. If the cost of infiltration infrastructure rises, or the price of an irrigated crop falls, managed aquifer recharge may no longer be worthwhile. We can investigate how much of a change would cause this crossover to occur. If we look at both uncertainties at once, even small changes in infrastructure cost and crop price may cause a crossover – managed aquifer recharge is even less viable. These vulnerabilities become scenarios we can discuss within our investment planning process (for a description of scenarios, see Bonnie McBain’s blog post). (Info-gap theory as described in Yakov Ben-Haim’s blog post does something similar to vulnerability analysis.)

At this stage, we have not needed to know why the infrastructure cost would rise or crop price would fall. Both remain unknown unknown black boxes. But we can add information if we have it: we might be able to get a fixed price contract for the infrastructure, and we might be able to use price forecasts to evaluate how worried we should be about that vulnerability. And importantly, we can do this while only partially opening the black box, by identifying the vulnerabilities introduced by our new information. A fixed price contract can be associated with a black box probability that the contractor will not finish the job. Our price forecast can be accompanied by an error or a probability distribution with unknown unknown drivers, used for example to maximise expected utility (for a description of expected utility see the blog post by Siobhan Bourke and Emily Lancsar).

Using vulnerability analysis to work backwards from outcomes provides a powerful way of working with unknown unknowns, gradually identifying known unknowns as we come across them, while making the best use of what we consider known knowns.

What has your experience been with vulnerability analyses? Have you seen them used to blackbox unknown unknowns in practice?

To find out more:

Arshad, M., Guillaume, JHA. and Ross, A. (2014). Assessing the Feasibility of Managed Aquifer Recharge for Irrigation under Uncertainty. Water, 6 (9): 2748–69. (Online) (DOI): http://dx.doi.org/10.3390/w6092748

Guillaume JHA, Arshad M, Jakeman AJ, Jalava M, Kummu M (2016) Robust Discrimination between Uncertain Management Alternatives by Iterative Reflection on Crossover Point Scenarios: Principles, Design and Implementations. Environmental Modelling & Software, 83: 326–43. (Online) (DOI): http://dx.doi.org/10.1016/j.envsoft.2016.04.005

Biography: Joseph Guillaume PhD is a DECRA (Discovery Early Career Researcher Award) Research Fellow in the Fenner School of Environment & Society at the Australian National University in Canberra, Australia. He is an integrated modeller with a particular interest in uncertainty and decision support. Application areas have focussed primarily on water resources. Ongoing work involves providing a synthesis of the many ways we communicate about uncertainty, and their implications for modelling and decision support.

This blog post is part of a series on unknown unknowns as part of a collaboration between the Australian National University and Defence Science and Technology.

For the six other blog posts already published in this series, see: https://i2insights.org/tag/partner-defence-science-and-technology/

Scheduled blog posts in this series:
December 3: Yin-yang thinking – A solution to dealing with unknown unknowns? by Christiane Prange and Alicia Hennig
January 14, 2020: Detecting non-linear change ‘inside-the-system’ and ‘out-of-the-blue’ by Susan van ‘t Klooster and Marjolijn Haasnoot
January 28, 2020: How can resilience benefit from planning? by Pedro Ferreira
February 11, 2020: Why do we protect ourselves from unknown unknowns? by Bem Le Hunte

Managing innovation dilemmas: Info-gap theory

By Yakov Ben-Haim

Author - Yakov Ben-Haim
Yakov Ben-Haim (biography)

To use or not to use a new and promising but unfamiliar and hence uncertain innovation? That is the dilemma facing policy makers, engineers, social planners, entrepreneurs, physicians, parents, teachers, and just about everybody in their daily lives. There are new drugs, new energy sources, new foods, new manufacturing technologies, new toys, new pedagogical methods, new weapon systems, new home appliances and many other discoveries and inventions.

Furthermore, the innovation dilemma occurs even when a new technology is not actually involved. The dilemma arises from new attitudes, like individual responsibility for the global environment, or new social conceptions, like global allegiance and self-identity transcending all nation-states. Even the enthusiastic belief in innovation itself as the source of all that is good and worthy entails a dilemma of innovation. Continue reading

What do you know? And how is it relevant to unknown unknowns?

By Matthew Welsh

Author - Matthew Welsh
Matthew Welsh (biography)

How can we distinguish between knowledge and ignorance and our meta-knowledge of these – that is, whether we are aware that we know or don’t know any particular thing? The common answer is the 2×2 trope of: known knowns; unknown knowns; known unknowns; and unknown unknowns.

For those interested in helping people navigate a complex world, unknown unknowns are perhaps the trickiest of these to explain – partly because the moment you think of an example, the previously “unknown unknown” morphs into a “known unknown”.

My interest here is to demonstrate that this 2×2 division of knowledge and ignorance is far less crisp than we often assume.

This is because knowledge is not something that exists in the world but rather in individual minds. That is, whether something is ‘known’ depends not on whether someone, somewhere, knows it; but on whether this person, here-and-now does. Continue reading

Four patterns of thought for effective group decisions

Community member post by George P. Richardson and David F. Andersen

George Richardson
George P. Richardson (biography)

What can you do if you are in a group that is trying to deal with problems that are developing over time, where:

  • root causes of the dynamics aren’t clear;
  • different stakeholders have different perceptions;
  • past solutions haven’t worked;
  • solutions must take into account how the system will respond; and,
  • implementing change will require aligning powerful stakeholders around policies that they agree have the highest likelihood of long-term success?

Continue reading

Designing scenarios to guide robust decisions

Community member post by Bonnie McBain

Bonnie McBain (biography)

What makes scenarios useful to decision makers in effectively planning for the future? Here I discuss three aspects of scenarios:

  • goals;
  • design; and,
  • use and defensibility.

Goals of scenarios

Since predicting the future is not possible, it’s important to know that scenarios are not predictions. Instead, scenarios stimulate thinking and conversations about possible futures. Continue reading

Trust and empowerment inventory for community groups

Community member post by Craig Dalton

Author - Craig Dalton
Craig Dalton (biography)

Community groups are often consulted by researchers, government agencies and industry. The issues may be contentious and the relationship vexed by distrust and poor communication. Could an inventory capture the fundamental sources of community frustration and highlight scope for improvement in respect, transparency, fairness, co-learning, and meeting effectiveness from a community perspective?

The trust and empowerment inventory presented below is based on the main sources of community frustration that I have witnessed over two decades as a public health physician and researcher liaising with communities about environmental health risks and it is likely to have broader relevance. Key issues include not being listened to; not being fully informed; Continue reading