Unknowns – Page 3 – Integration and Implementation Insights

What do you know? And how is it relevant to unknown unknowns?

August 11, 2023September 24, 2019

By Matthew Welsh

Author - Matthew Welsh — Matthew Welsh (biography)

How can we distinguish between knowledge and ignorance and our meta-knowledge of these – that is, whether we are aware that we know or don’t know any particular thing? The common answer is the 2×2 trope of: known knowns; unknown knowns; known unknowns; and unknown unknowns.

For those interested in helping people navigate a complex world, unknown unknowns are perhaps the trickiest of these to explain – partly because the moment you think of an example, the previously “unknown unknown” morphs into a “known unknown”.

My interest here is to demonstrate that this 2×2 division of knowledge and ignorance is far less crisp than we often assume.

This is because knowledge is not something that exists in the world but rather in individual minds. That is, whether something is ‘known’ depends not on whether someone, somewhere, knows it; but on whether this person, here-and-now does.

How can we know unknown unknowns?

December 13, 2021September 10, 2019

By Michael Smithson

In a 1993 paper, philosopher Ann Kerwin elaborated a view on ignorance that has been summarized in a 2×2 table describing crucial components of metacognition (see figure below). One margin of the table consisted of “knowns” and “unknowns”. The other margin comprised the adjectives “known” and “unknown”. Crosstabulating these produced “known knowns”, “known unknowns”, “unknown knowns”, and unknown unknowns”. The latter two categories have caused some befuddlement. What does it mean to not know what is known, or to not know what is unknown? And how can we convert either of these into their known counterparts?

Accountability and adapting to surprises

August 11, 2023August 27, 2019

By Patricia Hirl Longstaff

Image of Patricia Hirl Longstaff — Patricia Hirl Longstaff (biography)

We have all been there: something bad happens and somebody (maybe an innocent somebody) has their career ruined in order to prove that the problem has been fixed. When is blame appropriate? When is the blame game not only the wrong response, but damaging for long-term decision making?

In a complex and adapting world, errors and failure are not avoidable. The challenges decision-makers and organizations face are sometimes predictable but sometimes brand new. Adapting to surprises requires more flexibility, fewer unbreakable rules, more improvisation and deductive tinkering, and a lot more information about what’s going right and going wrong. But getting there is not easy because this challenges some very closely held assumptions about how the world works and our desire to control things.

Let’s not kid ourselves. Sometimes people do really dumb things that they should be blamed for. What we need is to be more discriminating about when finding blame and accountability is appropriate.

Designing scenarios to guide robust decisions

April 4, 2023July 2, 2019

By Bonnie McBain

What makes scenarios useful to decision makers in effectively planning for the future? Here I discuss three aspects of scenarios:

goals;
design; and,
use and defensibility.

Goals of scenarios

Since predicting the future is not possible, it’s important to know that scenarios are not predictions. Instead, scenarios stimulate thinking and conversations about possible futures.

Key goals and purposes of scenarios can be any of the following:

Managing uncertainty in decision making: What can we learn from economics?

August 11, 2023June 4, 2019

By Siobhan Bourke and Emily Lancsar

authors_siobhan-bourke_emily-lancsar — 1. Siobhan Bourke (biography)
2. Emily Lancsar (biography)

How can researchers interested in complex societal and environmental problems best understand and deal with uncertainty, which is an inherent part of the world in which we live? Accidents happen, governments change, technological innovation occurs making some products and services obsolete, markets boom and inevitably go bust. How can uncertainty be managed when all possible outcomes of an action or decision cannot be known? In particular, are there lessons from the discipline of economics which have broader applicability?

While uncertainty is often discussed alongside risk, a fundamental difference between uncertainty and risk is that risk involves events with known probabilities (or probabilities based on reliable empirical evidence), whereas under uncertainty probabilities are unknown and reflect an individual’s subjective belief concerning the likelihood of a given outcome. Given the subjectivity, that likelihood can differ from person to person. It can also involve a perceived zero probability in the case of unforeseen events (or ‘unknown unknowns’).

What every interdisciplinarian should know about p values

April 15, 2022April 30, 2019

By Alice Richardson

In interdisciplinary research it’s common for at least some data to be analysed using statistical techniques. Have you been taught to look for ‘p < 0.05’ meaning that there is a less than 5% probability that the finding occurred by chance? Do you look askance at your statistician colleagues when they tell you it’s not so simple? Here’s why you need to believe them.

The whole focus on p < 0.05 to the exclusion of all else is a historical hiccup, based on a throwaway line in a manual for research workers. That manual was produced by none other than R.A. Fisher, giant of statistical inference and inventor of statistical methods ranging from the randomised block design to the analysis of variance. But all he said was that “[p = 0.05] is convenient to take … as a limit in judging whether a deviation is to be considered significant or not.” Convenient, nothing more!

Knowledge asymmetry in interdisciplinary collaborations and how to reduce it

January 3, 2022February 26, 2019

By Max Kemman

How can tasks and goals among partners in a collaboration be effectively negotiated, especially when one party is dependent on the deliverables of another party? How does knowledge asymmetry affect such negotiations? What is knowledge asymmetry anyway and how can it be dealt with?

What is knowledge asymmetry?

My PhD research involves historians who are dependent on computational experts to develop an algorithm or user interface for historical research. They therefore needed to be aware of what the computational experts were doing.

Managing deep uncertainty: Exploratory modeling, adaptive plans and joint sense making

April 15, 2022August 15, 2017

By Jan Kwakkel

How can decision making on complex systems come to grips with irreducible, or deep, uncertainty? Such uncertainty has three sources:

Intrinsic limits to predictability in complex systems.
A variety of stakeholders with different perspectives on what the system is and what problem needs to be solved.
Complex systems are generally subject to dynamic change, and can never be completely understood.

Deep uncertainty means that the various parties to a decision do not know or cannot agree on how the system works, how likely various possible future states of the world are, and how important the various outcomes of interest are.

Sharing integrated modelling practices – Part 2: How to use “patterns”?

October 9, 2023June 1, 2017

By Sondoss Elsawah and Joseph Guillaume

authors_sondoss-elsawah_joseph-guillaume — 1. Sondoss Elsawah (biography)
2. Joseph Guillaume (biography)

In part 1 of our blog posts on why use patterns, we argued for making unstated, tacit knowledge about integrated modelling practices explicit by identifying patterns, which link solutions to specific problems and their context. We emphasised the importance of differentiating the underlying concept of a pattern and a pattern artefact – the specific form in which the pattern is explicitly described.

In order to actually use patterns to communicate about practices, the artefact takes on greater importance: what form could artefacts describing the patterns take, and what mechanisms and platforms are needed to first create, and then share, maintain, and update these artefacts?

While the concepts of ‘problem, solution and context’ should be discussed in some way, there is no single best way of representing patterns as artefacts. The form of artefacts will differ depending on many factors, including how the users perceive the ease of:

Sharing integrated modelling practices – Part 1: Why use “patterns”?

October 9, 2023May 30, 2017

By Sondoss Elsawah and Joseph Guillaume

How can modellers share the tacit knowledge that accumulates over years of practice?

In this blog post we introduce the concept of patterns and make the case for why patterns are a good candidate for transmitting the ‘know-how’ knowledge about modelling practices. We address the question of how to use patterns in a second blog post.

In broad terms, a pattern links a solution to a problem and its context. As a means of externalizing understanding of practices, the concept has been used productively in various fields, including architecture, computer science, and design science. For a more general introduction to patterns, see Scott Peckham’s blog post. While a “pattern” is ultimately a simple idea, there tends to be disagreement about a precise definition. This poses a problem for this blog post.

Argument-based tools to account for uncertainty in policy analysis and decision support

April 15, 2022February 23, 2017

By Sven Ove Hansson and Gertrude Hirsch Hadorn

authors_sven-ove-hansson_gertrude-hirsch-hadorn — 1. Sven Ove Hansson (biography)
2. Gertrude Hirsch Hadorn (biography)

Scientific uncertainty creates problems in many fields of public policy. Often, it is not possible to satisfy the high demands on the information input for standard methods of policy analysis such as risk analysis or cost-benefit analysis. For instance, this seems to be the case for long-term projections of regional trends in extreme weather and their impacts.

However, we cannot wait until science knows the probabilities and expected values for each of the policy options. Decision-makers often have good reason to act although such information is missing. Uncertainty does not diminish the need for policy advice to help them determine which option it would be best to go for.

Dealing with deep uncertainty: Scenarios

October 25, 2022January 5, 2017

By Laura Schmitt Olabisi

What is deep uncertainty? And how can scenarios help deal with it?

Deep uncertainty refers to ‘unknown unknowns’, which simulation models are fundamentally unsuited to address. Any model is a representation of a system, based on what we know about that system. We can’t model something that nobody knows about—so the capabilities of any model (even a participatory model) are bounded by our collective knowledge.

One of the ways we handle unknown unknowns is by using scenarios. Scenarios are stories about the future, meant to guide our decision-making in the present.