Ten dialogue methods for integrating judgments

By David McDonald, Gabriele Bammer and Peter Deane

authors_david-mcdonald_gabriele-bammer_peter-deane
1. David McDonald (biography)
2. Gabriele Bammer (biography)
3. Peter Deane (biography)

What formal dialogue methods can assist researchers in synthesising judgments about a complex societal or environmental issue when a range of parties with different perspectives are involved? How can researchers decide which methods will be most suitable for their purposes?

We review ten dialogue methods. Our purpose is not to describe the dialogue methods in detail, but instead to review the circumstances in which each method is likely to be most useful in a research context, bearing in mind that most methods a) were not developed for research, b) can be applied flexibly and c) have evolved into different variations.The methods are clustered into six groups:

  • Structured (or semi-structured) process for a cross-section of society with a stake in the outcome, covering
    • Future search conference
    • Nominal group technique
    • Soft systems methodology
  • Self-organised process for self-selected participants, covering
    • Open space technology
  • Expert consensus, covering
    • Consensus development panel
    • Delphi technique
  • Informed citizen consensus, covering
    • Citizens’ jury
    • Consensus conference
  • Alternative visions for the future, covering
    • Scenario planning
  • Story-based evaluation focused on valued outcomes, covering
    • Most significant change technique.

Before highlighting the key aspects of each method, some definitions are in order.

What do we mean by dialogue method, integration and judgment?

We use Franco’s (2006: p. 814) definition of dialogue, which is to “jointly create meaning and shared understanding” through conversation. A dialogue method is a structured conversation that seeks to ensure that all participants can express their views, as well as listening respectfully to others, and that moves beyond this exchange to joint meaning and shared understanding.

Dialogue is therefore a method for integration, for weaving together different insights into a composite whole. Our interest here is in integrating judgments.

We take a perspective offered by Yankelovich (1999) who pointed out that, in making a judgment, a person takes into account the facts as they understand them, their personal goals and moral values, and their sense of what is best for others as well as themselves.

Judgments are critical in dealing with complex societal and environmental problems, especially when there are substantial unknowns.

Choosing a dialogue method

We present the methods ordered by the six groups described above, highlighting the circumstances where each method is most useful in providing integrated judgments in a research context.

Future search conference is useful when:

  • diverse stakeholders can be expected to make different judgments about the implications of future change.
  • the aim is to include a significant cross-section of all parties with a stake in the outcomes (usually ranging from 60 to hundreds of participants).
  • seeking to develop common ground and a shared vision for the future, leading to action plans for implementation after the conference.

Nominal group technique is useful when:

  • a decision has to be made.
  • the participants are a limited group with a degree of common knowledge and background.
  • giving everyone an equal say and using voting to reach the decision is a key consideration.

Soft systems methodology is useful when:

  • a systems-based action plan is required.
  • the participants hold differing views about the nature and origins of the problem, how it can be addressed and what goals are to be worked towards.
  • the primary aim is to accommodate different, and possibly conflicting, world views.

Open space technology is useful when:

  • the nature of the problem is reasonably clear, but the way forward is unclear.
  • the aim is for people who are passionate about the topic to self-select and self-organise, expecting that an agreed way forward and any actions will be emergent properties of the process.
  • the participants, who can number between a handful to thousands, will record the outcomes of the discussions and proposed actions.

Consensus development panel is useful when:

  • a consensus statement from experts is required about an area in which there is both controversy and a body of strong scientific evidence.
  • around 15 panelists with general expertise and high standing are available to develop the consensus statement; they should not be experts in the particular subject matter and should have no financial or career advancement investment in it.
  • subject matter experts, advocates and others are available to present evidence to the panel.

Delphi technique is useful when:

  • an anonymous process is required (while there is no face-to-face conversation, there is a structured exchange of views and the creation of joint meaning and understanding).
  • a task-oriented process is required with a focus on content, influenced as little as possible by relationships, power differences and the biases that can occur in face-to-face meetings.
  • the problem can be tightly defined, even though it is multifaceted with areas of uncertainty.

Citizens’ jury is useful when:

  • the views are sought of a well-informed, representative group of 18-24 ordinary citizens on a complex issue.
  • subject-matter experts, advocates (on all sides) and other stakeholders chosen by the researchers are available to provide information to the jury.
  • funding is available to pay the jury to receive evidence and deliberate for 4-5 days.

Consensus conference is useful when:

  • the views are sought of a well-informed, representative group of 12-25 ordinary citizens on a matter of current interest, where advocates hold conflicting positions.
  • written evidence can be provided to the participants, who can then call expert witnesses.
  • the decision makers are willing to be involved in the process and intend to implement the findings (with reasons given if this does not occur).

Scenario planning is useful when:

  • planning for an uncertain future is required using the ability to compare different potential futures.
  • different combinations of trends, facts and assumptions can be taken into account to develop different alternative futures.
  • there is a process for using the scenarios as a basis for strategic decision making.

Most significant change technique is useful when:

  • insights are required into monitoring and evaluating complex interventions, with a focus on valued outcomes.
  • stories of both positive achievements and lessons learnt can be generated, focused on values.
  • stories are generated by those closest to the intervention, and used as a tool to communicate across all levels of the organization undertaking the intervention.

Concluding questions

What has your experience been using the dialogue methods described here in a research context? Can you suggest other ways these methods can be used in research? Are there other dialogue methods for integrating judgments that researchers might find useful?

To find out more:
McDonald, D., Bammer, G. and Deane, P. (2009). Research Integration Using Dialogue Methods. ANU Press, Canberra, Australia. (Online – open access): DOI: http://doi.org/10.22459/RIUDM.08.2009
As well as describing each method, this book provides case studies illustrating how each method can be used for integration in research, and provides references.

References:

Franco, L. A. (2006). Forms of conversation and problem structuring methods: A conceptual development. Journal of the Operational Research Society, 57: 813–21.

Yankelovich, D. (1999). The Magic of dialogue. Transforming conflict into cooperation. Simon and Schuster, New York, United States of America.

Biography: David McDonald is a Campus Visitor at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra. He is also the Director of the consultancy Social Research & Evaluation Pty Ltd. He is an interdisciplinary social scientist with research interests at the intersection of criminal justice and population health. He uses research integration and implementation insights to assist with building evidence-informed public policy, particularly in the alcohol and other drugs field.

Biography: Gabriele Bammer PhD is Professor of Integration and Implementation Sciences (i2S) at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra. i2S provides theory and methods for tackling complex societal and environmental problems, especially for synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns, and providing integrated research support for policy and practice change.

Biography: Peter Deane is a Research Officer on the Integration and Implementation Sciences (i2S) team at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra.

7 thoughts on “Ten dialogue methods for integrating judgments”

  1. Thanks Rick for the links. This is really interesting. Have you got any papers describing your approach and how it relates to other foresight methods?
    I personally am also interested in behavioral phenomena related to modeling. Today, we do not have many studies on the impact of the on-line format on the outcomes when compared to face-to-face sessions. Path dependence is one phenomenon which can play a role in foresight processes. For a blog on this see https://i2insights.org/2017/03/28/path-dependence-in-modelling/. So we and the younger researchers have rich area of research to be studied.

    Reply
  2. Hi David, Gabriele and Peter

    Re your question and scenario planning, I invite you to explore the uses of ParEvo.org: A free web-assisted process enabling the participatory exploration of alternative futures, which I developed, using the same underlying epistemology as Most Significant Change (MSC),

    You can find a list of past exercises, and planned exercises for the next few months, here: https://mscinnovations.wordpress.com/list-of-parevo-exercises/

    Plus videos of recent webinars about its use here
    2022: CEDIL Webinar: The collaborative exploration of alternative futures: A different approach to Theories of Change https://youtu.be/Hlbpqfw_ve4
    2021: Scanning the Horizon Methods Workshop: Par Evo, video of webinar for the International Civil Society Centre, Germany https://www.youtube.com/watch?v=CPVyFbjKNX8

    with bets wishes, Rick Davies, Cambridge, UK

    Reply
  3. Hi,
    This is an interesting summary of dialogue approaches. One aspect, which is seldom discussed, is the need for practicing. If the group starts to work on the real problem directly the benefits of dialogue can be lost as the participants are likely to stick to their old routines in group settings. In our approach called the Decision Structuring Dialogue we explicitly encourage to practice dialogue in a general context first before starting to work with the real issue at hand.
    Another aspect of current relevance is the running of dialogue processes on-line. What are the challenges and possibilities? In the beginning of this century electronic democracy was emerging as an interesting topic. Some 20 years we had an early European project with interesting perspectives raised already then.

    Refs:
    Decision Structuring Dialogue:
    https://www.sciencedirect.com/science/article/abs/pii/S2193943821000443
    or
    https://research.aalto.fi/files/31379608/SCI_Decision_Structuring_Dialogue_Hamalainen_Slotte_07_05_14.doc

    Towards E-Democracy project
    http://archives.esf.org/fileadmin/Public_documents/Publications/Towards_Electronic_Democracy_TED_.pdf
    https://link.springer.com/book/10.1007/978-90-481-9045-4
    Workshop program and slides
    http://ted.tkk.fi/program.html

    Reply
    • Your comment raises important issues, thanks! While it is useful to know about established methods – which is what we set out to provide – knowing about them is not enough to successfully implement them, as you highlight. Indeed there are numerous considerations that affect how well a dialogue method works. It is good to start with one that is fit-for-purpose, which our work aims to help with, but then the nature and experience of the group, the context, and the facilitation, all contribute to how effective the dialogue actually is. You work is an important dimension of this. Others may also have important insights to contribute. Some are already covered by i2Insights eg https://i2insights.org/tag/dialogue/ and https://i2insights.org/tag/facilitation/.

      You comment about e-dialogue is also timely. It would be useful to hear from others about where this is up to.

      Reply
    • Hi Raimo.

      Re Another aspect of current relevance is the running of dialogue processes on-line. What are the challenges and possibilities?
      As mentioned above, parevo.org is an online process. In follow-up surveys we have found high levels of reported enjoyment by participants in the process,.

      On other futures methods implemented online, i did come across this paper a while ago
      Raford N (2015) Online foresight platforms: Evidence for their impact on scenario planning & strategic foresight. Technological Forecasting and Social Change 97: 65–76. DOI: 10.1016/j.techfore.2014.03.008.
      and this one
      Hew A, Perrons RK, Washington S, et al. (2018) Thinking together about the future when you are not together: The effectiveness of using developed scenarios among geographically distributed groups. Technological Forecasting and Social Change 133: 206–219. DOI: 10.1016/j.techfore.2018.04.005.

      Reply

Leave a Reply to David McDonald Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: