Ten dialogue methods for integrating judgments

By David McDonald, Gabriele Bammer and Peter Deane

What formal dialogue methods can assist researchers in synthesising judgments about a complex societal or environmental issue when a range of parties with different perspectives are involved? How can researchers decide which methods will be most suitable for their purposes?

We review ten dialogue methods. Our purpose is not to describe the dialogue methods in detail, but instead to review the circumstances in which each method is likely to be most useful in a research context, bearing in mind that most methods a) were not developed for research, b) can be applied flexibly and c) have evolved into different variations. The methods are clustered into six groups:

• Structured (or semi-structured) process for a cross-section of society with a stake in the outcome, covering
  • Future search conference
  • Nominal group technique
  • Soft systems methodology
• Self-organised process for self-selected participants, covering
  • Open space technology
• Expert consensus, covering
  • Consensus development panel
  • Delphi technique
• Informed citizen consensus, covering
  • Citizens’ jury
  • Consensus conference
• Alternative visions for the future, covering
  • Scenario planning
• Story-based evaluation focused on valued outcomes, covering
  • Most significant change technique.

Before highlighting the key aspects of each method, some definitions are in order.

What do we mean by dialogue method, integration and judgment?

We use Franco’s (2006: p. 814) definition of dialogue, which is to “jointly create meaning and shared understanding” through conversation. A dialogue method is a structured conversation that seeks to ensure that all participants can express their views, as well as listening respectfully to others, and that moves beyond this exchange to joint meaning and shared understanding.

Dialogue is therefore a method for integration, for weaving together different insights into a composite whole. Our interest here is in integrating judgments.

We take a perspective offered by Yankelovich (1999) who pointed out that, in making a judgment, a person takes into account the facts as they understand them, their personal goals and moral values, and their sense of what is best for others as well as themselves.

Judgments are critical in dealing with complex societal and environmental problems, especially when there are substantial unknowns.

Choosing a dialogue method

We present the methods ordered by the six groups described above, highlighting the circumstances where each method is most useful in providing integrated judgments in a research context.

Future search conference is useful when:

• diverse stakeholders can be expected to make different judgments about the implications of future change.
• the aim is to include a significant cross-section of all parties with a stake in the outcomes (usually ranging from 60 to hundreds of participants).
• seeking to develop common ground and a shared vision for the future, leading to action plans for implementation after the conference.

Nominal group technique is useful when:

• a decision has to be made.
• the participants are a limited group with a degree of common knowledge and background.
• giving everyone an equal say and using voting to reach the decision is a key consideration.

Soft systems methodology is useful when:

• a systems-based action plan is required.
• the participants hold differing views about the nature and origins of the problem, how it can be addressed and what goals are to be worked towards.
• the primary aim is to accommodate different, and possibly conflicting, world views.

Open space technology is useful when:

• the nature of the problem is reasonably clear, but the way forward is unclear.
• the aim is for people who are passionate about the topic to self-select and self-organise, expecting that an agreed way forward and any actions will be emergent properties of the process.
• the participants, who can number between a handful to thousands, will record the outcomes of the discussions and proposed actions.

Consensus development panel is useful when:

• a consensus statement from experts is required about an area in which there is both controversy and a body of strong scientific evidence.
• around 15 panelists with general expertise and high standing are available to develop the consensus statement; they should not be experts in the particular subject matter and should have no financial or career advancement investment in it.
• subject matter experts, advocates and others are available to present evidence to the panel.

Delphi technique is useful when:

• an anonymous process is required (while there is no face-to-face conversation, there is a structured exchange of views and the creation of joint meaning and understanding).
• a task-oriented process is required with a focus on content, influenced as little as possible by relationships, power differences and the biases that can occur in face-to-face meetings.
• the problem can be tightly defined, even though it is multifaceted with areas of uncertainty.

Citizens’ jury is useful when:

• the views are sought of a well-informed, representative group of 18-24 ordinary citizens on a complex issue.
• subject-matter experts, advocates (on all sides) and other stakeholders chosen by the researchers are available to provide information to the jury.
• funding is available to pay the jury to receive evidence and deliberate for 4-5 days.

Consensus conference is useful when:

• the views are sought of a well-informed, representative group of 12-25 ordinary citizens on a matter of current interest, where advocates hold conflicting positions.
• written evidence can be provided to the participants, who can then call expert witnesses.
• the decision makers are willing to be involved in the process and intend to implement the findings (with reasons given if this does not occur).

Scenario planning is useful when:

• planning for an uncertain future is required using the ability to compare different potential futures.
• different combinations of trends, facts and assumptions can be taken into account to develop different alternative futures.
• there is a process for using the scenarios as a basis for strategic decision making.

Most significant change technique is useful when:

• insights are required into monitoring and evaluating complex interventions, with a focus on valued outcomes.
• stories of both positive achievements and lessons learnt can be generated, focused on values.
• stories are generated by those closest to the intervention, and used as a tool to communicate across all levels of the organization undertaking the intervention.

Concluding questions

What has your experience been using the dialogue methods described here in a research context? Can you suggest other ways these methods can be used in research? Are there other dialogue methods for integrating judgments that researchers might find useful?

To find out more:
McDonald, D., Bammer, G. and Deane, P. (2009). Research Integration Using Dialogue Methods. ANU Press, Canberra, Australia. (Online – open access): DOI: http://doi.org/10.22459/RIUDM.08.2009
As well as describing each method, this book provides case studies illustrating how each method can be used for integration in research, and provides references.

References:

Franco, L. A. (2006). Forms of conversation and problem structuring methods: A conceptual development. Journal of the Operational Research Society, 57: 813–21.

Yankelovich, D. (1999). The Magic of dialogue. Transforming conflict into cooperation. Simon and Schuster, New York, United States of America.

Biography: David McDonald is a Campus Visitor at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra. He is also the Director of the consultancy Social Research & Evaluation Pty Ltd. He is an interdisciplinary social scientist with research interests at the intersection of criminal justice and population health. He uses research integration and implementation insights to assist with building evidence-informed public policy, particularly in the alcohol and other drugs field.

Biography: Gabriele Bammer PhD is Professor of Integration and Implementation Sciences (i2S) at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra. i2S provides theory and methods for tackling complex societal and environmental problems, especially for synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns, and providing integrated research support for policy and practice change.

Biography: Peter Deane is a Research Officer on the Integration and Implementation Sciences (i2S) team at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra.