By Gabriele Bammer
How do different perceptions arise of what makes for ‘good’ research? How can researchers come to understand such differences and their impacts on how problems are framed, understood and responded to, as well as how they affect the ability of those contributing to the research to work together?
Differences arise because training in a discipline involves inculcating a specific way of investigating the world, including which types of questions are worth addressing; legitimate ways of gathering, analysing and interpreting data; standards for validation; and the role of values in the research process. Educating someone in a discipline aims to make the discipline’s specific approach to research ingrained and tacit.
While this makes for relatively smooth operation of discipline-based research, it is the source of hidden stumbling blocks when investigators from multiple disciplines seek to work together, leading to what Hubbs and colleagues (2021) have called the “problem of unacknowledged differences.” In research groups comprising members from different disciplines, these unacknowledged differences can come into play when deciding how the problem will be addressed, as well as in how research group members perceive the value of what colleagues from other disciplines have to offer.
The Toolbox Dialogue Initiative has drawn on the philosophy of science to provide a practical way for research groups to develop an understanding of the different perceptions of their members. This involves group members filling out a self-completion survey, where they rate ― from strongly agree to strongly disagree ― five or six (generally) statements about research. The statements serve as prompts that disclose key features of each researcher’s assumptions about what makes research good. The responses are not scored, but instead are used to stimulate dialogue at a facilitated workshop that brings these differences into the light.
In order to understand differences in perceptions of good research, the survey focuses on six areas:
- motivation
- methodology
- confirmation
- objectivity
- values
- reductionism–emergence.
Motivation
Differences in motivation concern the perceived relative importance of basic and applied research.
Examples of prompt statements:
- Knowledge generated by research is valuable even if it has no application
- The principal value of research stems from the potential application of the knowledge gained.
Methodology
Differences in methodology concern research goals, such as whether research should be hypothesis driven and whether the aim is to describe or predict. Differences also include approaches to data acquisition, for example, whether it is experimental or observational, and whether quantitative or qualitative data are seen as more useful.
Examples of prompt statements:
- Scientific research (applied or basic) must be hypothesis driven
- In my disciplinary research, I employ primarily qualitative methods.
Confirmation
Differences relevant to confirmation concern what processes are required for the research results to add to the storehouse of knowledge. Different disciplines put different emphases on, for example:
- measurement validity ie., determining whether what was measured was what was intended
- replication ie., the original team and other researchers being able to repeat the findings
- triangulation ie., getting the same results using unrelated methods
- generalisability ie., finding the same results in other settings and/or with other groups of research subjects.
Examples of prompt statements:
- There are strict requirements for determining when empirical data confirm a tested hypothesis
- Unreplicated results can be validated if confirmed by a combination of several different methods.
Objectivity
Differences in objectivity concern “whether a fully objective world exists independent of any perspective, or whether the world is to some extent constructed by those who investigate it” (Eigenbrode et al., p. 59).
Examples of prompt statements:
- Scientific research aims to identify facts about a world independent of the investigators
- The subject of my research is a human construction.
Values
Differences here concern whether research can be value-neutral or whether research is “infused with values” (Eigenbrode at al., p. 59). Proponents of the second view argue that values can affect the topics chosen, how results are implemented, whose perspectives are taken into account in the research and much more.
Examples of prompt statements:
- Determining what constitutes acceptable validation of research data is a value issue
- Allowing values to influence scientific research is advocacy.
Reductionism–emergence
Differences here concern whether the world can be fully understood by reducing it to its component parts, which are then studied (reductionism) or whether a systems view is required that focuses on context, interconnections among components and emergent properties (ie., new properties or behaviours displayed by an entity that its parts do not have).
Examples of prompt statements:
- The world under investigation is fully explicable as the assembly of its constituent parts
- The world under investigation must be explained in terms of the emergent properties arising from the interactions of its individual components.
Differences in perceptions of good research and how problems are framed, understood and responded to
How problems are framed is affected by different perceptions of what makes for good research; consider, for example, a reductionist approach that examines a component of the problem and focuses on finding a cause, versus an approach that looks for interconnections and emergence. Similarly, whether research is seen as legitimate will influence its contribution to understanding the problem and valid responses to the problem. For example, if qualitative research is seen as flawed, its findings may be rejected, and if potential responses are thought to result from research that is biased by values, they may be ignored.
Differences in perceptions of good research and how well those contributing to the research work together
One determinant of how well those contributing work together is whether they can come to an understanding and agreement about their different perceptions of what makes for good research. This is influenced both by the uptake of their own contributions, as well as what they think of the contributions of others.
The ability to work together will be enhanced when everyone feels their own contributions are understood, valued and used. This may involve agreeing to disagree and to incorporate different perspectives of what is considered to be good research.
Concluding notes
The topics and prompts discussed here are taken from the Scientific Research Tool Instrument which was developed early in the life of the Toolbox Dialogue Initiative and which has been used in around one-third of the dialogues organised and facilitated under the auspices of the Initiative. Nowadays the Toolbox Dialogue Initiative works with teams to tailor the topics and prompts to their needs, which can go beyond what makes research good. The process also goes beyond understanding differences to finding ways to accommodate them.
Although the focus here is on researchers, stakeholders can also be involved as they also often have specific views on what constitutes good research.
Anything to add?
Do you have additional perspectives to share about perceptions of what makes for good research, as well as how you learnt about different perceptions? What was the impact of these perceptions on approaching problems, as well as on the ability to work together?
Examples and lessons are particularly welcome. Is there anything you wish you had known when you were starting out?
If you are new to this topic, is there anything else on understanding different perspectives about good research that would be useful?
Sources and references:
The main sources are from the Toolbox Dialogue Initiative (http://tdi.msu.edu), and especially the references listed below, which also cite the work of others who influenced this approach.
Eigenbrode, S. D., O’Rourke, M., Wulfhorst, J. D., Althoff, D. M., Goldberg, C. S., Merrill, K., Morse, W., Nielsen-Pincus, M., Stephens, J., Winowiecki, L. and Bosque-Pérez, N. A. (2007). Employing Philosophical Dialogue in Collaborative Science. BioScience, 57, 1: 55-64. (Online – Open access): https://academic.oup.com/bioscience/article/57/1/55/224519
Hubbs, G., O’Rourke, M. and Orzack, S. H. (Eds.). (2020). The Toolbox Dialogue Initiative: The Power of Cross-Disciplinary Practice. CRC Press: Florida, United States of America. (Online – book description): https://www.taylorfrancis.com/books/toolbox-dialogue-initiative-graham-hubbs-michael-rourke-steven-hecht-orzack/e/10.1201/9780429440014
Hubbs, G., O’Rourke, M. and Orzack, S. H. (2021). Responding to Unacknowledged Disciplinary Differences with the Toolbox Dialogue Method. Integration and Implementation Insights, March. (Online): https://i2insights.org/2021/03/02/toolbox-dialogue-method/
Biography: Gabriele Bammer PhD is Professor of Integration and Implementation Sciences (i2S) at the National Centre for Epidemiology and Population Health at The Australian National University in Canberra. i2S provides theory and methods for tackling complex societal and environmental problems, especially for synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns, and providing integrated research support for policy and practice change. She is also a member of blog partner PopulationHealthXchange.
The Understanding Diversity Primer comprises the following blog posts:
Published:
1. Why diversity? (April 21, 2022)
2. Mental models (April 28, 2022)
This blog post:
3. Perceptions of good research (May 5, 2022)
Still to come:
4. Power (May 12, 2022)
5. Values (May 19, 2022)
6. Interests (May 26, 2022)
7. Culture (June 2, 2022)
8. Personality (June 9, 2022)
9. Team roles (June 16, 2022)
10. Advanced considerations (June 23, 2022)