Providing a richer assessment of research influence and impact

By Gabriele Bammer

How can we affirm, value and capitalise on the unique strengths that each individual brings to interdisciplinary and transdisciplinary research? In particular, how can we capture diversity across individuals, as well as the richness and distinctness of each individual’s influence and impact?

In the course of writing ten reflective narratives (nine single-authored and one co-authored), eleven of us stumbled on a technique that we think could have broader utility in assessing influence and impact, especially in research but also in education (Bammer et al., 2019).We wrote the reflective narratives for a special issue on Julie Thompson Klein’s academic work, in our case especially her impact in Australia and New Zealand. (A taste of Klein’s work can be found in her contributions to this i2Insights blog.)

The process of writing the reflective narratives was unstructured, with each of us writing as much or as little as we wished on whatever aspect of Klein’s work had resonated with us. Contributions ranged from a paragraph to a few pages. In addition to conventional influences, especially through her seminal book (Klein, 1990), our combined reflections illustrated the following:

• the breadth of Klein’s work that influenced us was noteworthy, with only a few overlaps among the ideas and publications we each drew on
• six of the reflections described fundamental influences on thinking and orientation that were not easily citeable in our own publications, including finding ‘permission’ in Klein’s work to do things differently
• two of the reflections showed the practical value of her ideas in making sense of and/or shaping events and circumstances in inter- and trans-disciplinarity in our own institutions
• two of the reflections illustrated catalytic effects when Klein’s ideas were combined with the ideas of others in particular educational and research settings.

These points illustrate that by each of us focusing as we saw fit, we uncovered dimensions of influence that might otherwise have remained hidden. In addition, we were pleasantly surprised by the greater whole that emerged from the sum of our individual written parts, especially as some of us were not sure we had anything worthwhile to contribute when agreeing to participate. The result reminded us of assembling shards of glazed pottery into a mosaic.

We suggest that there is merit in exploring how reflective narratives could be used more widely in assessing research impact. Three issues that require further consideration are:

1. selecting contributors
2. synthesis of the shards into a mosaic
3. highlighting unique strengths.

Selecting contributors

If the person whose influence is being assessed belongs to a field with a large, active professional association, drawing on members for contributions seems like an obvious step. In Klein’s case, this was not possible as the main professional association – the Association for Interdisciplinary Studies – is small and has no members in Australia and New Zealand. In our case, the original invited contributor (GB) found the additional ten contributors a) by seeking volunteers from attendees at a conference and a workshop where Klein’s work was featured and b) through personal networks.

One of the interesting features of our group is that only four of us knew Klein personally. In addition, we covered a range of ages and career levels, from early career researchers through to professors and contributors who are retired and emeriti. Nevertheless we were not a representative sample.

There will always be challenges in drawing a representative sample from whatever population is chosen. But perhaps seeking a sample that is representative is less important than ensuring that there are enough pieces to assemble a mosaic, recognising that such a mosaic is not only a product of the individuals involved, but also of the moment of writing. The same person writing at a different time, in a different context, may well contribute different shards to the mosaic.

Synthesis of the shards into a mosaic

We found that just producing the shards of reflective narratives was not enough. They have to be assembled into a mosaic that tells a story of the individual’s influence.

How best to achieve such synthesis needs further thought. In our case the driver was an academic publication and several of the group provided valuable insights that guided the synthesis, including the metaphor of the mosaic and links to literature that helped emphasise aspects of Klein’s work.

Highlighting unique strengths

What most appealed to us about this process was its ability to highlight an individual’s distinctive strengths and contributions. In Klein’s case these included the meticulousness and depth of her scholarship, as well as her generosity towards others in the field.

More generally, each of us brings a different set of knowledge, skills and personal qualities to our academic work, which are not captured by metrics such as H factors and the number of publications in high impact factor journals.

For some it is deep knowledge about particular methods, for others the focus is on concepts, still others can extrapolate from myriad cases, and so on. Some are qualitatively skilled, others are outstanding wordsmiths, others are expert at project design and more. Some are skilled in nurturing up-and-coming talent, others in working with senior leaders; some are good at starting projects, others at finishing them; some bring creative thinking, others attention to detail; and the list could go on.

Metrics tend to focus on targets to reach or exceed, rather than affirming, valuing and capitalising on the wide range of individual differences that exist. And it is differences that are critical for good inter- and trans- disciplinary research. It is combining differences that makes for richer understandings of problems and that yields new, creative insights for tackling them.

What do you think? What other benefits and limitations can you see from employing reflective narratives? How do you think this could work in an institutional setting?

To find out more:
Bammer, G., Mitchell, C., Elford, W., Dumaresq, D., van Kerkhoff, L., Small, B., Kaufman Hall, V., Kaufman, S., Browne, C., Brown, V. A. and Blessington, L. (2019). A Rich Mosaic of Impact: Julie Thompson Klein’s Scholarly Influence in Australia and New Zealand. Issues in Interdisciplinary Studies, 37, 2: 144-169. Available online on Researchgate or on request.
Addition made in October 2022: Also available in open access at: https://interdisciplinarystudies.org/docs/Vol37_2019_No2/10_Vol.37_No.2_pp_143-168_Bammer%20et%20al.pdf (PDF 300KB)

Reference:
Klein, J. T. (1990). Interdisciplinarity: History, theory, and practice. Wayne State University Press: Detroit, United States of America.

Biography: Gabriele Bammer PhD is a professor at The Australian National University in the Research School of Population Health’s National Centre for Epidemiology and Population Health. She is developing the new discipline of Integration and Implementation Sciences (i2S) to improve research strengths for tackling complex real-world problems through synthesis of disciplinary and stakeholder knowledge, understanding and managing diverse unknowns, and providing integrated research support for policy and practice change.

Gabriele Bammer is a member of blog partner PopulationHealthXchange, which is in the Research School of Population Health at The Australian National University.