Evaluation – Page 3 – Integration and Implementation Insights

Should I trust that model?

September 15, 2023October 6, 2016

By Val Snow

How do those building and using models decide whether a model should be trusted? While my thinking has evolved through modelling to predict the impacts of land use on losses of nutrients to the environment – such models are central to land use policy development – this under-discussed question applies to any model.

In principle, model development is a straightforward series of steps:

• Specification: what will be included in the model is determined conceptually and/or quantitatively by peers, experts and/or stakeholders and the underlying equations are decided

• Coding: the concepts and equations are translated into computer code and the code is tested using appropriate software development processes

What makes a translational ecologist? Part 3: Dispositional attributes

December 31, 2021July 4, 2016

By the Translational Ecology Group

This is the third and final blog post considering competencies to make ecologists more effective in informing and supporting policy and practice change (see the right sidebar for links to all four related blog posts on translational ecology). In other words these are the competencies underpinning a new discipline of translational ecology.

The two previous blog posts examined the knowledge and skills required in three major areas:

Socio-ecological systems
Communication across boundaries, with beneficiaries, stakeholders and other scientists
Engagement with beneficiaries, stakeholders and other scientists.

This blog post uses the same three areas to examine the dispositional attributes required.

What are dispositional attributes?

Each person’s internal cognitive and moral qualities are collectively known as dispositions.

Choosing a model: If all you have is a hammer…

September 11, 2023June 2, 2016

By Jen Badham

As a modeller, I often get requests from research or policy colleagues along the lines of ‘we want a model of the health system’. It’s relatively easy to recognise that ‘health system’ is too vague and needs explicit discussion about the specific issue to be modelled. It is much less obvious that the term ‘model’ also needs to be refined. In practice, different modelling methods are more or less appropriate for different questions. So how is the modelling method chosen?

Sometimes the colleague will actually name a technique, for example ‘we want an agent-based model of an intervention to encourage physical activity’. However, this choice may not be informed. It might reflect something that he or she saw in a paper, at a conference, or whatever some previous modeller used. On the other hand, the colleague may have sufficient knowledge to have consciously selected an appropriate method.

From the other direction, many modellers work with only system dynamics or agent-based modelling or some other specific technique.

Can co-creation achieve better outcomes for people and communities?

August 24, 2022May 5, 2016

By Deborah Ghate

The language of ‘co-processes’ is much in vogue in policy, practice and academic communities worldwide. In commerce, product design and politics, the power of the crowd has long been recognised, but can co-processes be harnessed for the public good? The answer, right now, appears to be ‘maybe’.

What are co-processes and what are they for?

The briefest survey of the literature on co-processes confirms there is substantial variation in how they are defined and what methods or techniques they include. A confusing multiplicity of related terms exists—co-construction, co-production, co-design, co-innovation, co-creation—all are in regular use, sometimes interchangeably, and often defined at an unhelpful level of abstraction (for more on this topic see the blog post by Allison Metz on Co-creation, co-design, co-production, co-construction: same or different?). Nevertheless, however we define co-processes, participatory methods, boundary-spanning and inclusivity to varying degrees are foundational principles that can be detected in most accounts. Beyond that, the stated purposes and proposed outcomes vary considerably.

Towards an evaluation framework for participatory modeling

August 24, 2022March 31, 2016

By Miles McNall

What are the results of participatory modeling efforts? What contextual factors, resources and processes contribute to these results? Answering such questions requires the systematic and ongoing evaluation of processes, outputs and outcomes. At present participatory modeling lacks a framework to guide such evaluation efforts. In this post I offer some initial thoughts on the features of this framework.

A first step in developing an evaluation framework for participatory modeling is to establish criteria for processes, outputs, and outcomes. Such criteria would answer a basic question about what it means when we say that a participatory modeling process, output, or outcome is good, worthy, or meritorious.