Bringing an evidence mindset to commercial products

By Natalia I. Kucirkova

Picture this: science seamlessly woven into commercial products for a better world. How can we foster more effective integration of science into commercial products? How can we help companies cultivate an evidence mindset?

An evidence mindset is a habit of mind committed to thoughtful engagement with research to guide understanding and actions. As a research professor leading a network of scientists supporting educational companies in integrating learning sciences into children’s products, I aim to help such companies balance their social mission with financial viability. This includes nurturing an evidence mindset.

When our researchers collaborate with companies, we assist them in establishing a research library, guiding them to actual studies instead of relying on vague phrases. We include research studies that support their ideas but also studies that contradict them. Acknowledging the existence of counter-evidence is an essential part of an evidence mindset, especially the understanding that transparent reporting of both positive and negative findings increases users’ trust in the rigour of the company’s commitment to evidence.

We support companies in developing individualized impact metrics for their products. We engage in discussions with the companies about the desired outcomes in the short and long term, how these can be measured using data, and how the outcomes contribute to the overall impact the companies have on diverse stakeholders. At workshops with the companies’ product teams and the companies’ research teams, we encourage the company representatives to adopt an evidence mindset, to ask questions throughout the process and to validate claims with evidence. To facilitate this approach, we have developed a set of questions, which we have refined and validated at workshops and strategic meetings with a range of educational organisations.

The evidence mindset questions that we developed tap into five areas of evidence of impact (the “5Es”): Effectiveness, Efficacy, Ethics, Equity and Environment (Kucirkova, 2023). All five aspects, including their combination, such as, for example, the joint impact of pursuing efficacy and environmental goals, are important. Here are ten example questions relevant to commercial products:

Effectiveness:

• How do we judge the quality of the evidence/studies that inform or support our product?
• How does science/research contradict our approach?

Efficacy:

• How are we encouraging transparency among our teams to openly share our efficacy outcomes and/or impact metrics?
• How are we leveraging multi-stakeholder engagements to bolster community engagement and foster long-term collaborations?

Ethics:

• How do we plan to address potential unintended consequences arising from the use of innovations in our products?
• How do we ensure that any research/data collection we conduct to inform the design and development of our solution adheres to ethical and data protection best practice?

Equity:

• How are we aligning our efforts to advance equity in the world, ensuring that our solution and approach reach and benefit all, especially those in underserved areas?
• How can we integrate artificial intelligence and any other innovative technologies in our product while safeguarding ethical standards related to data usage, user privacy, fairness and equity?

Environment:

• How are we integrating climate and environmental considerations into our approaches to contribute to global climate change mitigation?
• How are we leveraging our resources and influence to support both immediate and long-term solutions to social and environmental injustices, especially for communities most affected by adversity and climate change?

The questions are presented as a deck of cards and are intended to serve as inspiration for individual reflection or team discussions. Our aim is to foster a comprehensive understanding of evidence so that commercial products positively impact the world. This involves recognizing that people may perceive a problem differently based on their context or that achieving a positive impact requires a thorough understanding of diverse needs and accommodating a range of experiences.

These questions may be useful in cultivating an evidence mindset not only for educational, but also other companies. Do these questions look like they would be useful in your interactions with commercial companies? Are there other questions that you have used or that you think are needed? Do you think such questions would also be useful in researcher interactions with philanthropic organizations, non-government organisations, and policymakers who play a significant role in evidence-based decision-making?

Reference:

Kucirkova, N. (2023). How can philanthropy catalyse a system-wide change in EdTech? Alliance for Philanthropy and Social Investment Worldwide, ‘Latest from Alliance’ blog, 15 March 2023. (Online): https://www.alliancemagazine.org/blog/how-can-philanthropy-catalyse-a-system-wide-change-in-edtech/

Biography: Natalia I. Kucirkova PhD is Professor of Reading and Children’s Development at The Open University, in the UK and Professor of Early Childhood and Development at the University of Stavanger in Norway. She also leads the social enterprise WiKIT, co-funded by the University of Stavanger. She conducts foundational and empirical research on Educational Technology (EdTech) and has developed apps and worked directly with EdTech companies, non-government organisations and investors to support the integration of learning sciences into practice.