By Vivian Welch, Ruth Garside, Santina LeBrun, Sarah Miller, Darren Moore, Gavin Stewart, James Thomas
When we travel to work or take our kids to school, the “bricks and mortar” infrastructure of trains, railways and crossings allows us to get there efficiently and safely. It takes many years of planning the public transit infrastructure to ensure safe intersections, clear signage, appropriate speed limits and signals that allow us to get where we need to go safely and efficiently. And, when our rail lines break down, we need teams to fix the lines quickly and safely and provide alternate routes to avoid delays.
When we carry out evidence synthesis, the bricks and mortar to help us produce efficient, timely and high quality answers to important questions consists of people first, and secondly the methods and tools to support them. We need people trained in evidence synthesis who work in partnership with decision-makers to ask and answer relevant questions. And, we need tools to support them such as tools with in-built access to comprehensive datasets, automation, and artificial intelligence so that people can focus on the framing and interpretation where human skills and knowledge are essential. And, last but not least, we need methods that are fit for purpose and provide a framework for comprehensive, transparent, rigorous work. The Campbell Collaboration is actively involved in building all three components of the infrastructure 2.0 for evidence synthesis.
Glasziou and Chalmers estimated in 2009 that 85% of research is wasted due to poor methods or inadequate transparency and this situation has not improved much. An unfortunate case in point is that the landscape of evidence synthesis is varied in quality. In some Campbell evidence and gap maps, less than 10% of systematic reviews are rated high quality. In terms of relevance to decision-makers, relatively few reviews consider policy and practice important outcomes such as health inequalities and few include perspectives of people experiencing conditions and disadvantages associated with social inequalities.
The Campbell Collaboration has been actively involved in advancing training, supporting methods innovation and increasing access to tools for high quality, meaningful evidence synthesis ever since its inaugural meeting in Philadelphia in 2001. Some of the landmark contributions by Campbell methodologists include calculating the power of statistical tests in meta-analysis, evidence and gap map methods, and methods for qualitative evidence synthesis. More recently, Campbell published updated guidance for searching, and modernized Campbell Standards to improve transparency in reporting and adherence to accepted methods. Campbell also developed a set of free, online training materials in collaboration with Carnegie-Mellon University. In order to promote access to tools, Campbell provides secretariat support to the International Collaboration on Automation in Systematic Reviews (ICASR).
In the next year, Campbell has prioritized three areas for advancing methods: 1) tools to critically appraise non-randomized studies, 2) tools to advance policy relevance through considering certainty of evidence, context and equity considerations in user-friendly summaries, and 3) guidance and tools for responsible use of AI in collaboration with Cochrane, JBI, ICASR (International Collaboration for the Automation of Systematic Reviews), the EPPI- Centre and many other organizations.
The Campbell Collaboration community encourages and supports the development and use of innovative methods in systematic reviews and evidence maps, recognizing them as Campbell innovations. This development of methods within a systematic review are a way to co-design methods with end-user and content experts. As an example, a Campbell review of mobile apps to reduce depressive symptoms and alcohol use in youth convened youth to rank the importance of outcomes and choose the primary outcomes for analysis.
Imagine your usual rail route to work or daycare has been affected by construction to fix the line. To your delight, the city planners have set up detour signs to direct you to a replacement bus service to reach your destination. Infrastructure is not static; it needs to be maintained, and be modified to adjust to conditions. Campbell encourages ongoing methodological innovation and develops pathways for these methods to be implemented, such as training and guidance. In this way, we are living up to the call by Donald Campbell, the namesake of the Campbell Collaboration, to be an “experimenting society”. We invite you to join us in building the evidence synthesis infrastructure 2.0!