By Sarah Young, Principal Librarian at Carnegie Mellon University
The production and publishing of systematic reviews and meta-analyses has grown significantly in recent years. A title search for “systematic review” in Web of Science reveals a more than 10-fold increase in systematic reviews since 2010, with almost 40,000 in Web of Science indices in 2022 alone.
Policy- and decision-makers often turn to systematic reviews as “the best available evidence”, but the rigour and quality of many reviews is questionable. Numerous studies have shown that review quality is lacking in many fields.
Organisations like the Campbell Collaboration have worked to raise the bar for systematic reviews: offering guidelines, providing reporting and conduct standards, and including methods experts in the peer review and editorial processes.
Another component of improving the quality and rigour of published reviews is training and education. Review authors are usually deep experts in their fields, but may have little experience or knowledge of how to conduct a systematic review that minimises bias, and how to report their methods in a way that maximises transparency.
Training on systematic review methods is not easily available to research teams, especially social scientists. Little exists outside the fields of medical and health research, until now.
Campbell’s Training Group has recently published the first online course on systematic reviews and meta-analysis with a focus on social science disciplines (click here to read our news article). ‘Systematic reviews and meta-analysis: a Campbell Collaboration online course’ includes units on every step of the systematic review process from problem formulation to meta-analysis. It is offered through the Open Learning Initiative (OLI), based at Carnegie Mellon University and designed around evidence-based learning science principles.
As learners work through the course content, they are provided opportunities to test their knowledge and receive meaningful and immediate feedback.
In designing this course, we had in mind the many diverse ways that people learn and interact with educational material, and the different contexts in which researchers may be conducting or learning about systematic review methods.
How people might use the Campbell systematic reviews and meta-analysis online course:
As a fully self-paced asynchronous course, working through the content from beginning to end, in preparation for conducting a systematic review or in the context of professional development.- As a ready reference, diving into units and modules as they encounter steps in their systematic review project. This is recommended for researchers who already have some training and knowledge of systematic review methods, but need a quick refresher.
- As a course textbook for instructors and trainers in conjunction with synchronous training. The course can supplement in-class content and enhance student learning through flipped classroom techniques and by helping to identify and address pain points through the formative assessments built into the course.
The course is currently available for free to independent learners, and instructors wishing to use it as a textbook can work with OLI to set up a course instance, with integration into their learning management system. Feedback from users will be used to inform further development and iteration.
This course forms the basis of a consistent training approach for Campbell, and we hope that it will build capacity for rigorous systematic reviewing, as well as improve the evidence base for decision-makers globally.