The Campbell Collaboration
Home Home News Mail Print

Random Notes: A New Year's Resolution

If a review is worth doing, it is worth doing well. This was the message I took away from the Dhaka Colloquium on Systematic Reviews in December, having listened in particular to the plenary presentation of Julia Littell. She pointed out that the absence of any of a number of key features of a systematic review mitigates the accuracy of the review's findings.

Reviews which limit the literature search to published studies risk misrepresenting the evidence, given that studies which report significant results are published faster and cited more often. Reviews which do not practice duplicate data extraction - where two people do this independently of one another - run the risk of distorting the evidence, because single data extraction leads to more inaccuracies and omissions. Choosing narrative synthesis over meta-analysis is also found to give less accurate results.

In each case, there is a tendency to present conclusions that are more positive than the evidence warrants. Traditionally, it has been hard to explain to funders that the complexity of the review process is necessary, and that a high-quality review involves both time and resources. This made it all the more heartening to hear in Dhaka that policymakers are increasingly aware of this problem. Representatives of the UK's development agency stated that DFID had decided not to compromise on quality. The challenge now is to ensure that more genuinely systematic reviews are conducted and completed. How about that for a New Year's Resolution?

Eamonn Noonan, CEO