The scandalous underinvestment in the global evidence architecture

By Howard White 

This article is the third of three in a series "Reflections on the evidence architecture".

Today a child will sit in a classroom but not learn anything for want of the use of research evidence. Today a child will not go to school for want of the use of such evidence. Today a child will go hungry, be abused and even die, for want of the use of evidence.

It is nearly 15 years since the Center for Global Development lamented the waste of billions of dollars each year in When Will We Ever Learn?. Whilst there are more impact evaluations now than there were then, the fact is that most programmes go unevaluated, and most evaluations are not used.

This matters. The 80% rule tells us that 80% of things don't work, that is they have little or no effects. This means that billions of dollars are being wasted on ineffective interventions. Devoting just 1% of that wasted money to building the evidence architecture would have a truly transformational effect on the lives of millions of people.

I spend a lot of time promoting the evidence architecture to support the institutionalisation of the use of evidence. This institutionalisation is most firmly embedded in the upper layers of the knowledge brokering pyramid with evidence portals, evidence-based guidelines and evidence-based checklists. These evidence products inform decision-making without the decision-maker having to consult the underlying research.

This approach is most advanced for evidence-based health. The international the World Health Organization (WHO) produces evidence-based guidelines on a range of topics such as immunization, nutrition and community-based health interventions. Many countries around the world look to the WHO guidelines to inform their own national policies. So influencing WHO guidelines can have much wider policy impact.

With funding from the Bill and Melinda Gates Foundation, the International Initiative for Impact Evaluation (3ie) supported a series of studies on self-testing for HIV with a view to informing the planned WHO guidelines on the subject. The Campbell review on deworming informed the revision of the WHO guidelines on the topic last year.

Of course, other organisations produce guidelines. But the extent to which they are evidence-based varies. WHO's 'Guidelines on Producing Guidelines' state that WHO guidelines have to be based on high quality systematic reviews. And indeed, it may not be possible to make such evidence-based products at the top of the knowledge pyramid for other sectors such as agriculture and governance, as the lower levels are weak. You can’t build the top of a pyramid without a base!

Compared to health, there has been a massive underinvestment in research on what works. A global review of education RCTs, randomised controlled trials, found 1,017 studies. The 3ie database contains just under 5,000 impact evaluations for all sectors. But the Cochrane Registry of Clinical Trials, which is work in progress, contains over 1.5 million studies.

It is the same story for systematic reviews. A search of the Institute of Education Science's research database, ERIC, suggests there are around 1,000 education systematic reviews. The 3ie database contains just under 700 reviews of the effectiveness of development interventions. In contrast, Epistemonikos contains over 35,000 health reviews.

These numbers illustrate the scandalous underinvestment in the evidence architecture by multilateral agencies – both the UN system and the multilateral banks such as the World Bank – the global funds, and other agencies funding development research. It is not that money is not being spent on development research: it is. But it is not being spent in a systematic and strategic way to inform effective development. There is an urgent need to repurpose existing research, evaluation and knowledge translation funds toward building the evidence architecture, sector by sector, to mirror that in health.

Only with such a concerted effort can be put an end to failed programmes which disrupt people's live, bring them false hopes and sometimes actually harm them.

So join the evidence revolution today. Devote research funds and research time to studies which build the body of evidence to improve and save lives. Decisionmakers, media and the public: demand evidence.

To contribute to the global debate on these issues join us at the What Works Global Summit 2019 in Mexico City this October.

Contact us