By Vera Schölmerich, assistant professor in behavioral sciences and public health, Erasmus University Rotterdam
During the What Works Global Summit in London last September, I was excited to meet program officers from non-profits using rigorous evidence to improve their programs. At the same time, I am always surprised by how this evidence-oriented approach isn’t mainstream – yet. In this blog post I argue that a root cause of this is how research methods and statistics are taught at universities, and I have a plan to fix it.
I often hear non-profit staff feeling discouraged after consulting evidence from scientific sources – “it’s all so complicated anyway, and then also one study says that, another study says something else… I am not sure what to believe”. As a consequence, many give up engaging with existing evidence and instead rely on their intuition.
To effectively make use of existing evidence and distinguish between good and bad evidence, you need to have a basic grasp of research methods, which includes having the capacity to assess the quality of how studies were set up, and statistics. At least on an intuitive level, you need to understand concepts such as significant versus substantial effects, standard deviation, correlation versus causation, or the importance of randomization.
A university education – regardless of what program – should provide students with a basic grasp of research methods and statistics. But from my own time as a student and based on my experience by working at several different universities in Europe and the US, I think that universities are largely failing at this. I see students donating to dubious charities who use pre-post tests to claim that they have made a smashing impact. I see students drinking carrot juice and telling me that correlational studies proved that it was good for your health, not realizing that they should be cautious of drawing causational conclusions from such study designs. I see students listening to classical music while studying as they read that this would “significantly” improve their grades, not realizing the difference between significant and substantial.
While I find it frustrating to see students waste their time and money, I have a deeper concern here. Our current students will become our future doctors, lawyers, policy-makers, and also non-profit officers deciding on how to spend millions of dollars. These are people who will have a profound impact on our everyday lives and the state of our world, and they are not equipped with the basic tools to separate fact from fiction. Take president-elect Donald Trump’s tweet about autism, where he confuses correlation with causation:
Universities are churning out professionals who aren’t properly equipped to distinguish fact from fiction because they teach research methodology and statistics in a boring, technical way – or not even at all, as is the case for many law school students, for example. Most students are scared of these courses and don’t see how these skills could help them outside of academia. Scientists have been more successful at studying this phenomenon of ‘statistics anxiety’, but the problem has yet to be resolved.
My co-lecturer Kellie Liket and I have tried to come up with a way to both convey basic concepts in research methods and statistics and to show their relevance for life outside of academia. We just launched a free online course called ‘Deception Detox – how solid science can help you save the world’. The course is particularly relevant for university students and for professionals who need to engage with scientific evidence.
In this course we try to teach research methods and statistics in a non-technical, easily accessible way that shows why these skills are relevant. Unlike other online and offline courses about this topic, we don’t immediately dive into the boring and technical bits, such as memorizing what a p-value stands for. Instead, we first explore global challenges – such as poverty or climate change – and then discuss key approaches of statistics and research methods as they naturally arise.
By failing to convey basic literacy in research methods and statistics to students, universities are producing professionals unable to use rigorous science to solve real-life problems. This needs to change. In the words of astrophysicist Neil deGrasse Tyson: "Science literacy is a vaccine against the charlatans of the world that would exploit your ignorance."