1.2 A Crisis

Much of the scientific community was shocked in 2005 when researcher, John Ioannidis, wrote a paper claiming that most published research findings are false.

Ioannidis was criticizing the rigour of the scientific process used by researchers in medicine. His statement sparked a movement that started questioning whether or not published studies could be replicated: that is, could they withstand a test of their burden of proof?

Since Ioannidis published his paper, this movement has grown, seeking to transform the culture and practices of scientific research. As a society and as a species we face many challenges that require us to work together across geographical, social, political, and disciplinary boundaries. Solid science and public confidence in that science figure prominently in efforts to address such issues as health and wellness, climate change, and the need to make resource use sustainable.

A change in culture and practices to improve knowledge sharing, quality, accessibility, and trust in science is needed.

Why Most Published Research Findings are False

After reading the following abstract from John Ioannidis’s article, Why most published research findings are false, reflect on the following questions.

• What factors did Ioannidis identify for the lack of reliability in research studies?
• How might these problems be addressed?

Abstract

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field.

In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.