Richard Horton, the editor of The Lancet, the pre-eminent medical journal that was stung by one of the worst science frauds of the last decade (Andrew Wakefield’s phony vaccine-autism link paper), has a fascinating note reporting on the conversations at a recent conference of scientists in the UK about the problems of scientific review. A few of his statements are genuinely eye-popping:
“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted . . . this symposium—on the reproducibility and reliability of biomedical research, held at the Wellcome Trust in London last week—touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations.
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. . .
The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of “significance” pollutes the literature with many a statistical fairy-tale. We reject important confirmations. Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication. . . nobody is ready to take the first step to clean up the system. [Emphasis added.]
Half of the scientific literature may be untrue? Half! Cue Eddie Murphy.
Pretty clearly there’s a theme emerging here: the academic-scientific complex is as corrupt as a FIFA World Cup meeting. This is ought to be regarded as a crisis, but as Horton comments, “nobody is ready to take the first step to clean up the system.” (In fact, this sounds a lot like FIFA.)
Oh, and “endemicity”? No, I don’t think I want that to catch on. Lamosity was enough of a stretch.