It was barely a week ago that I was recalled the original Sokal Hoax and noting its recent sequel out of Eastern Europe. You’d think humanities journals would be on to this kind of stunt, but surely science journals wouldn’t fall for this. Would they?
The Economist reports this week that a complete hoax has indeed happened in scientific publishing. But not just in one journal; rather, the potential hoax passed muster at potentially dozens of supposedly reputable and peer-reviewed science journals:
John Bohannon, a biologist at Harvard with a side gig as a science journalist, wrote his own Sokalesque paper describing how a chemical extracted from lichen apparently slowed the growth of cancer cells. He then submitted the study, under a made-up name from a fictitious academic institution, to 304 peer-reviewed journals around the world.
Despite bursting with clangers in experimental design, analysis and interpretation of results, the study passed muster at 157 of them. Only 98 rejected it. (The remaining 49 had either not responded or had not reviewed the paper by the time Science went to press.) Just 36 came back with comments implying that they had cottoned on to the paper’s sundry deficiencies, though Dr Bohannon says that 16 of those eventually accepted it anyway.
I’m working on a theorem right now, and hoping perhaps eventually to attach some real numbers to back it up, that holds that if we abolished half of all academic journals—it wouldn’t matter which half—not only would no one notice, but that there would be a net improvement in the quality of academic thought. Maybe I’ll just make it all up and get it published in the Journal of Irreproducible Results.