About that “Johns Hopkins study”

A working paper by a Johns Hopkins economics professor and two other economists found that covid “lockdowns” did very little, if anything, to reduce the number of covid deaths. The paper has received a fair amount of attention from conservative media, but has been largely ignored by mainstream outlets.

The mainstream media’s lack of attention to the study is unfortunate. But then, conservative outlets generally ignore covid studies that find positive effects from lockdowns, masking, and the like.

Go figure.

Although the paper by the three economists deserves attention, I’m not persuaded by its finding that lockdowns have had very little, if any, impact on covid deaths. I do agree, however, that the damage caused by lockdowns probably outweighed their positive on health, even in the pre-vaccine period.

Here are my observations about the economists’ paper.

First, the paper isn’t really a study in the normal sense. Rather, it’s a study of studies. It takes more than 18,000 studies of the impact of lockdowns on covid deaths and selects 24 it deems methodologically acceptable. It derives its conclusions from these 24, most of which apparently are studies by other economists, not epidemiologists.

Had the authors ruled out fewer studies, or different ones, they likely would have reached a different conclusion. For example, the study of studies discussed here, which was almost as ruthless in excluding studies, appears to have reached a different conclusion about the effectiveness of lockdowns.

The three economists were transparent about their exclusion criteria, but I lack the expertise to evaluate their decisions in this regard. Samir Bhatt, professor of statistics and public health at the Imperial College London, questions the exclusion of synthetic control method papers, some of which have found a significant impact by lockdowns on mortality.

A critique of the paper’s inclusions and exclusions can be found here. The critic, epidemiologist Gideon Meyerowitz-Katz, finds that “the authors exclude many of the most rigorous studies, including those that are the entire basis for their meta-analysis in the first place.”

The working paper by the three economists has been criticized because it isn’t peer reviewed (not yet, anyway) and is the work of economists not scientists. Most covid research at Johns Hopkins comes from its Coronavirus Resource Center which is run out of the university’s medical school. But the Center wasn’t involved in producing the paper. I understand that John Hopkins didn’t even issue a press release when the paper came out.

Whatever else might be true of the paper, it should not be referred to as the Johns Hopkins study. The authors have basically acknowledged as much.

The lead author of the paper, Steve Hanke, is a senior fellow at the libertarian Cato Institute. He has consistently opposed lockdowns. He hates them so much that, reportedly, he recently posted a cartoon in which Justin Trudeau, dressed up as Adolph Hitler, is run over by a truck.

These criticisms shouldn’t be considered fatal. A paper can reach correct conclusions even if it hasn’t been peer reviewed and even if it doesn’t have a seal of approval from the Johns Hopkins medical program. Economists can reach correct conclusions about epidemiological questions based on an analysis of studies. An author’s strong predisposition against lockdowns doesn’t necessarily mean his analysis of them isn’t fair minded.

But given the degree to which the paper’s conclusion is driven by which studies the authors included, and given the exclusion of nearly 99.9 percent of all studies initially considered, I would have much more confidence in its conclusion if the paper had been peer reviewed, was the work of at least one epidemiologist, had some connection to Johns Hopkins’ Coronavirus Resource Center, and wasn’t spearheaded by a libertarian who hates lockdowns.

Other than the inclusion/exclusion criteria, what are the substantive objections to the paper, though? One revolves around what it counts as a lockdown. The paper defines a lockdown as one of more mandatory non-pharmaceutical interventions (NPIs). This includes wearing a mask. (In one of the tables, though, the authors apparently revert to the more common definition of lockdown.)

Different studies have reached different conclusions about whether masking is an effective anti-covid measure. However, there should be no controversy about whether a mask mandate is a lockdown. It isn’t.

The definitional problem doesn’t end there. As I understand the paper’s definition, a country whose only anti-covid measure was a five-day mandatory quarantine is treated the same way as a country with curfews and blanket closures of public venues. This doesn’t make much sense.

There are other objections, as well. One is that some countries locked down before seeing exponential growth in spread and therefore saw no reduction in deaths. In addition, mortality lags behind the spread of covid, so that one is unlikely to find much reduction in death in a comparison of mortality a month before and a month after a lockdown. The study has also been criticized for looking at too small a slice of the pandemic.

When studies conflict, along even with notions of what a good study looks like, it isn’t amiss to apply common sense to the question at hand. Common sense tells me that limiting person-to-person contact will limit transmission of (and therefore deaths from) a communicable disease.

Experience tells me the same thing. Until 2020, there had never been a calendar year in which I didn’t catch at least one cold or flu. Yet, I haven’t been sick in two years. Many friends tell me basically the same thing.

Surely, the change isn’t due to good luck. It must be due to a substantial reduction in the amount of contact with other people, especially in places like restaurants and bars. (I’m not happy with the tradeoff, but there it is.)

How could reducing the amount of close human contact not reduce the likelihood of catching a communicable disease? I understand the argument that nearly all of us will come down with covid eventually. It might even be true.

But the three economists didn’t analyze “eventually.” They analyzed a slice of “so far.”

And even if we’re all going to catch covid eventually, there are major advantages to getting it later rather than sooner. It’s better to catch it after being vaccinated or after it has mutated to a less lethal variant. This reduces the chances of dying or becoming seriously ill.

Similarly, it’s better to get covid after the ability to treat it has improved, whether due to new medications or more hospital capacity. And if “eventually” turns out to be five years, high risk populations will benefit from the passage of time itself. Some members will live significantly longer thanks to lockdowns.

In sum, and to be clear, I don’t reject the idea that, on balance lockdowns are a bad idea. But I’m not persuaded that they contribute little or nothing when it comes to preventing covid deaths.

It’s fine to hate lockdowns, but highly dubious to deny that, like nearly all policies, they have pluses and minuses and therefore entail tradeoffs.