COVID19: Let’s Talk About the Bakersfield Duo
by Raywat Deonandan, PhD
Epidemiologist & Associate Professor
University of Ottawa
(I add my credentials to these COVID-19 blog posts in case they get shared. I want readers to know that my opinion is supposedly an educated and informed one)
When I made the decision to serve as a public resource during this COVID-19 pandemic, I made a parallel decision not to express expertise in any area in which I did not have advanced training. That’s a hard thing for a supposed “intellectual” to do, as self-anointed eggheads like me always think we have something to offer to almost any conversation. (We’re insufferable that way.) But it was an important requirement, since in this time of extreme emergency, it is from genuine experts that the public needs to hear.
Despite many media requests to do so, I’ve been careful not to comment too much about economics or clinical medicine, for example —topics in which I have a fair amount of education, but not doctoral-level expertise. And so I feel it’s equally important that I respond to public expressions of epidemiology, as they pertain to COVID-19, that are rich with error, even though their progenitors have fancy sounding titles and degrees, but no actual training in epidemiology or biostatistics.
The Bakersfield Duo
That brings me to a certain viral video that many people have sent to me. It was recorded by two physicians (one MD and one DO), Dan Erickson and Artin Massihi, who run a private medical clinic called Accelerated Urgent Care in Bakersfield, California. If you want, you can watch the video here. It’s gotten a lot of traction, and I’m told the two will be on national Fox News later this evening. (Update: the original video has been removed, but here’s another link.)
In essence, they argue that the death rate and prevalence rate of COVID-19 are comparable to that of the seasonal flu. Moreover, it is already so prevalent in the community that contact with it should not be feared. They are essentially suggesting, without saying it outright, that due to undiagnosed cases at large, (the American) population is pretty close to herd immunity.
Drs Erickson and Massihi begin their video by extolling their expertise, specifically their “extensive classes in microbiology, biochemistry and immunology.” Glaringly missing were classes in epidemiology, statistics, and public health. But that will become clear further on, gentle reader. Don’t you worry.
To be honest, I got through the first 10 minutes and had to stop. It took true effort for me to restart the video. The number of glaring errors and data misinterpretations were so many that I almost wept. My physician spouse left the room after a mere three minutes, as the duo’s inability to understand basic population health concepts –much of which is middle school level math– was frankly upsetting.
So let’s go through some of the computational sins expressed by Drs Erickson and Massihi.
First, they do say some good things. Absolutely, it is concerning that people needing non-COVID medical care are kept away from clinics by fear. This has been one of the failures of the health system management approach, and of the public health communication deployment, across many jurisdictions. This level of social distancing and economic depression is really concerning, and I share their fears that it might hurt many people for a long time to come.
Second, it’s also true that many models predicted millions of deaths in California that did not manifest. A big reason for that is that mitigation and suppression efforts worked. It’s also because the initial CFR (case fatality rate, which you can read about here) was elevated, based on the atrocious Spain and Italy experiences. As the CFR went down, the expected deaths also declined. But it was always a mistake to consider these models to be forecasts; they were always worst case scenarios. For these two to claim that that they were “inaccurate” misses the points of such models, which immediately made me suspicious of their public health training.
Third, they start off their press conference by stating authoritatively that we “typically quarantine the sick. We’ve never seen when we quarantine the healthy.” Well, that’s not true. Historically, hiding in your house and keeping away from strangers was the first best method to avoid plague. It is literally an ancient technique. In some ways, it’s embarrassing that we’re still using it in 2020, as we should have better technological tools at our disposal. But when it comes to reducing the base reproduction number, which is our best measure of epidemic spread, nothing works as well as hand-washing and distancing, the latter of which is best achieved by staying in your home, away from others.
Okay, but those are small sins. Let’s get into the really fun stuff.
The Test-Positive Rate is NOT the Prevalence Rate
Dr Erickson states that Kern county (where they are located) has conducted 5,213 tests which revealed 340 cases. I will take his numbers as truth without looking them up. He then states –wait for it– “Well that’s 6.5% of the population.”
This is when, the first time I watched the video, I turned it off and cursed to myself for a while. I might have even shed a tear of two as I wept for the failure of modern public education. It took some strength to come back and watch the rest of it. I do this for you people. Thank me!
This is important: If you divide 340 positives by 5213 tests you do not get “6.5% of the population.” You get a test-positive rate of 6.5%. I explained the difference between these two measures in this post.
But Erickson doubles down on this error. He then states that in California, there have been 33,865 cases out of 285,900 tests (again, I will take those numbers on faith), and says that this means that 12% of Californians are infected.
No, no, no, no, no.
Hopefully you can see what’s going on here. If not, let me break it down. Erickson is conflating the test-positive rate with the prevalence rate. They are distinctly different concepts. To misuse them in this manner is a statistical crime akin to the Lindbergh kidnapping or putting anchovies on pizza.
If I asked 100 of Donald Trump’s best friends if they loved him, and 95 said yes (i.e., 95%), could I then conclude that 95% of the American population also loves Donald Trump? Of course not, because those 100 were not a representative sample of the American population.
True prevalence estimates require a representative sample to be tested. We’ll get to that below.
The test-positive rate, on the other hand, just tells us that among those in California who were tested, 12% were infected. Who gets tested? The symptomatic and those who are likely to have the disease. That is literally the rule for who gets tested.
So imagine if you now loosen the criteria for who gets tested. Let’s say it goes from those who have symptoms and who lived in a house with an infected person, to just those who had symptoms. Would you expect the test-positive rate to go up or down? You’d expect it to go down, because now your denominator is more likely to contain people who do not have the disease.
And if you loosen those criteria further, and started testing hypochondriacs and people with sniffles? Would you expect the test-positive rate to go up or down? Once again, down.
But Erickson literally states, “the more you test, the prevalence estimate goes up.”
Oh sweet Jeebus. The more you test, you will find more cases, but the denominator will rise faster, so the rate will either go down or stay the same. And if it’s a true prevalence measure with proper sampling, the number should stay roughly the same. If it goes up, you’ve detected a major outbreak.
But remember, this isn’t even prevalence. This is test-positive rate that he mistakes for prevalence.
So what is prevalence? Well, as I explained in this post, you get a prevalence estimate from grabbing a truly random sample of the population (i.e., not just people who are presenting with symptoms) and testing them. If the sample is truly random, then multiple random samples of sufficient size should render roughly the same result each time, assuming homogeneous mixing of the population (which is what Erickson is indeed assuming).
Oh, but he goes on.
New York State
He turns his attention to New York, the global epicenter of the pandemic. His numbers, which I do not dispute: New York state has 256,272 cases and has performed 649,325 tests. He divides those two numbers to get 39%… which again he mistakes for a prevalence estimate when it is in fact a test-positive rate.
He then multiplies this false prevalence estimate by the population of New York State (19.4 million) to get a false estimate of the true cases: 7.5 million.
As you will see further below, the true number is likely less than half that.
The Cause-Specific Mortality Rate
Then, Erickson takes the death tally in New York (19,410, he claims) and divides that into the state population of 19.5 million to get an estimate of the chances of dying from COVID in New York. In other words, the cause-specific mortality rate. He gets 0.1% and seems to think that’s a low number.
The cause-specific mortality rate in the USA for the single biggest killer, heart disease, is about 0.16%, and that’s for deaths over the entire year. We’re a few weeks into this epidemic, and COVID-19 is already the 2nd biggest annual killer, according to this metric. A cause-specific mortality rate of 0.1% is huge.
Getting the Death Rate Wrong
Hopefully, it is clear to you now that this presentation is some weapons-grade bullshit. But this is not over yet, my friends. Erickson then states that 92% of cases in New York recover, like that’s a good thing
He gets that number by taking the cases (256272) subtracting the deaths (19410) and dividing that by the cases again. It’s not wrong. But it buries the punchline, that 8% do not recover. That’s the CFR, the case-fatality rate. An 8% CFR is galling. Any infectious disease offering that CFR is a medical emergency, not something to be casually dismissed.
He describes the New York State experience as a “hotspot”, preferring to focus on the perfection that is California. Earlier, he stated that California had 1100 COVID-19 deaths (let’s accept that number) and divided that by his faulty prevalence estimate of 4.7 million to get a California CFR of, sigh, 0.02%.
I really hope you are now seeing how that number is total and complete male cow excrement. The probable true CFR for COVID-19 is substantially higher, which I will explain below. It’s important that you understand that for what Erickson says next.
Comparing COVID-19 to the Flu
This old chestnut. A good 25% of my time has been spent explaining to people how COVID-19 is much worse than the season flu. But I guess we have to do it again.
Erickson accurately cites CDC statistics that state that in the USA the flu typically kills 24,000-62,000 people per year, and that there are about 45 million cases nationwide. That gives a CFR for the flu of about 0.13%, he claims. (Note that he chooses the upper bound estimate, not the lower bound of 24000/45 million = 0.05%, because that would not suit his narrative).
He therefore concludes that since the COVID-19 CFR is 0.02%, it is an order of magnitude less dangerous than the flu.
First of all, as of today, the USA has already clocked in 56,796 COVID-19 deaths. That’s in under 2 months. It already has a death toll equivalent to that of the worst instance of the modern seasonal flu, and that’s with a nationwide lockdown and only a few weeks of outbreak. Give it a full flu season and full reign and see the devastation it would wreak.
That in and of itself should be sufficient to put an end to this flu comparison nonsense. But if not, consider that the flu has a basic reproduction number (the average number of people that a single infected person can infect) of 0.9-2.1. Meanwhile, the pre-lockdown reproduction number of the virus causing COVID-19 has a median of 5.7, meaning that its 2-3 times more infectious than the flu.
So even if the CFRs were equal, COVID-19 would infect so many more people that it would render 2-3 times more deaths.
But the CFRs are not equal, as we shall see next.
So What Are The Real Numbers?
Erickson and Massihi have commited the unforgivable statistical sin of thinking test-positive rate is a prevalence rate. As a result, they assume that there are way more people infected with COVID-19 than is currently assumed. They use that assumption to compute a false IFR and, further, to make horrifically inappropriate comparisons to the seasonal flu.
Actual prevalence studies are underway around the world. An atrocious one was conducted in Santa Clara, California, and found a likely prevalence of 2.8%. This is well below the duo’s assumption of 14%. Mind you, this study was far from the required random sample. They solicited volunteers from a Facebook ad. So while the results are interesting, they are largely useless as an estimator of true prevalence.
A better study was conducted in New York State and found that 14.9% of the state had likely been exposed to the virus. This is also a likely overestimate because the antibody test has a high rate of false positives. But let’s take it at face value.
It’s astoundingly smaller than the 39% that Erickson quotes. The estimate of 14.9% prevalence suggests that 2.9 million people in the state have been exposed (14.9% of 19.5 million residents). If you divide the number of deaths in the state (19,410, according to Erickson) by 2.9 million cases, you get an IFR of 0.67%. (If you’re confused by the terminology, remember that the IFR is the final true fatality rate once all the numbers have been counted, while the CFR is the fatality rate computed with the data we presently have).
That’s assuming that case estimate is not an overestimate (which it is, due to the false-positive rate of the test, and due to the fact that many deaths are not counted because they occur outside of the medical system). So that 0.67% is the lower bound of the true IFR estimate for COVID-19 in New York and likely all of the USA.
Remember that the flu’s death rate is 0.13%. You tell me which number is bigger.
If you need me to spell it out, I will: The upper bound estimate of the flu’s death rate is 0.13%. The lower bound estimate of COVID-19’s death rate is 0.67%.
In fairness and generosity, I will offer that I believe the true global COVID-19 IFR to be 0.3-0.5% (based on absolutely nothing). But even then, it’s three times more lethal than the flu.
What About Herd Immunity?
Oh yeah, that thing. One of the subtexts that always comes out of arguments that the disease is more widespread than we think is that it means herd immunity is around the corner. If you don’t know what that is, check out my description of herd immunity in this post.
Herd immunity for this disease kicks in when around 60% of the population has been infected, has recovered, and has antibodies preventing them from becoming re-infected and/or becoming carriers. Leave aside the controversy about whether recovery confers immunity for this disease; let’s accept that it does. Has any prevalence estimate –however false– suggested to you that we are anywhere near 60% penetration?
Even Erickson’s brazenly inflated estimate of 12% for California, or the more realistic 14.9% for New York State, are far throws from the herd immunity threshold. So put that dream back into the pipe because we are nowhere near there yet.
I think I’ve made my point, right? Erickson and Massihi are probably fine physicians. But they are shit epidemiologists and should have had an actual expert check their numbers before going public.
There are plenty of good reasons to want to end the so-called “lockdown”. There comes a point when the damage done by a depressed economy equals then exceeds the risk posed by a raging infectious disease. We can certainly have that conversation. But that’s not the conversation that Erickson and Massihi were having. Rather, they are trying to convince us that COVID-19 is no big deal.
Well it is a big deal.
There are no easy choices here. We have to be grown-ups and decide which sour pill we can swallow: economic pain or the pain of a rampaging disease. Massaging the statistics to tell a pleasant fairy tale is not a grown-up or useful way to make that choice.
A joint statement from the American College of Emergency Physicians (ACEP) and the American Academy of Emergency Medicine (AAEM): “As owners of local urgent care clinics, it appears [Dr. Daniel Erickson and Dr. Artin Messihi] are releasing biased, non-peer reviewed data to advance their personal financial interests without regard for the public’s health.”