The latest in the long succession of attempts at maximizing people’s fear of covid is the claim that it causes brain damage. And not just in those who have spent time in the ICU, in everyone, even if all they had was a mild cold. The claim is currently doing the rounds on social media (apparently alarmist propaganda only counts as misinformation if it’s going against the dominant narrative). The assertion comes from a paper that’s recently been published in EClinicalMedicine (a daughter journal of The Lancet). The paper is actually quite illuminating about the current state of medical research, so I thought it would be interesting to go through it in some detail.
81,337 individuals residing in the UK completed an on-line test of their cognitive function. They also provided information on their covid status (whether or not they thought they’d had it, and how sick they were), as well as a bunch of other demographic information. The data was collected from January to December 2020.
12,689 (16%) of the 81,337 participants indicated that they thought they had had covid-19. They were sorted by the researchers in to five categories based on the severity of disease, from “ill without respiratory symptoms” to “hospitalised and on a ventilator”. The results from these five categories were then compared with the results from the 68,648 people that didn’t think they’d had covid.
The reason the study is causing such a stir is because of the results. All five of the “I think I’ve had covid” categories performed worse on the cognitive function test than the “I don’t think I’ve had covid” category did. The reduction in performance was correlated with the severity of disease, with the people who had been on a ventilator performing worst – according to the researchers their results were equivalent to a seven point reduction on an IQ test. If we assume that the non-covid group have an IQ of 100, this would mean that the group that had been on a ventilator have an IQ of 93.
Ok, open and shut, right? Having covid makes you more stupid, and the more severe disease you have, the more stupid you become. Well, not quite.
The first thing that needs to be pointed out is that this was an observational study. Observational studies cannot usually say anything about cause and effect, because the participants haven’t been randomly assigned to the different groups (as they would have been in a randomized controlled trial). The inability to draw any conclusions about cause and effect is especially true when the difference between the groups is small, as it is in this study. There could well be major underlying differences between the groups that explain the differences in performance on the cognitive function test.
When we go through the demographic data, we see that this is actually the case, in particular when it comes to chronic conditions. Chronic liver disease (such as for example liver cirrhosis) was more common in those who thought they had had covid, and the relative rate increased the more severely sick people had been with covid. Chronic lung disease (such as COPD) and chronic kidney disease also co-varied with severity of covid. These underlying illnesses could on their own confound the results enough to explain the differences in cognitive performance seen in the study. People with underlying chronic diseases have worse cognitive function, and they’re also more likely to become severely ill if they get covid. Just because you see a correlation doesn’t mean there’s a cause and effect relationship!
The groups also varied in terms of the proportion in each category that had ADHD. The people who didn’t think they’d had covid were less likely to have ADHD than the people who thought they’d had covid. Oddly, severity of disease correlated quite closely with the probability of having ADHD. This matters, because it’s likely that people with ADHD will underperform on many parts of a cognitive function test. If the researchers wanted to, they could have interpreted this as showing that covid causes ADHD. But they didn’t, because that would be silly. Yet the exact same logic (correlation between two variables in observational data) was used to claim that covid causes brain damage.
It’s worth noting that for all the possible confounding factors that the authors of the study have asked the participants about and tried to account for, there are many more that they haven’t asked about, and that could also explain the results seen in the study. Confounding isn’t something that should be taken lightly, which is why conclusions about cause and effect shouldn’t be drawn from purely associational data.
The second thing that needs to be pointed out is that this study was cross-sectional. In other words, participants only had their cognitive function tested at one time. That in itself makes it impossible to say anything about whether the participants performance decreased after having had covid, because we have no idea what their performance was before they got covid. If you want to know if something has changed over time you need to do a longitudinal study, where you test people multiple times.
The fact that the study was observational and cross-sectional, and that there were big underlying differences between the groups, is on its own enough to disqualify any claims about this study being able to show that covid causes brain damage. But it gets worse. A lot worse.
A major problem with the study is that 97%(!) of the people who thought they’d had covid lacked testing to confirm the diagnosis. Of the 12,689 that thought they’d had covid, only 386 actually had a confirmed diagnosis. The only group in which the majority actually had a positive test confirming that they had had covid was the group that had been on a ventilator in an intensive care unit! If you can’t even be sure that 97% of participants actually had the disease you’re trying to draw conclusions about, then you really don’t have a leg to stand on.
I think it’s worth remembering that, even during the covid peak, only around 20% of covid tests were coming back positive. In other words, even when covid was spreading at its most rampant, most people who had a respiratory infection did not have covid. They had something else. It is therefore reasonable to think that at least 80% of the 97% (i.e. at least 78% of participants) that think they had covid, did not in fact have it. What that means is that the study is rubbish, and cannot make any claims about covid whatsoever. Yet it does. And it’s been published in a peer reviewed journal.
To me, the main lesson here is that we currently live in a world where junk science goes unquestioned and gets published in peer-reviewed journals as long as it feeds in to the dominant narrative. If this study had been claiming, say, that face masks didn’t work, then it would remain stuck at the pre-print stage forever, or, if it ever did get published, it would immediately have been retracted. It has become blatantly obvious over the past year and a half that it is not primarily the quality of studies that determines where and whether they get published, but rather their acceptability to the powers that be.