Six months ago I wrote an article discussing the evidence on hydroxychloroquine as a treatment for covid. My conclusion then, based on the small number of trials that had been completed at that point, was that hydroxychloroquine did not appear to be an effective treatment for covid. Many of you were unhappy with that conclusion, and I’ve since then received multiple requests to look at the evidence again. So that’s what I’m going to do now.
A systematic review of randomized controlled trials was published by the Cochrane Collaboration a few weeks back. Unfortunately it only looked at data available up to mid-September 2020, so it’s missing more recent data. But it’s a good starting point for our quest to figure out whether hydroxychloroquine has any role in the treatment or prevention of covid.
I know some of you think we should be looking at the observational studies that suggest hydroxychloroquine is effective against covid. But those studies are rife with confounders, and I don’t think it makes sense to look at a lower quality form of evidence when a higher quality form is available. I often look at observational studies when it comes to nutrition, because higher quality data usually isn’t available. I don’t usually look at observational studies when it comes to medications, where higher quality data is available.
The reviewers identified ten trials that provided data on mortality, with a total of 8,270 participants. The trials varied in size from tiny to huge, with the largest study having 4,716 participants.
Two criticisms have frequently been levelled at the trials of hydroxychloroquine – that they gave the drug too late in the disease course to have an effect, and that they gave either a dose that was too low to have an effect or a dose that was so high that it was toxic. So let’s look at whether that was the case in these ten trials.
Two of the trials, with a total of 716 participants, started giving the hydroxychloroquine within three days after symptom onset. This is great. An additional three studies, with a total of 5,416 participants, started giving the hydroxychloroquine at around day six to nine after symptom onset. This might be too late to have an effect on the disease course. One trial, with 150 patients, didn’t give the hydroxychloroquine until around day 16 after symptom onset, which is almost certainly far too late to have any meaningful effect on the disease course (among those who die of covid, many have already died by day 16). The remaining six trials strangely didn’t report the average time point at which the patients received hydroxychloroquine.
Ok, so this is clearly a problem. If we think hydroxychloroquine needs to be given early in the disease course to have an effect, then only two of the trials clearly qualify for inclusion.
What about the doses given? Were they reasonable?
Hydroxychloroquine is primarily used to treat rheumatic diseases. A normal dose for these conditions is 400 to 600 mg per day. Most of the trials gave 400 to 800 mg per day, which is a reasonable dose, and unlikely to cause problems. One trial gave 1400 mg on day one, which is quite high, followed by 600 mg per day. The two biggest trials, with a total of 6,569 participants (in other words, 79% of all participants in the systematic review) gave the patients very high doses. One gave 1600 mg on day one, followed by 800 mg on the following days. The other gave 2,000 mg on day one, followed by 800 mg on the following days. These doses are much higher than those used for rheumatic diseases, and potentially high enough to induce toxic side effects. I guess these high doses could be motivated by the hope that you will have an increased probability of seeing benefit, but they also increase the risk of side effects significantly.
Both these trials did in fact show harm, with a 9% increase in the relative risk of death among those treated with hydroxychloroquine in one, and a 19% increase in the relative risk of death among those treated with hydroxychloroquine in the other. This harm could have been due to giving the patients a toxic dose. It’s certainly not possible to say what would have happened if the patients had been given a more normal dose.
Now we’re going to look at what results that Cochrane review had, but it’s clear that we’re also going to have to do some meta-analyzing of our own, to see what happens if hydroxychloroquine is given early, and also what happens if only non-toxic doses are given. So, what did the reviewers find?
There was a 9% increase in the relative risk of dying in the group treated with hydroxychloroquine, but it was not statistically significant. So, basically, the meta-analysis showed no benefit in terms of effect on mortality.
Now let’s see what happens if we do a little meta-analyzing of our own. In case you haven’t heard of meta-analyses before, a meta-analysis is basically just when you add the results of a bunch of different studies together in order to get a result that has greater statistical power. Considering that most randomized trials in health and medicine are too small to detect differences even if they do exist, this is often a necessary step in order to figure out if a treatment works or not.
Let’s look first at what happens if we remove the two studies that gave very high doses of hydroxychloroquine (which unfortunately means removing the vast majority of participants from the analysis). I’m also not going to bother to include the study (Tang et al.) that gave hydroxychloroquine 16 days after symptom onset (seriously, what were they thinking?). That leaves six studies. Here’s what we get when we meta-analyze them:
Unfortunately, there is still no signal of benefit. 2% of patients died in the group treated with hydroxychloroquine, and 1,7% died in the control group. The marginal difference seen is nowhere near statistically significant, and there is still not even a hint that hydroxychloroquine might be beneficial.
But of course, deaths aren’t the only thing that matter. If people can be kept out of hospital, or prevented from developing severe disease that requires invasive ventilation, that is also a win.
Only one of the studies (Cavalcanti et al.) looked at the ability of hydroxychloroquine to keep people from requiring invasive ventilation. In the hydroxychloroquine group, 7,5% required invasive ventilation, compared with 6,9% in the control group. This marginal difference was again nowhere near statistically significant.
So there is no signal that you get better results if you give more moderate doses of hydroxychloroquine. Let’s see what happens to mortality if we just look at the two studies (Skipper et al. and Mitja et al.) that gave hydroxychloroquine within three days of symptom onset. In these two studies, one person died in the hydroxychloroquine group, and one person died in the control group. Unfortunately, the total number of participants was only 700, which simply isn’t enough to tell if hydroxychloroquine given early has any effect on mortality, for the simple reason that the vast majority of people who get covid don’t become sick enough to die, regardless of what is done with them.
These two studies did however also look at the ability of hydroxychloroquine to keep people out of the hospital. Here’s what happens if we do a meta-analysis of hospitalizations instead:
Both studies appear to show a reduction in the number of people who go on to require hospitalization, and the reduction is actually quite big, with a relative risk reduction of 38%. However, the reduction is not statistically significant.
Of course, that doesn’t mean the reduction isn’t real. If you see a big difference between groups that isn’t statistically significant, that usually means the data set is too small. As mentioned, the data set here is only around 700 people, which is why even a huge 38% reduction in hospitalizations is not statistically significant. So this data suggests that there is a beneficial effect of hydroxychloroquine when given within three days of symptom onset, but more data is required before any conclusions can be drawn.
So, what can we say so far?
The evidence that exists from the randomized trials that had been published up to September doesn’t suggest any benefit from hydroxychloroquine when given at one week after the beginning of symptoms or later in the disease course. We can be pretty confident of this conclusion, since it remains even if we take away the two studies that used very high doses of hydroxychloroquine, and there isn’t even a signal of benefit in any of the trials. This means that hydroxychloroquine doesn’t have any role to play in the hospital, since people usually aren’t sick enough to require hospitalization until they’ve been symptomatic for at least a week.
There is a suggestion from from the two studies that did look at giving hydroxychloroquine early in the disease course that it might significantly decrease the risk of requiring hospitalization. A much larger trial would need to be carried out to see whether that signal is real or not.
So, has any new evidence appeared since September that might change or strengthen these conclusions?
Yes, there have been a few new trials produced since then, which we can use to update our meta-analysis. There’s a French study by Dubée et al. that gave hydroxychloroquine to high risk groups, with the start of treatment on average five days after the beginning of symptoms. This study actually showed a 46% reduction in the relative risk of death in the hydroxychloroquine group, but the study only had 250 participants and 17 deaths overall, so as with most of the earlier studies, this one was too small to produce a statistically significant result, even if one did exist. The result could easily be due to chance.
There’s a Mexican study by Cardenas et al. that gave hydroxychloroquine on average ten days after the beginning of symptoms, with a total of 214 participants. This study showed a marginal 7% reduction in relative risk of death in the hydroxychlororquine group, but again, the reduction wasn’t anywhere close to being statistically significant, and in terms of absolute numbers, there were 40 deaths in the hydroxychlororquine group and 44 in the placebo group, a difference that is so small that it could easily due to chance. The fact that 84 out of 214 participants in the study died is shockingly high, and makes me wonder a bit what they’re doing to their patients in Mexican hospitals. I guess they were only recruiting the very sickest patients in to the study.
There’s a study carried out in the United States by Self et al. that gave hydroxychloroquine an average of 5 days after symptom onset, and that included 479 patients. 25 patients died in the hydroxychloroquine group, and 25 patients died in the placebo group. This was a big, high quality trial, and the hydroxychloroquine was given relatively early, so the lack of even a hint of an effect on mortality is disappointing, and strongly suggests that even if hydroxychloroquine does have an effect on the risk of dying of covid, that effect is modest at best.
And finally there’s a study carried out in Qatar by Omrani et al. that gave hydroxychloroquine to young, otherwise healthy people with mild or asymptomatic disease (i.e. people without symptoms but who had a positive covid PCR test when they were screened). There were 152 people in the treatment group and 152 people in the control gorup. Not surprisingly, there were no deaths in either group. When it came to hospitalizations, there were three hospitalizations in the treatment group and four in the control group.
Here’s what we get when we add these studies to the meta-analysis:
What we see now is a marginal 7% reduction in the relative risk of death, that is nowhere close to being statistically significant. I therefore think it’s pretty safe to conclude that hydroxychloroquine is either completely ineffective when given on day 5 after symptom onset or later, or at best so marginally effective that it’s probably not worth bothering about. For comparison, remember that when we meta-analyzed the trials of ivermectin, there was an 87% reduction in relative risk of death that was highly statistically significant! Here’s the meta-analysis of ivermectin again, as a reminder of how impressive those results were:
Unfortunately, there haven’t been any new trials looking at hydroxychloroquine administered within the first three days after symptom onset, so we still don’t know whether it’s effective at preventing hospitalization when given that early or not. It’s a shame that question still hasn’t been answered, a full year in to the pandemic.
The study by Omrani et al. that I mentioned earlier was giving hydroxychloroquine early, but they never specify exactly how early, and a large portion of participants in that study weren’t actually sick, they were just asymptomatic carriers. But let’s add that study to our meta-analysis of hydroxychloroquine given within three days to prevent hospitalizations, just for fun. Here’s what that meta-analysis looks like:
Unfortunately, even with the addition of that trial, we’re still nowhere near statistical significance. To be fair, there is still a suggestion here that hydroxychloroquine, when given early, can decrease the risk of hospitalization. All three trials are pointing in the same direction. But even when added together, there is still not sufficient statistical power to say whether we are seeing real benefit or just random chance creating the appearance of benefit. What we would really need now is a big study, with at least a thousand participants, preferably two thousand, that were given hydroxychloroquine within three days of symptom onset. That would allow us to answer this question definitively.
Some people have argued that hydroxychloroquine only works when combined with azithromycin and/or zinc. Let’s examine that. The idea that hydroxychloroquine and azithromycin are effective when combined (and also that hydroxychloroquine is effective on it’s own, for that matter) is based on a tiny little French study (Gautret et al.) of very dubious quality that was conducted in early 2020, in which eight patients received hydroxychloroquine plus azithromycin, while 18 received standard care.
Personally I usually don’t even bother looking at studies this small because they’re far too small to yield any useful information, and really just waste everyone’s time. Anyway, what the researchers found was that the patients treated with hydroxychloroquine became negative on a covid PCR test more rapidly than the patients receiving standard care, while the patients getting hydroxychloroquine and azithromycin together became PCR negative a little faster still. From such small, questionable findings can big mountains of nonsense be built.
Since that study, three reasonably large studies looking at the combination of hydroxychloroquine and azithromycin have released their results (in fact, we’ve already talked about two of these studies a bit, because they also had a treatment arm that only received hydroxychloroquine).
The first (Cavalcanti et al.) gave 800 mg hydroxychloroquin per day and 500 mg azithromycin per day to 217 patients. These were compared with a control group consisting of 229 patients. Treatment started on average seven days after the beginning of symptoms. Five patients died in the intervention group and six patients died in the control group.
The second (Omrani et al.) gave 600 mg hydroxychloroquine per day and 500 mg azithromycin on day one, followed by 250 mg of azithromycin on the following days. There were 152 people in the intervention group and 152 people in the control group. As mentioned earlier, the study was only treating people with very mild covid, and unsurprisingly there were no deaths among the participants. Nor was there any difference in hospitalizations. Four people were hospitalized in the group receive hydroxychloroquine and azithromycin, and four people were hospitalized in the control group.
To be fair, this study was never statistically powered to look at hard end points. Instead it was powered to look at the probability of going from having a positive PCR test to a negative PCR test after six and 14 days from the beginning of treatment. So basically it was a higher quality version of the French study that started the whole hydroxychloroquine+azithromycin discussion.
Personally, I think this is a nonsensical endpoint because it is completely irrelevant to patients whether they still have a positive PCR test or not at a certain time point. But since this is what they were looking at, I’ll report what they found. At day six there was no difference between the groups. There was a trend towards decreased PCR positivity at 14 days among those treated with both hydroxychloroquine and azithromycin (30 people with a negative test in the intervention group vs 45 in the control group), but it wasn’t statistically significant.
Moving on. The third trial (Furtado et al.) was carried out in Brazil. It wasn’t comparing azythromin and hydroxychloroquine to placebo. Rather it was comparing the combination against just getting hydroxychloroquine. Treatment started on average eight days after symptom onset. Patients had to be quite sick to begin with to be included in the study. At the very least, they had to be getting 4 liters of oxygen, and half the patients were being invasively ventilated at the time they were included in the study. 214 patients received the combination therapy, while 183 just received just hydroxychloroquine. Overall, 42% of the patients in the combination therapy group died, compared with 40% in the group that only received hydroxychloroquine. In other words, there was no signal of benefit.
Based on these studies, I would say that there is no good evidence at this point that a combination therapy consisting of hydroxychloroquine and azithromycin has any role to play in the treatment of covid-19. Apart from hydroxychloroquine and azithromycin, there also has been a separate discussion about combining hydroxychloroquine with zinc. Is there any evidence to support this combination?
I have been able to identify exactly one completed randomized trial that has looked at combining hydroxychloroquine with zinc (Abd-El Salam et al.). 96 patients were treated with a combination therapy consisting of hydroxychloroquine (800 mg on day one, and thereafter 400 mg per day) and zinc (100 mg per day), while a control group consisting of 95 patients only received hydroxychloroquine.
The patients varied in how sick they were at the time point when they were included in the study, from only mildly ill to critically ill. No information is provided about the time point in the disease course at which patients started getting the treatment. Five patients died in the combination therapy group and five patients died in the hydroxychloroquine only group. There was no statistically significant difference in any of the outcomes studied.
Ok, this is just one relatively small trial. So the main thing we can conclude about the trial evidence on hydroxychloroquine combined with zinc is that it is extremely limited, really too limited to draw any firm conclusions. However, there is no signal of benefit in the one trial that has published results.
Two observational studies, both carried out in New York (Carlucci et al. and Derwan et al.), appear to show promising results in patients treated with a combination of hydroxychloroquine, zinc, and azithromycin. The first, involving 932 paitents, showed a 55% reduction in relative risk of death among those getting zinc in addition to hydroxychloroquine and azithromycin, as compared with those just getting hydroxychloroquine and azithromycin. The second, involving 518 patients, showed an 80% reduction in relative risk of death among those getting the full triple therapy, when compared with a reference sample that didn’t get any of the drugs.
That seems impressive. However, these are low quality observational studies that are seriously limited by the methodology used. The scope for confounding effects is huge. Therefore, these studies should be considered exploratory and hypothesis generating. They certainly should not be considered evidence of any cause and effect relationship.
Ok, let’s wrap this up. What can we conclude from all these studies?
The evidence that exists at the present point in time does not support the use of hydroxychloroquine as a treatment for covid. There is a signal that hydroxychloroquine could potentially decrease the risk of serious illness when given within three days of symptom onset, but there still isn’t enough trial data available to know whether that signal is real or not.
There is also a signal from observational studies that triple therapy consisting of hydroxychloroquine, zinc, and azithromycin is effective against covid, but no randomized trials have yet been done of the triple therapy that can corroborate that effect. The trials that have so far been published that looked at hydroxychloroquine in combination with azithromycin, and hydroxychloroquine in combination with zinc, have failed to show any benefit.