How accurate are the covid tests?
One of the most frequent questions I’ve been getting recently is how accurate I think the covid tests are, and in particular the PCR tests. As it happens, a systematic review has recently been published in Evidence Based Medicine that looks at the covid tests (both PCR and antibody), so I thought it would be interesting to look in to the evidence together. This article gets a bit technical and math-heavy in places, so please bear with me. I think the payoff is worth it.
First, let’s make sense of what the two types of test are and how they work. The PCR (Polymerase Chain Reaction) test is designed to detect a specific sequence of nucleotides, and when it comes to detecting SARS-CoV-2, the sample is usually taken from the back of the throat. Nucleotides are the building blocks of genomes, and the idea is that if you can detect a string of nucleotides that is specific for a certain organism, then that proves the organism is present at the sample site. Since PCR is designed to detect bits of viral genome that are currently present in your respiratory tract, its purpose is to detect a currently active infection (as opposed to a past infection).
PCR works by repeating a series of chemical reactions over and over. If the sequence of nucleotides that is sought is present in the sample, then each time the reaction is repeated, the number of copies of the sequence will double, so that more and more copies accrue.
So, if you start of with one copy of the nucleotide sequence you are looking for, then after one cycle you will have two copies. After two cycles you will have four copies. After three cycles, you will have eight copies. After four cycles, you will have 16 copies. And so on. As you can see, the fact that each cycle doubles the number of copies means that the numbers quickly build to massive levels. The covid PCR tests frequently keep going up to 40 (or sometimes even 45) times.
If you start off with just one copy of the viral nucleotide sequence in the sample, then after 40 doublings, you will have over 1,000,000,000,000 copies (that’s one thousand billion copies). The reason you do this repeated cycle of doubling, is that once you get enough copies of the sequence you’re looking for, then you can use other technologies to detect it. For example, you can add molecules to the sample that visibly light up if enough copies of the sequence are present. So after enough copies are present in the sample, then they can be detected, and you get a positive result.
The number of times you choose to cycle through the steps of PCR before you decide that there was no virus in the sample after all is known as the cycle threshold. The number of cycles used to get a positve result is actually a pretty important number, because it tells you how much virus is in the sample. The lower the number of cycles required, the more virus is in the sample. The higher the number of cycles, the more likely that the result is a false positive, caused perhaps by having a tiny amount of inactive virus in the respiratory tract, or by contamination of the sample in the lab. Like I said, after 40 cycles, even a single copy of the viral sequence has become over one thousand billion copies.
One thing that’s important to understand at this point is that PCR is only detecting sequences of the viral genome, it is not able to detect whole viral particles, so it is not able to tell you whether what you are finding is live virus, or just non-infectious fragments of viral genome. If you get a positive PCR test and you want to be sure that what you’re finding is a true positive, then you have to perform a viral culture. What this means is that you take the sample, add it to respiratory cells in a petri dish, and see if you can get those cells to start producing new virus particles. If they do, then you know you have a true positive result. For this reason, viral culture is considered the “gold standard” method for diagnosis of viral infections. However, this method is rarely used in clinical practice, which means that in reality, a diagnosis is often made based entirely on the PCR test. A systematic review looking at the ability to culture live virus after a positive PCR test found that the probability of a false positive result increased hugely with each additional cycle after 24 cycles. After 35 cycles, none of the studies included in that review was able to culture any live virus.
In most clinical settings (including the one I work in), all the doctor is provided with is a positive or negative result. No mention is made of the number of cycles used to produce the positive result. This is a problem, since it’s clear that a positive result after 40 cycles is almost certainly a false positive, while a positive result after 20 cycles is most likely a true positive. Without information about the number of cycles, you have to assume that the patient sitting in front of you has covid and is infectious, with all the downstream consequences that entails.
Anyway, enough about the PCR test for now. The other main type of test is the antibody test. Here, the sample is usually taken from the blood stream. There are five different types of antibodies, but most antibody tests only look for one type of antibody, IgG, which is the most common type. Generally it takes a week or two after a person has been infected before they start to produce IgG, and with covid, you’re generally only infectious for about a week after you start to have symptoms, so antibody tests are not designed to find active infections. Instead the purpose is to see if you have had an infection in the past.
One common method that is used for antibody tests is ELISA (enzyme linked immunosorbent assay). In this method, you have a plate on which you’ve fixed antigen that the antibody you are looking for can bind to (antibodies bind to antigens – antigen is short for “antibody generator”, and it’s basically the molecular structure that a certain antibody is specifically designed to bind to).
You then add the blood sample that you want to study to the plate, at which point the antibodies in the sample will bind to the antigens (assuming the antibodies you want to find are actually present in the sample). After that you wash the plate, so that any other antibodies in the sample that you’re not actively looking for are washed off (since there’s no antigen for them to bind to).
Next you add a signaling molecule that can bind to antibodies, and which has the ability to change color when exposed to a certain enzyme. You then wash the plate again. If there are no antibodies stuck to the plate for this molecule to bind to, it will wash off. If the antibodies you are looking for were present in the blood sample, they will have stuck to the antigen on the plate, and this new molecule will in turn have stuck to them.
Finally you add an enzyme that changes the color of the signaling molecule. If the signaling molecule hasn’t been washed off in the previous step, then you will see the plate change color, and the antibody test is positive.
Apart from understanding how the tests work, we also need to understand two important terms before we get in to the details of the recent systematic review. Those terms are sensitivity and specificity, and they are critical for all diagnostic tests used in medicine, because they tell you how good a test is.
Sensitivity is the probability that a disease will be detected if the person actually has the disease. So, for example, a test for breast cancer with a sensitivity of 90% will detect breast cancer 90% of the time. Nine out of ten patients with breast cancer will correctly be told that they have the disease. One out of ten will incorrectly be told that they don’t have the disease, even though they do.
Specificity is the opposite of sensitivity. It is the probability that a person who doesn’t have the disease will be told that they don’t have the disease. So, a specificity of 90% for our imaginary breast cancer test means that nine out of ten people who don’t have breast cancer will be correctly told that they don’t have it. One out of ten people who don’t have breast cancer will incorrectly be told that they do have it.
To put it another way, sensitivity is the ability of a test to detect true positives. Specificity is the ability of a test to avoid producing false positives. A perfect test will have a sensitivity and specificity of 100%, which would mean that it catches everyone who has the disease, and doesn’t tell anyone they have the disease if they don’t. No such test exists. In general, sensitivity and specificity are in conflict with each other – if you push one up, the other will go down.
If I just told everyone I meet that they have breast cancer, my sensitivity for detecting breast cancer would be 100%, because I wouldn’t miss a single case, but my specificity would be 0%, because every single person who doesn’t have breast cancer would be told that they do. So, when designing a test, you have to decide if you’re going to maximize sensitivity or specificity. If you design a covid PCR test with a cycle threshold of 40, then you are going for maximal sensitivity – the probability of missing a case is minimized, but you’re going to get a lot more false positives than if you set the threshold at 30.
Ok, now that we know what a PCR test is and what an antibody test is, and understand sensitivity and specificity, we can move on to the recent systematic review. The review included 38 studies of PCR tests (and LAMP tests, an alternative technique that is similar to PCR). The overall sensitivity for PCR/LAMP was between 75% and 100% in the different studies, while the overall specificity was between 88% and 100% . 16 studies, with a total of 3,818 patients, were able to be pooled together to get a more accurate estimate of sensitivity. In the pooled analysis, sensitivity was determined to be 88% . It wasn’t possible to determine a pooled specificity value, since the studies included in the pooled analysis were all of people who were already known with complete certainty to be infected with covid.
The review included 25 studies of antibody tests, but only ten of these (with a total of 757 patients) provided enough data to allow sensitivity to be calculated. The sensitivity of the antibody tests varied from 18% to 96%. 12 studies provided enough information for specificity to be determined, and in these it varied from 89% to 96% .
Ok, it might be hard to understand what these numbers mean in practical terms, so we’re going to play around with them a bit in order to clarify this, and I’m going to focus on the PCR test in this final discussion, since that is what’s generating much of the hysteria around covid. As mentioned, the sensitivity of the PCR test seems to be around 88% . A good value for the specificity is harder to determine, but it’s somewhere between 88% and 100%, so if we assume a specificity of 94% (halfway between the two values) we’re probably not far off.
Let’s say the disease is spreading rampantly through the population, and one in ten people are infected at the same time. If we test 1,000 people at random, that will mean 100 of those people actually have covid, while 900 don’t. Of the 100 who have covid, the test will successfully pick up 88. Of the 900 who don’t have covid, the test will correctly tell 846 that they don’t have it, but it will also tell 54 healthy people that they do have covid. So, in total 142 people out of 1,000 are told that they have covid. Of those 142 people, 62% actually have the disease, and 38% don’t.
That’s not great. Four in ten people getting a positive test result don’t actually have covid, even in a situation where the disease is so common that 10% of people being tested really do have the disease.
Unfortunately, it gets worse. let’s assume the disease is starting to wane, and now only one in a hundred people being tested actually has covid. If we test 1,000 people, that will mean ten will really have covid, while 990 won’t. Of the ten who have covid, nine will be correctly told that they have it. Of the 990 who don’t have it, 931 will be correctly told that they don’t have it, while 59 will be incorrectly told that they do have the disease. So, in total, 68 people will be told that they have covid. But only 9 out of 68 will actually have the disease. To put it another way, in a situation where only 1% of the population being tested has the disease, 87% of positive results will be false positives.
There is another thing about this that I think is worth paying attention to. When one in ten people being tested has the disease, you get 142 positive results per 1000 people tested. But when one in a hundred has the disease, you get 68 positive results. So, even though the actual prevalence of the disease has decreased by a factor of ten, the prevalence of PCR positive results has only decreased by half. So if you’re only looking at PCR results, and consider that to be an accurate reflection of how prevalent the disease is in the population, then you will be fooled, because the disease will seem to be much more prevalent than it is.
Let’s do one final thought experiment to illustrate this. Say the disease is now very rare, and only one in a thousand tested people actually has covid. If you test 1,000 people, you will get back 61 positive results. Of those, one will be a true positive, and 60 will be false positives. So, even though the prevalence of true disease has again decreased by a factor of ten, the number of positive results has only decreased slightly, from 68 to 61 (of which 60 are false positives!). So by looking just at positive PCR tests, you can easily be convinced that the disease is continuing to be roughly as prevalent in the population, even as it goes from being present in one in a hundred people to only being present in one in a thousand. The rarer the disease becomes in reality, the less likely you are to notice any difference in the number of tests returning positive results.
I want to restate this again, in a slightly different way, to make sure the message sinks in. As the disease drops enormously, by a factor of 100, from affecting one in ten to one in a thousand tested people, there is little more than a halving in PCR positive results, from 142 to 61. So a huge reduction in real infections only causes a small reduction in PCR confirmed “cases”. In fact, the disease could vanish from the face of the Earth, and you would still be getting 60 positive results for every 1,000 tests carried out!
The same trend is seen even if the PCR test were to have a much better specificity than we are estimating here, of say 99% . Here’s a quick illustration, since I don’t want to tire you with too many more numbers. If one in ten has the disease and you test 1,000 people, you will get back 97 positive results, of which 88 will be true positives and 9 will be false positives. If one in 100 has the disease, you will get back 19 positive results, of which 9 will be true positives and ten will be false positives. If one in 1,000 has the disease, you will get back 12 positive results, of which 11 will be false positives.
So, even if the test has a very high specificity of 99%, when the virus stops being present at pandemic levels in the population and starts to decrease to more endemic levels, you quickly get to a point where most positive results are false positives, and where the disease seems to be much more prevalent than it really is.
As you can see, the less prevalent the disease is in reality, the more likely the test is to generate a false positive result, and the less useful the test is as a method for figuring out who actually has covid. And the less prevalent the disease is, the more prevalent it will seem to be in relation to reality. If decisions about covid continue to be made largely based on what PCR tests show, we might never be able to call off the pandemic!
And that, ladies and gentlemen, is why PCR positive cases are a very poor indicator of how prevalent covid is in the population, and why we should instead be basing decisions on the rates of hospitalization, ICU admission, and death. If we just look at the PCR tests, we will continue to believe that the disease is widespread in the population indefinitely, even as it becomes less and less common in reality. And that is assuming the rate of testing doesn’t increase. If we combine this built-in problem with accuracy, with a massive increase in testing (as has happened in most countries over the course of the pandemic), then we can create the impression of a disease that is continuing to spread wildly through a population, even when it isn’t.
You might also be interested in my article about how deadly covid actually is, or, if you want to dig further in to the problems created by testing, you might be interested in my article about breast cancer screening.
Thank you Sebastian , this is invaluable and the best article on PCR for covid that has been written.
Hopefully it will help more people understand the reality.
Do you have any idea of the cycle threshold of PCR testing in various countries?
I can’t find such information, even though it is crucial .
Thanks Samuel,
I’m not sure, but my guess would be that most countries do 40+ cycles. I know that’s the cases here in Sweden, and at present I only know of one hospital (Sahlgrenska in Gothenburg) where doctors are provided with the CT value along with the result.
Hi Sebastian!
Is Sweden really doing 40+ cycles? Norway’s standard is 33 cycles, which makes much more sense.
https://www.fhi.no/nettpub/coronavirus/testing-og-oppfolging-av-smittede/testkriterier/#hvordan-tolke-proevesvar
In my opinion, it is essential to follow this advice from rationalground.com “The FDA should update their guidance to recommend no more than 34 cycles, require labs to communicate the number of cycles required to detect the virus for each positive test, and [this is the crucial] require labs to disclose the cycle threshold for all previous COVID tests (if that data is available) to clean up the inflated statistics (cases, hospitalizations, and deaths) associated with test results that exceeded 34 cycles.”
https://rationalground.com/why-mass-pcr-testing-of-the-healthy-and-asymptomatic-is-currently-counter-productive/
I see little reason for not presenting a full disclosure of the cycle threshold for every possible PCR-test we can get hold of for SARS-Cov2 from all around the world the last year. Or at least a list from the standard being used for every country. Imagine what a full clean up of inflated statistics could do for a world that is unreasonably anxious because of this virus.
Shouldn’t the false positive rate be a function of the amount of virus in each of the swabs – the more there is, the more the chance of cross contamination and vice versa?
Thank you for your explanation -the first time I’ve managed to grasp what a ‘false positive’ is and how it can corrupt the statistical evaluation of the real state of a population’s infection.
These numbers ensure that we will always have enough ‘cases’ above the, then maybe not so arbitrarily chosen, threshold of 50/100.000 even if the disease was eradicated.
Although the fact that we were often below that number during the summer suggests a better specificity- or how would you explain that?
I just want to add that Germany’s Drosten test is based upon a ridiculously high 45 cycles, whilst Fauci himself has stated that anything beyond a 35ct is meaningless, see link, and that the more or less case-free NFL is using tests with a number of 30ct (link at jordan schachtel).
https://blog.nomorefakenews.com/2020/11/06/smoking-gun-fauci-states-covid-test-has-fatal-flaw/
In addition, I have read that instead of properly searching for 3 DNA snippets of the virus (whether the snippets are chosen properly in light of the virus seemingly not yet done proper isolation is a whole different story and can of worms) and adding a confirmation phase on top of the initial search phase, many if not most tests and labs these days search for only one, the E snippet, and omit that confirmation phase, leading to further inflated and completely false positive results.
(I have also read that the choice of the primer can make a huge difference, but cannot evaluate that.)
Prof. Hockertz has stated that in every other medical area but Covid, people who sell or performed such uncertified and non-standardized tests would immediately be jailed.
That lack of standardization and certification alone renders the tests, testing and all comparisons and decisions based upon them and it completely obsolete and fraudulent, which is why Dr. Fuellmich is going to sue Drosten&co.
I suppose, that standardization and certification will eventually come, at a low rate of say a 25ct, as this would be the best and easiest, if not even only, way to end the casedemic, and also, the best, if not even only, way to wrongly credit the vaccines for that eradication and termination and thereby make them, and the strategies chosen sofar, seem a success.
Thank you
Explains the ludicrousness of the PCR lark
In the UK its standard practice to test up to 40 or 45 cycles depending on your assay – we take anything at 37+ with a pinch of salt and usually repeat it on another assay with different target or methodology. In my lab we use a two target LDT PCR and we usually use a TMA assay or vice versa for any confirmation tests required.
Thank you for walking the reader through the mathematics of testing. Astonishing how doctors are not being informed on the specific testing procedure (PCR CT value) although it makes such a big difference in the test’s validity to find active infections!
The reporting of Covid “cases” all over the world, seen in the light of science, is pure scaremongering without further verification of actual infections. Even in “science-based Sweden”… I find this very irritating indeed, and hope that doctors like you will be listened to by the health authority!
Thank you also for your article on how deadly Covid really is!
this is absolutely excellent. It amazes me that when I first mentioned you to my Swedish wife she told me a few minutes later that you didn’t exist according to her internet sources!. Anyway my question is to do with Koch’s postulate and whether the virus has been isolated and whether it is now mutating and vulnerable to vaccines. Or should we abandon vaccination as an unnecessary goal in favour of therapeutics like HCQ, zinc and Azithromycin? thanks so much for all your hard work informing us of the science in this complex area.
Haha, thanks Peter. It’s been at least a few months now since I received an e-mail from someone asking if I’m a real person and a real doctor, so I think a general acceptance that I am real has now been reached!
Well, that’s a difficult question. We’ve known about the four ”common cold” coronaviruses for decades now, but no vaccine has yet been developed. At the same time, I doubt anyone has really tried.
Coronaviruses don’t mutate very rapidly, so it should be possible to create a vaccine that is at least moderately effective.
I agree with pcr not being reliable for an ongoing infection and that it is better to look at ICU admission etc but I think your exercise here is primarily a theoretical one. Meaning there is nothing that says the sensitivity or specificity scales equally at any given number of tests done. Testing is done at different places which cannot all scale up equally, contamination depends on the location where the sample is taken and studied and the infectivity of people in the surrounding. These all vary with time to make it more complex.
Thanks for very clear explanations and great analysis. However, the positivity rates being reported (the faction of tests that come back positive) show that the estimate you are using for the specificity of PCR is much too low. Here in Massachusetts, the positivity rate for the two weeks Oct.11-24 was 1.55%. That means 98.45% of the people tested got a negative result. Even if all of those were false positives, that would imply a specificity of 98.45%. Since some of those tested were true positives, the actual specificity is higher than that. Furthermore, in the particular Massachusetts city I live in, the positivity rate was 0.36%, which implies a specificity of more than 99.64%. I have not looked at the particular studies you mentioned that claimed a lower specificity than that, but the tests here in Massachusetts have a much higher specificity. In those states that have a higher positivity rate than Massachusetts (which is most of them), we can be sure that most of the PCR positives are true positives.
Thanks Irwin,
That’s why I also included calculations based on a specificity of 99% .
In that scenario, the numbers aren’t quite so bad, but you still quickly get to a point where most positives are false positives and where the disease seems more prevalent than it is. Do you know what cycle threshold is used in the city where you live?
Once again, from the popular press- tabloid, broadsheet and news ‘magazines’, – not a column-centimetre of explanation / investigation into the Foundational Stone of the Narrative. Just regurgitation of fear, lies and manipulation.
Thank you again for the clear simple & succinct illumination of dark deeds !!!
More like the Pure Gold value of PCR. – It can make ‘something’ out of ‘nothing’ so no further lies required 😉
That’s a very good explanation. Thanks!
I want to add that on the ground at least here in Malaysia, multiple PCR tests are conducted over an extended period of time to rule out false results.
A person is not considered Covid19 positive until all tests are out to rule out false positive or negative results.
I expect similar practice in most other places
Excellent explanation. I coded a visualiser for False Positives a couple of months ago if anyone wants to try with different Sensitivity, Specificity and Prevalence values.
https://hotbeer.com.au/pcr-false-positive-calculator.php
Sorry for the ugly interface, my partner normally does all the styling so this is my poor effort.
Great article. Explains why more testing interests those who want to keep people locked up and scared.
Australia now has very low positive test numbers following a much smaller “wave” than affected US and Europe – suggests our specificity is within a bees whisker of 100% rather 94%. (my lab quotes 100% fwiw) Good explanation of the complexities of medical test interpretation though – Thankyou.
I would be interested to know how many cycles are performed in the PCR test in your lab?
Thank you. Since asking the question, I have found an official Australian government document that states that over a CT of 24 the virus is not recoverable. That indicates that Australian testing is very highly specific, which explains our very low case numbers.
Those countries doing a CT of 40 really need to get properly educated !
Sebastian on Thursday Friday in Australia, the state of NSW conducted a total of about 35,000 tests.
Results were about eight new infections. Our figures don’t seem to match your predictions. Care to comment?
Hi John,
As Samuel posted above, it seems Australia is using a much lower CT than many other countries, which would result in a much better specificity.
Agree. But most patients do get tested within a week of having symptoms and so epidemiologically it gives some idea of prevalence of infection. I think everyone is looking at all parameters and the social behaviour dynamics of each society and their vulnerability to serious covid is worth looking at too
Excellent article – very clear and shows the scale of the problem.
I have created a quick and dirty Google sheet to do some of the calculations. You can play around with the Key Inputs and see the results in the tables below.
https://docs.google.com/spreadsheets/d/1nhFY_RxIwjBNE5pJlZ5ixi5DcDlvkG9NrSYkgcbS9Mk/edit?usp=sharing
Hi Sebastian,
nice work. It is always a pleasure to read your blog.
Regarding PCR tests and specificity: Does this systematic review show sth. like the difference between theory and practice?
Following the evidence distributed by the RKI in the media (at least in germany) anyone who doubts that the specificity is below 99,X % is “naive” (twitter speak: “stupid as hell”).
The claim is that the current PCR tests are testing for at least two primers and therefore one single pcr test has (nearly) the accuary of two independent PCR tests. This mostly ends up in a specificity of about 99,X %.
Any thoughts on this?
Best Regards
Sebastian
Hi,
Thanks. No, it doesn’t look in to whether there is a difference between the tests in theory and practice. In general, most things perform worse in the real world than they do in studies.
“…why we should instead be basing decisions on the rates of hospitalization, ICU admission, and death….”
That would be a good idea, but how do we know if they dont use the same PCR tests and get the same results ? As far as i know the people in icu in the hospital near where i live are tested with the PCR test no matter if they have symptoms or not.
Thanks for your great article and explanations.
I think they are very well educated, and commit this fraud on purpose.
Yet another excellent article. Thank you so much for your good and valuable work.
I wanted to share this information from Iceland, that people might be interested in. You can see the number of tests done per day at the border, as well as the number of positive test results. It’s interesting to see that they were often running over 1000 tests with zero positive test results, suggesting that the specificity is very high (possibly 99.9% if you look at a few days of border screening). The data has been reported daily since August. There were also tests done domestically, but these tests were mostly on symptomatic people or those who had been exposed to positively diagnosed individuals, which I imagine would reduce the false positives / (true positives + false positives) ratio.
From what I can gather, here in Iceland we use so-called RT-PCR tests that use RNA as template rather than double-stranded DNA used by PCR. I haven’t studied the differences in detail, but I will also link to a study that compares them. I think that most countries are not using RT-PCR.
Data: https://www.covid.is/announcements
PCR vs RT-PCR: https://www.researchgate.net/publication/318305886_Difference_Between_PCR_and_RT_PCR#:~:text=PCR%3A%20DNA%20polymerase%20is%20used,as%20enzymes%20in%20RT%2DPCR.&text=PCR%3A%20Forward%20and%20reverse%20primers%20are%20used%20in%20the%20PCR.
Hi Erling,
When I write PCR, I mean RT-PCR. All PCR tests for covid use RT-PCR, because covid is an RNA virus. It’s genome is coded in RNA, not DNA, so regular PCR isn’t possible. Do you know what CT they were using for border screening?
Hello, Dr. Rushworth. I would like to translate your post into Italian. It is for publication on an Italian facebook site called “Pillole di Ottimismo” https://www.facebook.com/pillolediottimismo. I hope it is ok with you and congratulations for the good work you are doing!
Ugo
Thanks Ugo,
Sure, I’d be happy for you to translate it.
Wow. Thank you for methodically walking the reader through this. The PCR information is especially helpful. I have a much better understanding now.
Question: Given that the European Union (and many other nations) showed test positivity rates at or below 2% or 3% for much of last spring (May-July, for instance) would it be fair to assume the the actual PCR specificity is perhaps just a bit higher than your (suggested) 94%? If your assumption of 94% was correct, then we would never see positivity rates below 6%. Is my thinking on this correct? Thank you again.
Hi John,
Yes, that is correct. The real world testing in parts of the world suggests a better specificity than has been found in the studies. It is strange, and I don’t know why that appears to be the case. That’s why I also included calculations based on 99% specificity.
Dear Dr Samuel Zagarella
” I have found an official Australian government document that states that over a CT of 24 the virus is not recoverable.” Would you be able to share the reference please? That is very valuable.
A very nice, because is a very simple lesson.
Thanks for showing it.
Thanks, Sebastian. It is very simple to understand because of your effort to explain something complex in a simple way. It amazes me that the test result is so confused with the diagnosis of the disease. The bottom line is that we are experiencing a CRP pandemic.
HI Kora,
Sure, here it is. I am assuming Australian testing follows these recommendations.
https://www.health.gov.au/sites/default/files/documents/2020/08/cdna-and-phln-joint-statement-revised-australian-criteria-for-the-release-of-persons-recovered-from-covid-19-from-isolation-attachment—summary-of-evidence.pdf
Thanks to you!
Where did you find that official document? It would be very interesting to share it with others.
Sebastian
Going of what you have written it seems reasonable to think that many are using a lower CT cycle number , Yes? Also my understanding is that in New Zealand anybody who tests positive is tested again and if that second test is negative a third test is done , which would tend to reduce false alarms.
Hi John,
Yes, that would be my guess, that countries are in fact using a lower cycle number cutoff, or in some other way decreasing the risk of false positives further than would otherwise be the case.
I’m wondering if the extremely sharp rise in infections across Europe is brought about by the onset of the fall/winter rise seasonal viruses and everyone with any symptoms is rushing to get tested – this may mean that many more folks with nearly asymptomatic infections are being tested. This may mean our testing is more efficient and closer to seeing the true prevalence as well as being buoyed by the seasonality of corona viruses. One thing that is interesting is the rise of deaths from the virus at rates that seem steeper than cases and with much shorter delays between cases and deaths – a new strain circulating or lack of care in overwhelmed health systems or poor statistics?
It’s the link I shared above, and specifically the reference I was referring to from that article was this one:
Bullard J, Dust K, Funk D, Strong J, Alexander D et al. Predicting infectious SARS-CoV-2 from diagnostic samples,
Clinical Infectious Diseases, ciaa638, https://doi.org/10.1093/cid/ciaa638 (accessed 18 June 2020
Thanks Sebastian for another brilliant article.
”For SARS-COV-2 there is no technical specification developed” (for the PCR-test), that is a direct quote from FHM’s home page. FHM go on to state that this is according to the policy of EU Kommissions, and thus the same in all EU countries. Further down in the text it says that instead of a technical specification for SARS-COV-2, one can use samples from other deseases and modify them (whatever that means).
That sounds a lot like what I heard from other sources, that there is still no Koch’s Postulates for Covid-19. So the question becomes, how do we then know that this present increase in the ”Cases”, are not the ordinary Cold and Flu virus, which always increases in the Autumn? Especially since all the usual statistics from the Cold and Flu has disappeared, it’s gone, now everything is ”Covid Cases”, or more accurate ”Covid-19 m.m”.
If I understand it correctly, a prevalence of a RNA fragment which then is used and enhanced in a PCR test, does NOT mean that you’re sick or even infected, it only means that you had a RNA virus (dead or alive) in your throat. So, if your immune system works perfectly well and kill, or going to kill the virus, it might still show up as positive in the PCR test, depending upon how many cykles has been used, which then makes the question about how many cycles are really used, very interesting and very important.
A research study made in England May 2020 shows in ”Table 2” a graph, where on the Y-axis you have ”Culture-Positive” (gold standard from the lab), and on the X-axis you have Ct-value (or cycles). If you then look at 35 cycles on the X-axis, then go to the PCR reference line, proceed to the left and you get about 10% Culture-Positive on the Y-axis, which gives about 90% false positives. I understand that there is more things considered in real life, but it’s strange that in Sweden we use 45 cycles as standard, why?
If 24 cycles should be the top for accurate tests, why use anything above?
Madrid sees a relaxation at the moment – is it to do with changed testing strategy?
https://www.tellerreport.com/news/2020-11-08-this-is-how-the-covid–miracle–was-created-in-madrid–antigen-testing–restrictions-and-%22individual-responsibility%22.ByGEh7yHYv.html
I wonder if antigen testing used perhaps in conjuncture with PCR, to establish whether an active infection is going on AND whether it’s due to SARS-CoV-2, would lead to more reliable results?
I see. Thank you for explaining that to me.
Sadly, no, I don’t know what CT they were using. If I ever find out, I’ll be sure to let you know.
Unrelated, but not complete. I know they were taking samples from both the throat and nose. I hear that can lead to more accurate results than taking only one of those. Not sure if they then mix the samples or run two tests, one for each.
Hi,
can you find out when the infection was based on the results?
How many cycles do you need to say 1, 3 or 5 months ago the person was infected?
Thanks.
Hi Holger,
The answer is both yes and no. If you have 20 cycles or less, you most likely have a currently active infection. If you have 35 cycles or more, you could either have an infection that is ramping up or an infection that is in the past. Some people continue to be PCR positive for weeks or even months after infection, but in these situations the number of cycles is invariably high.
Will appear tomorrow morning!
Reporting of deaths is based on ‘deaths within 28 days of a positive COVID test’, so if the test results are inflated, won’t the death stats be inflated as well? So how can we find out what is really happening?
Hi Linda,
True, which is why the most important statistic is overall mortality. That is the one number that doesn’t lie.
That’s strange. I found that also in Italy we use 45 cycles. Do you have a reference to the paper you are citing?
Sweden’s death count per nov 6: https://www.statista.com/statistics/525353/sweden-number-of-deaths/
Overall mortality of course doesn’t tell what people have died from. But I’d like to know how far the stress on hospitals and ICU capacities in this season exceeds that of other years?
I’d like to know that too. I haven’t seen the Swedish data, but I have seen the UK data, and they are in line with how they normally are this time of year. Covid is not currently putting any extra stress on the system in the UK beyond what would normally be seen in the autumn respiratory virus season.
https://www.youtube.com/watch?v=yPG8d3e43Xk
Does Liverpool show the same “wonder” as Madrid? First they had great numbers of positive PCR tests. Then they did antigen mass testing with very few positives, showing that there is no wave of active infections going on.
Your post is on line in Italian at https://www.facebook.com/pillolediottimismo/posts/200895161650963 — after an hour, we had already 26,000 visualizations. Not bad, congrats!
UB
Wow, that’s great!
Multiple people have argued that specificity is a lot higher than your article assumes, and I can add another data point: in Italy this summer the number of positive results went below 0.5%, indicating that specificity was higher than 99.5%. You article spends a lot of time on a hypothetical case of 94%, and then a brief paragraph with a “very high specificity of 99%”, which is instead too low. As a consequence, it gives the reader a really misleading impression of what’s going on.
Maybe you could post an update using a lower bound of 99.5%? Numbers will come out quite differently.
The 94% is based on the average from the study discussed. A separate review found an average specificity of 96% . I can’t explain why the real world numbers don’t match the studies, normally real world results are worse than studies, not better. Even with a specificity of 99,5% you will quickly get a high proportion of false positives as the disease becomes rarer or you test people in which the probability of having covid is low.
Hey, very cool article! I was also trying to take a closer look to the PCR technique, but I stumbled across one thing I still wasn’t able to figure out myself. Perhaps I can find an answer to my question here.
The traditional PCR technique (as far as I understand) is based on a fixed number of cycles before an analysis is performed with so-called gel electrophoresis. To my understanding in addition to a fluorescent signal also the _weight/length_ of the amplificates is an observable to check whether the correct sequence was amplified. This was done by comparing the way length in the gel to some reference ladder, where the travelled distance is somewhat anti-proportional to the weight.
Taking a comparative look to the real-time qPCR I didn’t find the information yet whether or not (besides the, again, fluorescent signal) this weight/length is also measured there. If not, then the specificity should even be worse in the qPCR. Does anybody know this?
Thanks for any reply! 🙂
As far as I know, in Italy a standard of 20-24 test cycles was used in Summer, it was later switched to 45 in late September. These numbers are not official. Some say the standard is now 34, others that it is 50. Unfortunately, nobody seems to know what is being done in the labs. In any case, the difference in the number of cycles may explain why some testing campaigns in Italy reported zero positives this summer. But things seem to have changed in a way that makes Sebastian’s calculations close to reality
I think the main take-away message is that you’ll keep a pedestal ot new infections as long as you don’t know exactly the specificy/false-positives _and_ correct for it. In particular (to my knowledge at least in Germany) the tests are neither calibrated nor standardized, so you would need to determine the operational precision of the tests for each individual lab on a regular basis. Nothing of that happens as far as I know.
So the bottom line is that the tests are useless IN REALITY. In a true pandemic the prevalence of disease quickly overruns the capacity to test and verify (testing lose its meaning) and later as the disease fades out the tests produce an overly large proportion of false positives. Yet we are using these tests and governments and media are shouting about “cases”, almost as if it’s a contest.
Brilliantly explained – infact now you put it that way I’d say this test is perfect for creating fear and panic and falsifying a pandemic or new wave to shut the economy and its people down – so where you see this test being used prolifically you could assume that is the case and the real intentions are not noble. If you work to tell those in authority of the downfall of PCR and they listen then may be not BUT if they don’t listen then definitely yes they are not acting with integrity.
Dr. Rushworth, I used the information that you provided in this article to model and estimate COVID prevalence in my home state. For the week ending 11/17 there were 318,948 PCR tests administered and 43,488 positive results. Assuming 88% sensitivity and 94% specificity, that testing volume would result in that level of positive tests at ~ 9% prevalence. Approximately 14k would be identified as + in error. Am I on track with that? Thanks
Hi Rick,
That seems reasonable, but as others have pointed out in the discussion here, the real world specificity seems to be significantly higher.
What spec would you suggest?
I think it probably varies from location to location. I hope countries have been using the same CT throughout the pandemic and not been varying it up and down, because if that is the case then case statistics are even more useless than they seemed to be before. Assuming the same CT has been used throughout, and since we know many countries had 0,5% positive tests (or in some cases even lower) during summer, it seems a specificity of 99,5% is reasonable. But that assumes labs haven’t been changing the CT at different points. If they have, then the specificity will also have varied up and down a lot.
Thank you
Hello Dr, any thoughts on this write-up?
https://virologydownunder.com/the-false-positive-pcr-problem-is-not-a-problem/
Hi Terry, the author seems to know a lot about PCR, but he is missing the point completely. Every test has a sensitivity and a specificity, and that includes PCR. And in every test you have to choose whether to prioritize sensitivity or specificity. The higher your CT, the more you are prioritizing sensitivity over specificity, which will push up sensitivity and push down specificity.
The big question right now is, what is the specificity of the test? As far as I can tell, he is not able to provide an answer to that question.
People can continue to be PCR positive for up to three months after infection, and PCR can detect traces of virus that are present in the airway but that are not sufficient to cause infection, and there is always a risk of laboratory contamination, so there is always a scope for false positives, and that rate could vary hugely between countries depending on disease prevalence, what CT is used, and how good the labs are.
Sorry Sebastian, you don’t grasp this problem. There is a specific issue going one with PCR because it can have two separate mechanisms of false-positivity, depending on the question that you are asking.
1.There is an issue of the PCR detecting RNA when there is simply no RNA. Because of the design of the technique, this generally should only occur if there was some kind of contamination of a sample. This is what Mackay is alluding to: in populations where there is no virus circulating, almost no tests come back positive. So we can basically say that this form of false positivity is very uncommon, say <0,1%. So assuming samples are handled appropriately, specificity here is very high.
2. Then there is the issue of detecting RNA of old virus, which is something that we know about from any other use of PCR, or also a problem with TB diagnostics (staining): detecting DNA/RNA or detecting a positive stain does not mean live viable virus. In general clinical practice, we put this together with clinical impression to make a working diagnosis, frequently pending confirmation.
So we have a specificity for 'active' infection. However, a positive PCR still basically means you at sometime in the past carried SARS-COV2, and likely more recently (past month) than before: some studies have looked at this, and decline in PCR positivity can make it so that 33% are still positive after 3 weeks and 10% after 1 week for people who had a mild infection. This lingers on longer for patients with severe infection.
Now if you randomly swab the population, you will probably 'catch' a lot of the past infections. However, this does not invalidate serial swabbing of the population to infer trends in virus prevalence- it just means that we can't be completely sure about what exact % the virus prevalence is. If the prevalence of serial swabbing schemes show doubling of positivity, we can be confident that the doubling is not caused by merely accumulation of past infections. If this doubles again, then we are even more confident about this. If average CT value goes lower, we are even more confident. That is what the serial swabbing scheme in the UK found, swabbing representative rounds of the population in subsequent months.
https://www.medrxiv.org/content/10.1101/2020.09.11.20192492v2.full.pdf
What this also means is that, the specificity of this 'false positivity mechanism' (it is only false positive if you are looking for active infection, it is likely a true positive if you are looking for recent infection) is dynamic and contingent on prevalence trends: in the setting of increasing prevalence, specificity goes up, in the setting of declining prevalence, specificity goes down.
Now maybe you'd think we should use viral culture as a gold standard. But that's also problematic: you can imagine that culturability of virus from nose and throat swabs would diminish as you mount a mucosal immunity response. This is actually what we see: culture positivity in hospitalised patients (in which we would tend as Bayesians to believe a + PCR) is lower than in symptomatic community patients.
https://www.medrxiv.org/content/10.1101/2020.10.13.20211524v2 (compare to graphs in supplementary files)
Hi Jaime, thanks for the interesting and thoughtful comments. I’m not sure I understand exactly what point you’re trying to make though…?
Your main argument is that PCR positive cases are not a good marker and this is based on (wrong) static estimates of specificity. The above sources show that specificities must be much lower that 99%. When we look at amount of reported cases, we generally look at amount of cases but also the test positivity: this basically means that SARS-COV2 is not only causing more resp symptoms, but also a higher share of them in the population. We can therefore reasonably assume that true prevalence is increasing, but we cannot know the exact prevalence. As a safeguard, we may do unbiased surveillance samples. Your counter: use hospitalisations, is also not the exact prevalence from hospitalised cases either: this reflects infections in higher risk groups (the elderly) and we know transmission can be much higher in younger risk groups.
Hey Dr,
I guess my confusion is — if Australia is doing tens of thousands of tests per day and they have zero cases, this would seem to mean they’re getting zero false positives?
Yes it would, which would be pretty amazing, because it would mean the real world specificity is several factors higher than the specificity seen in the studies.
Jaime Borjas writes above “However, a positive PCR still basically means you at sometime in the past carried SARS-COV2, and likely more recently (past month) than before: some studies have looked at this, and decline in PCR positivity can make it so that 33% are still positive after 3 weeks and 10% after 1 week for people who had a mild infection. This lingers on longer for patients with severe infection.”
The consequence of this is that many people who are not infectious are quarantined, their family and contacts as well, schools are closed down etc pp and countries may not come out of this vicious cycle. That’s why we need a smarter strategy of testing, and also of handling test results with regard to restrictions mandated for the affected and for the whole society.
I’m very curious about what can lead us out of this dilemma.
Kora, that’s why in general we ask people to get tested *quickly* once they have *symptoms* AND not to retest if they recently had a confirmed infection. This of course conveniently aligns with prerequisites of effective contact tracing: if you get tested late, it will be difficult to trace back your contacts.
If you do a random PCR swab of an asymptomatic person in the population, it would be reasonable to assume that in about 50% of the positive samples, we are looking at a non-recent infection. So when we do this now in England, about 1% comes back positive. So about 0,5% would be expected to be old infections. However, test positivity in test centres is 7% https://www.gov.uk/government/publications/national-covid-19-surveillance-reports. We can therefore infer that 0,5%/7% are ‘false positives’, which means just less than 10% are false positive in the sense of non-recent infections.
The state has not been forthcoming regarding CT. Late this summer a reporter noted that it was 38. Someone posted this morning that it had been increased to 40-44 but I have no idea if they are a credible source. I inquired of the reporter who originally wrote the story regarding CT but unfortunately he is less than curious about this. Am I correct in assuming that a higher CT would result in a lower % specificity?
Hello Jaime,
two thoughts:
– looking at how a wave of infections seems to travel through the countries during the recent weeks (Spain was affected harder first than it is now, while Sweden might be still on the rise), how would we be pretty sure that recent infections are about half of all positives? I think the percentage of no-longer-infectious cases could be much higher, the longer the time span since the first cases having appeared. If a PCR test shows positive for perhaps 8 or 10 weeks altogether, but infectiousness is given in only about 8-10 days, I’d assume that only a seventh part of the positive test results would belong to actually infectious cases, 2 months after the wave’s onset (depending on the cut-off of the used PCR test).
– given the relative uncharacteristic symptom profile, how can we be pretty sure that the symptoms of a patient tested positive for SARS-CoV-2 actually stem from that virus? Couldn’t they have been in contact with the Coronavirus in september and remained asymptomatic, and now have attracted (say) a rhinovirus that is not tested for, or bacteria?
Which way of testing can discriminate these?
First of all: we can deduce that PCR positive wanes over time by looking at random swabbing schemes: % went down after lockdown, to very low levels (99% of the positives will eventually turn negative. However, in a declining epidemic, we can expect a higher proportion of people from random swabbing schemes to be non-recent infections, as prevalence was higher in the period before than afterwards. So indeed, test % may lag and decline slower than true prevalence. However, I don’t see a scenario where this would problematize positive predictive value in the clinic: if prevalence was 1% last week but 0,8% now, and we still see test positivity of 5%, PPV doesn’t drop dramatically.
The problems with SARS-COV 2 PCR should also be a problem with competing PCRs, so principally that concern is asymmetric. A lot of sentinel swabbing schemes actually screen for competing viruses, so you can see how many viruses actually compete at this time. For the UK, this only important virus is rhino virus. Let’s say you would always test for both, how would you decide that there’s only a rhinovirus infection going on? Perhaps if the CT value is really high, but this could be due to early infection. So probably you would retest in a day or so.
Thanks for your answer Jaime!
I’m taking the viewpoint of the tested – and quarantined – citizen and of those who can’t go to their gym, who are out of job etc because of the hunt for SARS-CoV-2.
In disease and infection we can have a cocktail of contributing factors, and I think it would be more meaningful to find out which ones are making people more vulnerable and to aim to address those in particular. So yes, when testing we would need to look at the whole microbiome and immune reactions influencing it.
Are there statistics comparing the typical hospitalisations and mortality in november (e.g. in Sweden) with seasonal influenza, corona- and rhinoviruses, with what we see now? The latest comparison I saw ended in the beginning of november and was unremarkable. https://emanuelkarlsten.se/number-of-deaths-in-sweden-during-the-pandemic-compared-to-previous-years-mortality/
Sorry if I’m getting off-topic from the main theme of this article.
Kora, now you are conflating two things: the performance of the test versus asking people to isolate and what to do about it. That latter is political. Now, if the benefits if a person isolating are collective, then it follows politically that costs of that should be collectivized. Beyond the political, it incentivises adherence to actually getting a test and self-isolating if asked to do so, using (imho preferably) carrots and as little stick as possible (knowing you will get fined disincentivizes getting tested). It seems that most governments just use the stick (fines). Ergo: we should actively support those who we ask to self-isolate/quarantine in the broadest sense of the word. Irrespective of the false positive rate! For any purposes, I think a lot of policy response surrounding lockdown here is backwards- in the UK, during lockdown everyone was ‘furloughed’, but if you have to stay at home because of TTI outside of lockdown, no universal income support. Beyond unfair, it’s also just not smart policy in turns of cost-effectiveness: furloughing whole swaths of the population is much more expensive than only specifically the infected and their contacts.
Cut the gordian knot.
How about we toss all non-therapeutic interventions? Then we can toss testing for public health policy and just use high Ct testing for infection control in hospitals and nothing else.
We have antiviral therapeutic interventions for community cases to limit hospital overloading.
My thoughts exactly – the super rapid growth in cases in the second waves made me think that folks are going forward for testing in much larger numbers due to symptoms from other seasonal viruses. These symptoms from other viruses may be bringing in those who had corona virus infections a long time ago and still test positive. How long before that reservoir of folks with inactive infections dries up? Perhaps when the first wave of other autumn viruses wanes so those folks no longer come forward for testing???
There is a problem with the # of deaths attributed to COVID too! Hospitals are paid 20% more $ from medicare for patients who have died from COVID than from other causes. Do you not see a problem with that? Example: Patient is terminal because of lung cancer. Patient dies. Patient is swabbed for COVID and PCR test is run – comes back positive. COVID is put down as cause of death. Without this BRIBE from medicare, the hospital would have properly reported the cause of death as cancer.
Procedures that hospitals normally perform are way down. Many hospitals are probably financially strapped because of this. They will absolutely take this extra $ to keep their hospital from going under.
Only about 4- 6% of reported COVID deaths in the US are from COVID alone.
Here’s an article where the author does an analysis of deaths during the so-called COVID pandemic. The # of COVID deaths was greatly inflated by misrepresenting several other categories of death to COVID – pretty much 0 excess deaths (linked to John Hopkins University):
https://web.archive.org/web/20201126223119/https://www.jhunewsletter.com/article/2020/11/a-closer-look-at-u-s-deaths-due-to-covid-19
In past years, we have had 60 – 100,000 deaths in the US during a flu season and nobody even noticed. What changed in the 2019-2020 flu season? (It had nothing to do with getting rid of PDJT, right doctors? Do a study on that.)
Doctor, isn’t it true that the COVID 19 virus has never been isolated or purified , and has never met Koch’s postulates as an infectious agent? The response of a positive PCR test is negligible to moot as Dr Kary Mullis, Nobel prize winning scientist who discovered PCR never meant it for diagnostic use, just for research only. The cycles, or amplifications will result in 100% positive of ANYTHING YOU ARE LOOKING FOR- even if it doesn’t exist. There are between 200 and 1100 Corona viruses that can turn any PCR test positive. scientists admit that the RNA particles that might be COVID are essentially manufactured and modified artificially to produce a full strand that could might (falsely) be recognized. In fact,since there is NO identified COVID virus, no test can find it, no antibodies can be found from it, and no vaccine can eradicate it since it doesn’t exist.
Rick,
What data did you use to determine a 9% prevalence in your calculation? Since Influenza and Covid-19 share the same symptoms, it seems to me that prevalence should be determined by using State Health Departments ILI (Influenza Like Illness) reports; not an arbitrary PCR test. Sensitivity and Specificity percentage of a developer’s assay stays basically the same, but its PPV and NPV changes with Prevalence. Massive testing during low prevalence will produce very high false positive results,
A Portuguese court recently ruled PCR-tests unreliable regarding their use in testing for SARS-Cov-2.
https://off-guardian.org/2020/11/20/portuguese-court-rules-pcr-tests-unreliable-quarantines-unlawful/
A review of the Corman-Drosten paper. Quite stunning if correct.
https://cormandrostenreview.com/report/
First of all, thanks alot for the work and the help in getting some light into this international desaster!
I am from Germany and I am watching this whole covid drama with an increasingly bad feeling, as here in Germany the crisis is used to alter the constitution and take away basis human rights, all based on dubious numbers and and a PCR test that is known to have false positives, and the German government even openly admits they have no clue about the false-positive rates…
Anyhow, the German media is censoring any ciritcal information and they only sometimes have a quick view towards Sweden with gonzo headlines like “Second wave hits Sweden badly”. Checking the fatalities for Sweden on https://www.worldometers.info/coronavirus/country/sweden/ I can’t really agree to that. Yes, fatalities are higher than in summer, but that’s to be expected with a respiratorial desease in winter, right? And I would assume it’s not much different than other years (assuming the cause of death is documented correctly, which in many cases around the world does NOT happen).
Well, so I assume they refer to the dubious PCR test results and the so generated “cases”. More and more scientists start to criticize the Drosten PCR test procedure (this is interesting: https://cormandrostenreview.com/report/), of course the German media sweeps this under the table.
I think for many people Sweden is the light in the dark, and the “Corona’s Witnesses” desperately want you guys to fail. Because if you don’t act with strict lockdowns and masks and stuff like that and don’t fail, you are very dangerous to the other governments’ narratives.
So, my main question is, if the cases are rising in Sweden but it’s still no more danger to be expected than a severe flu and fatalities don’t explode – why do you still join the game of the PCR generated “cases” and use the dubious Drosten PCR test? Right now, it seems that’s what everybody is looking at…
Thank you very much, best regards from Germany,
Kai
Hi Kai,
I can’t answer for what is going through the minds of the Swedish government, but they seem to have panicked more now than they were in spring, even though if you look at the numbers the situation was clearly much worse in spring, and they have been pushing heavily for increased PCR testing.
hi Sebastian,
thank you very much for the quick reply and the confirmation that the “medical situation” is not worse than it was in spring.
Though it seems the “political situation” is getting worse … and there are enough theories about that out there (though none of them hopeful).
Thanks again and best regards,
Kai
Whilst the math is all good and well, sorry Seb, but it just doesn’t “add up”.
The main point I was going to make was one that was already made by Ian Mackay here: https://virologydownunder.com/the-false-positive-pcr-problem-is-not-a-problem/
Your claim that there is going to be “all these false positives going around”, especially as the disease becomes rarer doesn’t seem to come to fruition, as Ian shows by Australia’s testing results.
For example, over the last 3 days there has been over 80k tests done and 69 positives results, doing some math this worked out to ~85 positive results per 100k. Now, using your math from above if 10 people truly has Covid it allegedly would correctly pickup ~9 of these. That leaves 99,990 that DON’T have Covid and according to you claims we should be expecting 5994 false positives!! But looking at the data the REAL figure is nowhere NEAR that. Out of the 85 that tested positive you can play around with the actual number of people that truly have covid, but it doesn’t really change the results much as you are still left with a huge number remaining that is going to generate an “apparent” huge number of “false positives”.
*** Continued in next post as impossible to make a long post as the submit section gets pushed down and vanishes **
** Can”t see the first part of my post I submitted, I guess it’s in moderation, so here is the second part **
Also, it appears you have contradicted yourself in another story – with your claims here you seem to be stating that there is a MASS amount of “false positives” out there, which thus must then mean that the true death rate is FAR higher than is reported (we are counting too many “false” cases in our math), but then you claim in your other article that the death rate is “really low” and go on to compare it to the “flu” (very poor form for someone in the health industry):
https://sebastianrushworth.com/2020/10/24/how-deadly-is-covid-19/
Not only are your claims in this article based on fallacies (the WHO report was based on serology tests which are known to be unreliable, and ironically DO give a lot of false positives for other coronaviruses) but it contradicts your claims in this article.
This same point is presented here: https://twitter.com/KHInnovation/status/1329891784927965184?s=20
Which is it Dr. – are there too many false positives or is it not as deadly as we think?
As other commenters have noted, the real world specificity appears to be much better than the specificity in the studies. This article is based on the specificity in the studies. Normally studies show better results than the real world- why the real world in this case shows much better results than the studies, I can’t say. As to your second point, the serology tests have a much higher rate of false negatives than false positives, so if anything, serology should understate infection, not overstate it.
Same has happened in Netherlands and Germany in September. CT cycles have been increased dramatically, purpose of this is allegedly “not to miss any infections” next ot that in Netherlands they stopped testing on 3 genomes and now test only one in many places and the reason for this is to increase testing capacity.
As a result they started testing everyone, so also without symptoms, from early december. So you can predict what the curves will do quite easy.
I asked my wife about PCR specificity, and she quickly googled and quoted a figure of 95% to me, which immediately flagged my BS detector because that would mean 1 in 20 tests returning a false positive, yet we just had around 37,000 tests performed today in our state with only a single positive, indicating a specificity of 99.997% or better. Now, the 95% figure was not from our country, so I don’t necessarily know where that figure came from, but after discussing this with my wife and reading wikipedia it became apparent that the specificity figure quoted might (and I stress might) be accounting for false positives due to sample/equipment contamination (among other factors).
If that is the case (and I’m not sure that it is) it would mean that the specificity figure would be a function of the prevalence of the virus, i.e. if there are many positive samples going through a lab due to a widespread pandemic in the area there is a higher chance of a contamination occurring leading to a false positive, so a lower specificity. However, in areas where the virus is almost stamped out there will be hardly any or no virus in the lab in the first place to be able to contaminate any samples with, naturally leading to a much higher specificity in areas where the pandemic is under control.
But anyway, I’m not sure if any of that is right or not – If contamination is not a major factor in quoting the (IMO) rather low 95% specificity for something that is supposed to be testing for a specific strand of RNA (which I would have naively expected to be a much higher specificity, more in line with the numbers I’m actually seeing in Australia)… what is?
You are absolutely right. I have also thought about this, and I think it largely explains the extremely low rate of positives in places like Australia where the prevalence of covid has been very low. As prevalence of the virus rises, false positives also rise to an extent, as the scope for contamination in the lab increases exponentially, and also as the probability of finding bits of virus in the respiratory tracts of people who aren’t infected rises exponentially. So strangely enough the highest false positive rate might not be seen when the real prevalence is super-low, but rather somewhere in the middle.
Ian do you know what ct-value was used when obtaining this information? When the value is low, I’d expect a lesser number of positives, right?
This is interesting: if I understand this correctly, the testing with PCR is now to be made so that positive results should correspond to Covid symptoms ( = “clinical presentation”)!
Quote:” Users of IVDs must read and follow the IFU carefully to determine if manual adjustment of the PCR positivity threshold is recommended by the manufacturer.
WHO guidance Diagnostic testing for SARS-CoV-2 states that careful interpretation of weak positive results is needed (1). The cycle threshold (Ct) needed to detect virus is inversely proportional to the patient’s viral load. Where test results do not correspond with the clinical presentation, a new specimen should be taken and retested using the same or different NAT technology.
WHO reminds IVD users that disease prevalence alters the predictive value of test results; as disease prevalence decreases, the risk of false positive increases (2). This means that the probability that a person who has a positive result (SARS-CoV-2 detected) is truly infected with SARS-CoV-2 decreases as prevalence decreases, irrespective of the claimed specificity. ” https://www.who.int/news/item/20-01-2021-who-information-notice-for-ivd-users-2020-05
Might that mean that now we’re not hunting for “asymptomatic carriers” any longer??
Might that mean that we now are ready to get the curves down?
Hi Sebastian!
Thank you for your articles and your insightful explanations.
I’ve been wondering about the Swedish dominant opinion and policy about the covid false positives, but haven’t found much/any information about it. Do you know what has been done about it over there?
Do you only/mostly test symptomatic people, did you reduce pcr cicles, …? Do you use more PCR or Antigen tests? Any place where we can find information about that for the different countries?
Thank you and best wishes,
Teresa
Hi Teresa,
We’re not using PCR for screening in Sweden, pretty much only to test symptomatic people (although people are able to book tests themselves so it’s possible some people who are asymptomatic are booking tests too, although that’s not the intention). We’re not using lateral flow testing, just pcr to discover active infection and antibody testing to discover past infection. As in other countries, no-one really knows what the false positive rate is.
I understand the problems of false-negative/positives, but I don’t see how this translates to the real world.
Regarding the problem of false positives – If the tests have a significant percent of false positives (as you stated), how does this fit with the low ratio of positive results in many countries (such UK, Australia, Iceland, Norway, Portugal, etc.) with positives of 5% positives as an indicator for indicate testing). I would also expect the percentage of positive cases in those tested is higher than that of the population, as a whole.
What percentage would you expect to be false-positives?
How bad do you think the problem is?
Thanks for your courage and interesting articles Sebastian. If I would make a wish it would be an article on whether the SARS-Cov-2-virus has been properly “isolated” or not. And if not, how does this effect the ability to uniquely identify the virus in a patient. There seems to be a lot of confusion surrounding this on the internet. A popular rumor is that requests have been made for documented evidence under the “Freedom of information” regulations but that the answer from government officials has been that they have not been able to locate such documents.
All the best,
Hi, since you wrote this about pcr tests the wawes of covid has come and gone over and over. What are you’re thoughts about the tests now with Omikron spreeding? Is it of any use? Why are we still not able to live our lives normally!
Hi,
Does Positive Predictive Value and Negative Predictive Value encompass the issues of sensitivity and specificity as the disease prevalence varies? In light of this article what do you think about the PCR tests now during omicron variant?