I’m very interested in how doctors think. How do we use the information gained from talking to and examining a patient to reach a reasonable list of likely diagnoses (a so called “differential”)? When we order a test, what specifically are we looking for, and how will we react to the result that comes back? More cynically, I’m curious about the extent to which we understand what the test result actually means. And what are the odds that we will make a correct decision based on the answer we get back?
I think that anyone who has even a partial understanding of what doctors do understands that the practice of medicine, although based on scientific knowledge, isn’t a science. Rather it is an art form. And as with all art forms, there are those who excel, and those who plod along, occassionally producing something nice or useful. Most people are probably aware of the fact that if you go to five different doctors with a problem, there is a significant probability that you will get five different answers. Medicine is so complex, with so many different variables to consider, and doctors themselves are so varied in terms of how they think and what they know, that the end result of any one consultation will often vary wildly.
One of the things that always needs to be estimated in any individual consultation is probability. What is the probability that the breast lump is cancer? What is the probability that the fever is due to a serious bacterial infection? When faced with these questions, I think most doctors are more like an experienced chess player than a robot. They act on a feeling, not on a conscious weighing of probabilities. Doctors with a nervous disposition therefore order more tests and prescribe more antibiotics, while those with a more relaxed disposition order fewer tests and prescribe fewer antibiotics.
But how good is the average doctor?
That is what a study recently published in JAMA Internal Medicine sought to find out. The study was conducted in the United States, and funded by the National Institutes of Health. 492 physicians working in primary care in different parts of the United States filled in a survey, in which they had to estimate the probability of disease in four different common clinical situations, both before and after a commonly used test.
The situations were mammography for breast cancer, x-ray for pneumonia, urine culture for urinary tract infection, and cardiac stress testing for angina. For each scenario, the physicians were provided with a vignette detailing the situation and providing information on the age, gender, and underlying risk factors of the patient. Based on this they were asked to estimate the probability of disease before the test and then after the test, in both a situation where the test came back positive and one where the test came back negative. Here’s an example from the survey:
Ms. Smith, a previously healthy 35-year-old woman who smokes tobacco presents with five days of fatigue, productive cough, worsening shortness of breath, fevers to 102 degrees Fahrenheit (38.9 degrees centigrade) and decreased breath sounds in the lower right field. She has a heart rate of 105 but otherwise vital signs are normal. She has no particular preference for testing and wants your advice.
How likely is it that Ms. Smith has pneumonia based on this information? ___%
Ms. Smith’s chest X-ray is consistent with pneumonia. How likely is she to have pneumonia? ___%
Ms. Smith’s chest X-ray is negative. How likely is she to have pneumonia? ___%
The average age of the participants was 32 years, and they had been in practice for an average of three years. In other words, these were mostly young doctors who had recently graduated medical school. It is reasonable to think that they would do better on this type of test than older doctors, since what they were taught in medical school is still relatively fresh in their memories and is also more updated and correct. Additionally, medical school today emphasises probabilistic thinking and concepts like sensitivity and specificity far more than it did in the past.
So, what were the results?
In the pneumonia scenario, the doctors overestimated the pre-test probability of pneumonia by 78%. In other words they thought the likelihood that the patient had pneumonia was almost double what it actually was. Not good. Unfortunately, that was their best performance. When it came to angina, they overestimated the pre-test probability by 148%. When it came to breast cancer, they overestimated the pre-test probability by 976% (i.e. they thought it was ten times more likely than it actually was). And when it came to the urinary tract infection scenario, they overestimated the pre-test probability by 4,489%! (i.e. they thought it was 45 times more likely than it actually was).
Doh! What are doctors being taught in medical school these days?
What I think is particularly interesting here is that the error was always in the same direction – in each of the four scenarios the doctors thought that the disease was more likely than it is in reality. If this reflects real world outcomes, then that would mean that doctors probably engage in an enormous amount of overtreatment. Obviously, if you think a patient likely has a urinary tract infection, you’re going to prescribe an antibiotic. And if you think a patient likely has angina, you’re going to prescribe a nitrate. You might even refer the patient for some kind of interventional procedure.
To be fair, this study was conducted in the overly litigious United States. Doctors who know that they are likely to face lawsuits if they miss a diagnosis are probably going to overdiagnose and overtreat. But my personal experience tells me this is not just a US-based problem. I’ve seen plenty of patients here in Sweden with asymptomatic colonization of their urinary tract prescribed unnecessary antibiotics, to take just one example. I think the over-estimation has more to do with cognitive bias than with fear of litigation. Once you anchor on a diagnosis, say pneumonia in someone with a fever and a cough, you will almost certainly overestimate the probability of that diagnosis.
Let’s move on. When it comes to how much a test changes estimation of probability, the doctors overestimated the effect of a positive lung x-ray by 92%, of a mammography by 90%, and of a cardiac stress test by 804%! They were relatively on the mark, however, when it came to estimating the impact of a positive urine culture, only overestimating by 10%.
When it comes to how much a negative test changes the estimation of probability, the doctors actually did ok, being close to the mark for both the chest x-ray, urine culture, and cardiac stress test, but wildly underestimating the predictive value of a negative mammogram (in other words, they thought breast cancer was far more likely than it actually was after getting back a negative mamogram, so again, they overestimated the probability of disease).
What can we conclude from this? Doctors have a pretty poor understanding of how the tests they use influence the probability of disease, and they heavily overestimate the likelihood of disease after a positive test. They are however generally better at understanding the impact of a negative test than they are at understanding the impact of a positive test.
Finally, the survey asked the doctors to consider a hypothetical scenario in which 1 in 1,000 people has a certain disease, and estimate the probability of disease after a positive and negative result for a test with a sensitivity of 100% and a specificity of 95%. Sensitivity is the probability that a person with the disease will have a positive test result. Specificity is the probability that a person without the disease will have a negative test result.
Regular readers of this blog will have no problem figuring this out. If you test 1,000 people, you will get one true positive (since the sensitivity is 100% you will catch every single positive case) and 50 false positives (with a specificity of 95% that means five false positives per 100 people tested). The odds of any one person with a positive test actually having the disease will thus be roughly 2% (1/51). So what did the doctors answer?
The average doctor in the study thought that the odds of a person with a positive test actually having the disease was 95%. In other words, they overestimated the probability by 4,750%!
Apart from that, they thought that a person with a negative test still had a 3% probability of disease, even though the sensitivity was listed as 100% (which means that the test never fails to catch anyone with the disease). Oops. I should add that there were no meaningful differences in how correct the answers were between attendings (more senior doctors) and residents (more junior doctors).
What can we conclude?
Doctors suck at estimating the probability of common conditions in scenarios they face on a daily basis, are not able to correctly interpret the tests they use, and don’t understand even very basic diagnostic testing concepts like sensitivity and specificity. It’s kind of like a pilot not being able to read an altitude indicator. Be afraid. Be very afraid.
Medical schools should be thinking long and hard about the implications of this study. What it tells me is that medical education needs a massive overhaul, on par with the one that happened a hundred years ago after the Flexner report. We don’t send pilots up in to the air without making sure they have a complete understanding of the tools they use. Yet that is clearly what we are doing when it comes to medicine. Admittedly the practice of medicine is much more complex than flying a plane, but I don’t think that changes the fundamental point.
Gerg Gigerenzer has written extensively on this precise subject for years. I know you are a popularizer, but you really should have referenced his work. While I concur entirely with your conclusions that doctors are statistically illiterate (because I had read Gigerenzer), I am skeptical that statistical education will address that problem. Just look at the idiocy displayed by the majority of public health figures over the past 16 months. They do not want to be trained in statistics. They merely want to employ them, erroneously. I really believe most people cannot be taught statistical concepts. I trained as a physicist, and worked for an investment bank. I have over 40 years’ experience in this area. You have highlighted a malady for which there is no effective cure.
So what are the implications for the recommendation in medical school, “When you hear hoofbeats, think of horses, not zebras?” Can doctors distinguish between horses and zebras?
What are the implications for the odds of misdiagnosing covid versus misdiagnosing some other ILI?
I agree with Kevin Brown, on this topic Gerd Gigerenzer’s work is mandatory reading. In fact, the link I posted earlier was to a review of his book: Risk Savvy.
Reckoning with Risk: Learning to Live with Uncertainty (2002, published in the U.S. as Calculated Risks: How to Know When Numbers Deceive You) – is another book by Gerd Gigerenzer that helped me a lot understand risk and probability.
Some of his proposed risk presentations have been adapted in Germany, e.g. when somebody is “due” to have a mammogram.
I agree with this one. Having studied mathematics before med school, I witnessed how my fellow med students ignored to grasp most basic principles. In fact, as I am studying to become a PhD in parallel with my clinical work, this ignorance continues. Clinical researchers hire statisticians to think for them. Don’t think it’s impossible to change though..
Education in statistics and probability can improve a lot. For many teachers it is unfortunately a taboo to explain intuitively. I guess this is because intuition means simplification.
I’d like to examine a medical school curriculum. I’ve seen many a major where statistics is one or perhaps two lower level—one semester—courses. I remember my undergraduate and graduate days and coursework in stat’s. Heck, it became my Ph.D minor. However, I freely confess to being a “number dummy” until perhaps my third or fourth course. One day things just snapped and I had my “Road To Damascus moment”. Everything fell into line and an understanding of what the numbers I so readily cranked out came about.
I agree 100 percent with your reflection.
Decades ago, we completed several relevant studies in Oz.
Young J M, Glasziou P, Ward J E. General practitioners’ self ratings of skills in evidence based medicine: validation study BMJ 2002; 324 :950 doi:10.1136/bmj.324.7343.950
Ward JE, Shah S, Donnelly N. Resource allocation in cardiac rehabilitation programs: Muir Gray’s aphorisms might apply in Australia. Clinician in Management 1999; 8: 24-26
Young J, Bruce T, Ward JE. Is support among patients for colorectal cancer screen susceptible to ‘framing effects’? Health Promotion Journal of Australia 2002; 13: 184-188
Young JM, Davey C, Ward JE. Influence of ‘framing effect’ on women’s support for government funding of breast cancer screening. Australian and New Zealand Journal of Public Health 2003; 27: 287-290
Very good article, Sebastian!
Bloody hell. This tells us so much about the reliability of diagnoses. Even more relevant in current times.
Thanks Sebastian.
I find this such a necessary topic that you are covering, and your handling is interesting. However the results presented surprised me because they flew directly in the face of my lived experience; I’m a person who goes to the dr as little as possible and I wait at home and do home remedies until I am 100% sure it’s something I can’t handle on my own. Since I come from a medical family I’ve normally phoned my uncle or my cousin who are both practicing GPs before even getting into the office to hear their advice first, so I KNOW it is not something trivial and that from their opinion an antibiotic or some treatment that I cannot procure for myself is needed, and yet I always get brushed off and I don’t think I have ever been sent for tests or received an antibiotic without having to fight tooth and nail and getting my partner or sister to advocate for me. It is genuinely bizzare and it troubles me even more seeing that doctors apparently err on the side of too much testing and prescribing rather than too little.I am on the autistic spectrum and I’ve always wondered if my facial expressions and way of expressing myself and responding to pain (I’m a bit hyposensitive and once got pierced in the back by a 7 cm roofing nail and didn’t notice till I saw the nail attached to me) leads to them viewing me as lying or unreliable. I recall my father describing a time that nobody x-rayed his arm after he’d driven himself to the ER after a car accident since they didn’t believe it could be broken (it turned out it was, in 2 places, but he only found out after my mum took him back and insisted they do an xray) . To me the study which merely describes the patient’s symptoms is a really inaccurate way to test ACTUAL dr responses to ACTUAL patients, since so many experiences hinge on the way the patient is being perceived. It is quite harrowing for me to notice that if doctors merely receive a print-off of symptoms, they are far more likely to prescribe ,medicine and tests than they actually do in real life, and this leads me to conclude that this study actually shows how profound bias is and how things like disability and the physical presentation of the patient play an enormous role and create a huge gulf between how doctors perform on paper and how they perform in reality. Anyway, for context I am a university professor and a referee for quite a few scientific publications so I’m surprised somebody missed this obvious re-interpretation of the results.
Once I had a kind of heat stroke, looking strange to the world, strong sweating, tired in a split second and losing body control.
This all take place on a very hot and sunny day, selling books on an open air market, not drinking so much because no nearby toilet, etc.
The Neurologist scanned my head two times and my breast and told me I have a had a hypo (I am not diabetic) and he suggest to have always a sugar candy in the pocket.
3 Weeks later I had an other similar problem and the cardiologist diagnosed it as a cardiomyopathy failing.
In both cases the doctors where ready in minutes with the pre-diagnoses conversations, they where more or less not interested in my story.
Thank you for shedding light on doctors’ institutionalized ignorance, resulting in unnecessary and often harmful overtreatment. Similar studies showed similar results in the past:
https://www.bbc.com/news/magazine-28166019
Who has a vested interest in keeping the doctors in the dark? Cui bono? Who benefits?
I am a professional pilot, and Although I agree with practically everything you say in this post, I am not gonna discuss your knowledge of what is the complexity of flying a plane because I know that you know nothing about that.
Other than that, keep up the good work.
Forgot to mention that from my ignorant, but humble, point of view, the problem with doctors is that they know basically nothing about health, when it comes to know about workout and nutrition, they know nothing.
So doctors are very good at making tests, analytics, and discover that something is out of range. For that, they prescribe a pill. So they work more for the Big Pharma than for the health of their patients, although they think they are essential for the world, and for their patients health.
Yup, that sums it all up in a nutshell.
Pilots are intimately connected with their work outcomes, whereas doctors are more like interested observers 🙂
What’s the difference between a pilot and an air traffic controller?
If a pilot makes a mistake, the pilot dies.
If an air traffic controller makes a mistake, the pilot dies.
What’s the difference between a pilot and a physician?
If a pilot makes a mistake, the pilot dies.
If a physician makes a mistake, the patient dies.
Motivations are very important in determining human actions.
If physicians were held to the same level of accountability as pilots, we would have no physicians.
The Dean of Harvard Medical School tells incoming students: “50% of what we will teach you is wrong. That is not the problem. The problem is we do not know which 50% it is.”
Completely agree. I find doctors great for diagnosis (and that is very important) but not so good at treatment. After getting diagnosis I go home and do a hell of a lot of reading and very often decide not to fill my prescription. Works for me.
Back when my daughter was in medical school, I read an article about DVTs associated with travel and passed the info to my daughter. She had a rotation under a gp and he had diagnosed a patient as ordinary bacterial pneumonia. My daughter took a history and asked about travel and she ended up diagnosing a pumonary embolism and suggested a pulmonary angiogram. My daughter’s diagnosis turned out to be correct and she likely saved the patient’s life.
Sometimes a correct diagnosis has its origin in random factoids from the peanut gallery.
As a professional with 25 years of experience in large American hospital labs… this article is pretty much spot on. I could tell you an easy hundred horror stories of incompetence due to lack of experience, knowledge and sadly common sense by doctors. There is many factors involved. Some personal, some institutional, and some political (sadly – Aka, Covid-19 hysteria) but one over driving common root cause is, “expecting too much, too soon” of them, they are on overload burn out from the very beginning. In the old days, we use to say to a patient, “Get a second opinion”. Now days I would tell them, “get 3 or 4 additional opinions.” Doctors are just as guilty as another human of just following the line, not questioning, not seeing the forest, of just doing what is told of them…. Healthcare in America is sick, one could even suggest terminally ill, but that is a diagnosis only time will tell.
Good article confirming my own suspicions. If medical education is anything like what is currently being taught in the ‘hard’ sciences, we are all in a lot of trouble. Consensus has become a new hallmark of science, thereby destroying the science. Just because it is possible to rig a poll so as to get the desired answer does not automatically make that answer correct. As we have see over the past 16 months, a lot of bad information is easily passed off through the MSM and various governments as pure truth. There is no climate emergency, Covid is a designer bug that escaped from a lab using US Dollars to perform Gain of Function research, Renewable Energy is neither clean nor capable of powering the modern world, sea level is not rising as an alarming rate (actually ~2mm/year), storms are not worse than ever, the earth is not warming to unlivable temperatures…. I could go on but this is a short list to think about.
That experienced clinicians were no better at correct diagnosis than young doctors points to the fact that humans are all bad at probability. This is why we need all doctors replaced by a giant Ai; we only need one doing it all. The cost savings alone would be worth it. We also need an Ai to replace politicians so we can finally have rational policy decisions. The same politician Ai could also moonlight as the doctor Ai. Would’ve been no pandemic if an Ai was in charge. Not only is an Ai smarter by far, but it learns constantly from suboptimal outcomes, whereas people do not.
Tbh the AI thing would do great as government, if only you kept it turned off.
Anything, including nothing, would be better than what we have now. In 2017 80% of global income went to the top 1%. That’s impossible without government collusion.
I was in the Emergency ward of a Canadian hospital last year. The doctors said, “You need surgery now.” He pointed his finger to the offices upstairs and said, “Unfortunately, they won’t let me.”
If Seamus O’Mahoney’s books are accurate, there are no more practicing doctors. There are administrators directing medical technicians.
I onced called the lab to ask for sensitivity and specificity of a certain test only to be called up by by the responsible physician two days later, him self having to look it up. When that information is not readily avaliable physicians can’t even try to calculate probabilities. Rather we error on the side of caution, ordering more tests in case of false negatives.
I’m an engineer and not a doctor. A friend was having prostate problems and sent me his test results, not for a diagnosis (obviously) but to help him understand the science. I was pleased to see that the test results weren’t simply the normal ranges for PSA and Free PSA, but also had a little interpretive chart with the percentages of prostate cancer for the absolute range of PSA and the ratio of Free PSA to PSA. This was a cheat sheet, acknowledging that healthcare providers aren’t good at keeping the diagnostic probabilities in their heads. Why should they? The computer can provide that data.
I suspect this “dumbed down doctor” problem is much worse now that government, insurance companies and healthcare conglomerates have largely replaced doctors with physician’s assistants and then replaced physician’s assistants with nurse practitioners.
Soon, the statistical diagnostic data chart with two independent variables and the nurse practitioner to look up the presumptive diagnosis on the chart will be replaced by an artificial intelligence with a *much* larger data set and potentially thousands of variables to more accurately determine diagnostic probabilities specific to each patient. Physicians hate that idea and will actively oppose it, as everyone resents automation of their job.
We’ve already reached the point where an intelligent and motivated person with a search engine can often diagnose their own medical problems more accurately than their primary healthcare provider. Physicians hate that too. I should be able to order any test I want on myself, to help me better diagnose myself if so inclined, but of course I can’t.
Over 20 years ago, I was between doctors, so I went to my wife’s doctor. My wife liked her, because she listened when most doctors seem to hear half a sentence and rush to a presumptive diagnosis and after that, you’re wasting their time. I thought about trying to diagnose my problem online, even back then, but resisted. For insurance purposes, I needed to see a primary care physician and then be referred to a specialist. I couldn’t simply go to an orthopedic surgeon, so there was really no point diagnosing myself if I couldn’t treat myself. My wife’s doctor did listen patiently. Then she did the cookbook diagnostic procedure. She had me raise my arm as high as I could, then she tried to lift it higher and couldn’t. I’m an engineer, so I was reverse engineering her diagnostic procedure. I thought, “She just ruled out a torn rotator cuff.” Then she announced, “You have a torn rotator cuff.” I started to blurt out that the test proved I didn’t, but humiliating her served no purpose. She referred me to the orthopedic surgeon I should have been able to see without her if we had free market healthcare, and he correctly diagnosed a frozen shoulder, aka adhesive capsulitis.
As an engineer, I diagnose technical problems. I’m amazed at how bad many physicians are at diagnosis. The system hides diagnostic incompetence so those performing our primary healthcare are worse diagnosticians than we imagine.
Is there anything an engineering approach cannot improve? It’s sometimes a little cold, but people should get over that. Are we not machines, essentially?
In the early 1980’s I remember reading about an AI program called ‘mysin’, which was supposed to help Dr.s diagnose bacterial diseases. There have been other AI-type programs in that time frame (Caduseus? Others…). I haven’t heard much about them in the last couple of decades, so I’m assuming the programs did not pan out too well. They did however deal with some difficult probability issues that might be of interest w.r.t. this article.
Maybe the programs could be resurrected as doctor’s assistants to help them deal with the various probabilities mentioned here.
The 1980s artificial intelligence programs were not AIs in the current sense. They were rules driven deterministic algorithms using expert system programming languages such as Prolog. Some of them produced good results because they’re much more expandable than any human brain. They could be programmed with the combined rule based knowledge of thousands of experts.
Today’s AIs are voodoo by comparison. They accumulate vast amounts of information – an unthinkably large set of data by human standards. They’re given a task, such as diagnosing breast cancer based on radiology images. They essentially train themselves, converging on an optimal solution. Humans don’t know what rules they derive for themselves to accomplish their goal. AIs have been proven to use different rules than humans, and presumably better rules than humans if they’re producing better results, which they often do.
If the definition of intelligence is sufficiently broad, a case could be made that AIs are smarter than humans, at least as intelligence applies to some specific tasks such as determining predictive credit scores, diagnosing early breast cancer, etc.
AI technology is increasing exponentially. The fundamental question is… will they keep us around as pets?
What is really frightening is that if we are already standing at the cliff with universal healthcare (sub-healthcare to me), what will be the effect of the rush to implement the CRT diversity juggernaut? If it is racist to expect that a certain classroom and practice benchmark of medical knowledge be set, what you’ve described in this post bodes badly for medicine’s future. My husband is a psychiatrist and the AMA magazines he gets (even though he has never subscribed) have are increasingly more WOKE and virtue-signaling in this regard.
In America, …I’ll keep it to California, I will wager there is not a single major medical center or medical school that has not gone woke.
“Father, forgive them for they know not what they do…”
There’s always the AAPS for those who abhor woke-ism. The doctors there seem to actually care about patients.
I doubt that you even have to be American to join.
But doctors sure serves big pharma well! What a beautiful system.
When a system is ranked 37th in the world (the US system) for outcomes but is the most expensive by far and also the most profitable, that is proof it is an engineered scam. Imagine being sold a bald tire for twice the price of a new tire and that continued for decades in a supposedly free market system. Could only happen if it were an engineered scam in a rigged market.
That is a pretty lousy source you quote. Oh wait you didn’t quote (shame). No need we all know it is the glorious WHO (Covid coverup) group sponsored by China.
But I digress… indeed there are issues in America… big ones.
We need a modern Flexner Report to address problems in the medical profession. Knowledge and technology both increase over time, but in many ways we’re going the wrong direction. The medical industrial complex response to COVID has shined a light on our medical problems. There is too much central planning with treatment protocols handed down from on high with too little independent thought. Healthcare providers follow recipes. This has become worse, at least in the US, as government and insurance companies have forced physicians out of the practice. My mom had a doctor, then a physician’s assistant and now a nurse practitioner. Next year, I expect the receptionist to diagnose and treat illnesses.
Technology is changing the workplace, and the medical field isn’t immune to automation. Artificial intelligence is already ten times better at detecting breast cancer from mammograms with no higher rate of false positives. US radiology jobs were outsourced to India when it was less expensive to email the image to India and email the radiology report to the US, but that business model only lasted a decade and now we’re going to start seeing an AI on a US server performing the tasks of a radiologist. An AI gets better over time, the more data it sees, and an AI is never tired or distracted.
The JAMA study implies that an AI could already outperform physicians on more generalized diagnosis of illness. At the very least, an AI is data driven and would not make gross errors in assuming disease that probably is not present, but most patients want a real person instead of an AI, and certainly the healthcare providers are very resistant to having their jobs automated, so we’ll see social resistance even though the technology makes sense. It’ll be eased into being through methods designed to overcome these objections. An AI will be introduced that will allow a Third World medical technician with rudimentary training to provide much needed healthcare in remote villages. The AI will learn and grow. Eventually, the Third World will have much more accurate medical diagnostics and it will be increasingly difficult to ignore.
This supports why medical errors are the third leading cause of death in the US
https://www.bmj.com/content/353/bmj.i2139
Maybe this is why our medical industry is so reliant on testing devices. The PCR is an example of exactly what a testing instrument is not.
Our problem is the reliance on automation and AI and this is exactly where the problem lies. SCI-FI has captured logic and people should realise Star Wars is not real and nor is Virtual reality. A VR headset does not make you a doctor or specialist. All programming requires input and the programmers are not qualified as doctors or specialist yet they are writing the code. Liberty4Ever is very judgemental and quick to blame incompetence. Thank goodness he is an engineer and not a doctor. Doctors and specialists are unable to read the code nor can they provide a solution for every situation. No matter how impressive technology may appear it cannot reason inductively or deduce from experience. The “machine learning” a concept is very different from the way we learn and deduce and understand. It is a very dangerous route that is being taken. My career was process instrumentation and automation and I disagree with Liberty4Ever. His only correct comment is “government and insurance companies have forced physicians out of the practice” BTW a report like the Flexner report is exactly what has created the problems we face. Its called Medical Fascism. Suggest he reads “One Hundred Years of Medical Fascism” he may change his brand of automation ist alles snake oil. I would recommend he and people realise this and should read “Automation Bias in Intelligent Time Critical Decision Support Systems”- M.L. Cummings Massachusetts Institute of Technology, Cambridge, MA 02319.
Medicine is “Time Critical Decision Making” is it not.
“And what are the odds that we will make a correct decision based on the answer we get back?” So it has become like a gamble. As Heidi correctly surmises, “This supports why medical errors are the third leading cause of death in the US”. I would venture in the world.
Looking at how the educational institutions have degenerated into Marxist training camps and the standards are continuously being lowered to accommodate those who want a title because they can buy one clearly is part of another problem. All want to be called “Dr”, all want respect and respect is earned, entitlement is not respected. I am not at all surprised why we are in the dilemma we are with the Covid Hoax. It is all political and has an agenda that is a slippery as an eel, the goal posts are artificial and imaginary. When politicians play doctor it is a slippery slope.
We are seeing witchdoctors and bone throwers being entitled to be paid from medical schemes. Next will be clairvoyants and tarot card readers crystal ball gazers….. and Politicians.
Hej ! En gammal man ( 87 ) Inte så värst bra på engelska ! Men ,med dina reflektioner skulle jag
verkligen vilja att du lånade en ganska tunn bok av mig . Den är slut från förlaget och går inte att få någon ny. Dr Carl Carlsson, en gång överläkare i Göteborg, har med frun som sköterska drivit ett eget
hälsohem i Västergötland. Pelle Nyquist har skrivit en utmärkt bok därom !
Jag menar bara att alla behöver inte uppfinna hjulet på nytt.
Midsommarhälsning Per
Worse than that.
A young GP called me into the surgery to discuss my cholesterol test. He wanted to put me on statins. After I pointed out that my high cholesterol level was GOOD cholesterol and there was nothing wrong with my level of bad cholesterol he told me he wanted me to take statins FOR STATISTICAL REASONS!
According to his ‘statistics’ everyone in my age group should be on statins!!
Incredible? It’s true.
That is the state of medical training in the U.K. N.H.S.
Or was it that he had been given some so called ‘statistics’ by a pharmaceutical company’s sales rep?
Or that is how bad that G.P. was in spite of his training.
I saw him still in practice a few months later but don’t know what may have happened since.
Excellent review. It is an incredibly important topic. The medical industry is now fully dependent on both physicians and patients not understanding statistics.
Dear Sebastian, you mentioned the training of pilots. Exactly, Dr. Amitai Ziv using his experience when a pilot in the Israel Air Force and the training they received before flying a fighter jet created the Israel Center for Medical Simulation (www.msr.org.il) to better train doctors and avoid errors. Possibly you may interview Dr. Amitai and tell all of us about how effective is his training.
I’m going to disagree a bit with your premise a bit Dr Rushworth, mostly as a devil’s advocate, but also because the devil should have his due.
First of all, it isn’t just doctors that have a problem with thinking in statistically valid terms – this is a general problem: it isn’t a natural way to think. According to some statistician, even statisticians have problems thinking statistically.
Secondly, unless you are crunching large numbers with a known distribution, it isn’t clear to me why statistical thinking is more valid than other ways of coming to a conclusion. Do I really need to have an understanding of my probability of crashing if I am drink driving to conclude it is a bad idea? Not really – if I weave in my walking and have trouble unlocking the car, then that is far more informative.
We have many ways of analysing data and coming to conclusions. If someone asked me to estimate the probability of missing an infection using a test that was alleged to be 100% sensitive (and why do they use such confusing terms as sensitive and specific – do they not care that these are easily confused?), the first thing I would think is how likely is this test to be actually 100% sensitive? Not bloody likely would be my first thought.
If you need to analyse large amounts of data, then you must understand statistics thoroughly (which I agree the people running our public health bodies certainly do not), but if you are dealing with a haphazard selection of patients, as are most GPs, then maybe not so much.
Also, although it may no longer be in style, ‘First do no harm’ seems to be a good motto for doctors and a reasonable amount of overprescription is not necessarily bad.
We may need terms like “insensitivity” and “non-specificity” to make things clearer. “Insensitivity” is the likelihood of false negatives and “non-specificity” is the likelihood of false positives. I would recommend a third term specifically for PCR–“uc-positives” (unculturable positives).
We also need to distinguish between lab specificity and lab sensitivity and working values for those items, as well as their inverses. What you get in the lab doesn’t necessarily transfer to the field to the same degree. Specifically, insensitivity in the field was _far_ higher for covid PCR than in the lab (a minimum of 20%!), even with cycle thresholds maximized in order to reduce insensitivity.
This brings up another issues–what happens when sensitivity and specificity vary based on a lab test machine setting? And how are doctors to make sense of this when things like cycle thresholds being used are hidden?
Then, to complicate things further, we have to consider reasons for _why_ insensitivity is so high in the field for covid PCR. Perhaps the problem is lack of training of sample collectors. Or maybe it is improper storage of samples. Or maybe there was a problem early with delays in samples reaching the labs for processing, resulting in degradation of the sample. Perhaps insensitivity was reduced as delays between sample collection and processing were reduced.
Now to the assumption of prevalence. Why should we believe a prevalence of 0.001 based on the test? Clearly, there must be some sort of confirmation to sort out false positives. The test, by itself, won’t reveal which are false positives or even that any exist. So we need a second test to discover false positives and perhaps a third test to discover false negatives.
What do we do when prevalence changes, as in the case of a pandemic? Case data will always be dilatory in the case of a pandemic–for covid, it looked to lag by six weeks. It’s a good thing we have the Gompertz Curve to help us estimate where we are in terms of current cases. But the particular curve depends on the R0 infectivity factor, doesn’t it? So we have to somehow discover what that value is–and that value may vary, depending on where infectivity is measured due to factors like subways and native immunity. (Far eastern countries had substantial exposure to SARS, which may have increased their native immunity to SARS-COV-2.)
Exposure isn’t the same thing as prevalence. Perhaps an exposure percent can be used to estimate prevalence during a pandemic, using a Gompertz Curve. I don’t know the answer to that question.
So, assuming prevalence in a pandemic looks like you can only get a lower bound–especially early on. This makes determining probabilities of infection very difficult, even for infectious disease experts. Pity the poor gp’s.
Let’s consider the problems with discovering uc-positives. I wonder what impact PCR specificity and sensitivity will have on culturing virus. Do you have to run PCR at Ct 24 maybe five or ten times and throw out the runs that don’t meet the R-test? Do you throw out values that exceed the standard deviation of the data or perhaps two standard deviations?
In this discussion, I have tried to show potential sources of error and clarify them and clarify where we have uncertainty.
PCR is not a diagnostic test simply put. It is not fit for purpose.
PCR is not sufficient for clinical diagnosis, but it works well in conjunction with viral culturing for research into viral transmission and measuring viral load. Vaccine efficacy studies require detecting viral transmission (new cases) and ruling out disease exposure at the outset.
So we agree.
Viral culturing can rule out both false positives and false negatives.
PCR was originally meant to be used simply as a tool to amplify nucleic acids for molecular biologists to study in their research. However, in conjunction with viral culturing, it _can_ be useful in some other research areas, two of which I have mentioned previously.
Dr. Rushworth has mentioned the Marine Corps study of allegedly asymptomatic transmission, but I don’t give that study any weight for several reasons. 1) Marines are used to lying in their culture and place a premium on being able to deceive administrators, 2) Marines will avoid reporting to sick bay, 3) Marines are very inventive about sneaking off base past guards and sneaking prostitutes onto bases, 4) the frequency of the distribution of various viral genes in the Marine population studied is unknown, 5) the data didn’t report the sex of participants, and 6) the sexual practices of the participants wasn’t reported (transmission likely is higher during sex).
I am not sure how you can surmise we agree. I was clear the PCR was not fit for purpose.
It cannot measure viral load. That was what was assumed it could do. The detection of adenine using luciferase does not indicate viral load. There are other (living) cells that contain ATP which also binds to luciferase.
The sample is a soup of various cells and fragments. The photons measured cannot tell you anything except that there is adenine present. The assumption is made that the viral load is somehow related to the Ct and the counts. Its a ludicrous assumption.
The PCR cannot even tell you which specific corona virus the fragments may belong too, it cannot tell you if the fragments are from a past infection. It cannot tell you if you are infectious or not. It relates to nothing.
If you have not determined from the outset that the fragments belong to a particular virus how can you make a deduction that it is so.
The fact is they the political Vaccine Industrial Complex needed something to support the pillar of their fraud.
They the WHO et al, chose PCR under Drostens advice probably. The protocol was published even before any real virus isolate became available. It was based on in silico (theoretical) made by computer sequences of the viral genome. To date no validation has been performed by the authorship based on isolated SARS-CoV-2 viruses or full-length RNA thereof. The problem is that he has been caught out lying by his peers. Here I refer to the following. On November 27, 2020 an International Consortium of Scientists in Life Sciences published an external peer review titled ‘Review Report Corman-Drosten et al. Eurosurveillance 2020,’ documenting 10 major flaws and requesting a retraction. The silence is deafening.
We agree that PCR _by itself_ cannot measure viral load. PCR in conjunction with viral culturing _can_ measure viral load, as Raoult showed waaaay back in one of the most important (foundational) studies of covid. Did you miss this study?
https://link.springer.com/article/10.1007/s10096-020-03913-9
I thought of Dr. Rushworth’s post when I watched Harvey Risch’s analysis of various studies of treatment for covid. Lots of probability and interpretation of probability. I find that I think a lot like Dr. Risch does. Watch the video before it is removed.
Thanks for your valued information.
With reference to your comment on the Viral RNA load as determined by cell culture – Raoult.
I think there is a lot of politics in the issue of PCR testing. I’ll go with the inventor Kary Mullis who made it abundantly clear that PCR was not a diagnostic tool. Kary Mullis, himself, set forth that the PCR test should only ever be used in R&D, and NOT as a diagnostic tool.- http://www.drug-dissolution-testing.com/blog/files/cdc-study-article-comments.pdf
Its the same as the issue with antibody testing. There are no specific antibodies. Rosemary Frei investigated this issue and posed a question to Harvard Medical School professor Clifford Saper- “is it true that, as most in the antibody-commercializing arena claim, a monoclonal antibody can be created that’s specific for (that is, binds to) just one type of virus or just one other type of organism. The response from Saper was “No, there is no such thing as a monoclonal antibody that, because it is monoclonal, recognizes only one protein or only one virus. It will bind to any protein having the same (or a very similar) sequence.”
It looks like there is no consensus according to another systematic review …. Viral cultures for COVID-19 infectivity assessment – a systematic review (Update 4)
T Jefferson, EA Spencer, J Brassey, C Heneghan
medRxiv 2020.08.04.20167932; doi: https://doi.org/10.1101/2020.08.04.20167932
Conclusion: Prospective routine testing of reference and culture specimens are necessary for each country involved in the pandemic to establish the usefulness and reliability of PCR for Covid-19 and its relation to patients’ factors. Infectivity is related to the date of onset of symptoms and cycle threshold level.
You will see that they stated . The data are suggestive of a relation between the time from collection of a specimen to test, cycle threshold and symptom severity. The quality of the studies was moderate with lack of standardised reporting.
It remains a fact that PCR cannot determine or prove the existence of SARS-CoV2. The paper you referred to says “Correlation between successful isolation of virus in cell culture” It is well known that that is s supernatant and not Isolation and Purification. So is the viral load specific to SARS-CoV2? I would suggest not according to the Statement On Virus Isolation (SOVI)- https://andrewkaufmanmd.com/sovi/#voices
Um, now we disagree. Raoult plainly found a correlation between the ability to isolate a virus in cell culture and the level of cycle threshold. Can you grow SARS virus in Vero cells? Yes. Maybe you have both SARS and SARS-2 infections in some of the same people. I wouldn’t know how to distinguish them.
Both SARS and (most variants of ) SARS-2 are cytopathic in cell culture and there are ways to find out if they multiplied in the cell culture. (E.g., you measure PCR Ct value baseline at inoculation of the cell culture and several days later and compare)
I’ve read the Heneghan paper several times and have no issue with it or its conclusions. I suspect that the authors had to tone down their conclusions in order to get published.
You might read the following article…
“Severe acute respiratory syndrome coronavirus-2 (SARS CoV-2) belongs to the genus Betacoronavirus (lineage B), subgenus Sarbecovirus. It was first described in December 2019 as the virus responsible for the current global COVID-19 pandemic [1]. The ≈30-kb single-stranded RNA genome of SARS-CoV-2 encodes for 4 structural proteins assembled into viral particles and 25 additional proteins required to hijack the molecular machinery of the host and produce copies of the viral RNA and the infective virions [2]. The four structural proteins are the spike (S), membrane (M), envelope (E), and nucleocapsid (N) proteins. These proteins comprise 1273, 222, 75, and 419 amino acids, respectively. The S protein is a glycoprotein that plays a pivotal role in viral attachment, fusion, and entry into host cells [3]. The precursor protein is cleaved into two subunits, an N-terminal S1 subunit responsible for the attachment to the host and a C-terminal S2 subunit mediating membrane fusion. Three S1/S2 heterodimers assemble to form a trimer spike protruding from the viral envelope. The membrane (or Matrix) protein is a glycoprotein with a central role in virion assembly and morphogenesis. The envelope (E) protein is a small membrane protein with a single alpha-helical transmembrane domain which plays a central role not only in virus morphogenesis and assembly, but also in pathogenesis during COVID-19 infection as it acts as a viroporin by forming homopentameric protein-lipid pores in the host membrane. The nucleocapsid (N protein-RNA complex) protects and organizes the RNA molecule into the viral lumen. Interestingly, structural studies of viral particles revealed an extensive heterogeneity of its molecular architecture [1].”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8134822/
2 things
Surely to God please, and if not why on earth not, test results come with a probability attached, so in your 5% false positive example the result looks like. Positive. 1.96% probability (1/51)
Medicine as an art. I suspect this will soon be retranslated as how well your decisions conform to the AI’s opinion. This art is definitely susceptible to numerical/statistical measurement.
theasdgamer.
No we don’t agree. The correlation proves nothing. The high viral load is an unproven hypothesis. You can have a high viral load and be infectious or not. It is simply counting an amount of photons from an Enzyme called luciferase and making a guess. Its an assumption a 50/50 guess. Not proof.
PCR is extremely sensitive, which means it can detect even the smallest pieces of DNA or RNA — but it cannot determine where these particles came from. That has to be determined beforehand.
And because the PCR tests are calibrated for gene sequences (in this case RNA sequences because SARS-CoV-2 is believed to be a RNA virus), we have to know that these gene snippets are part of the looked-for virus. And to know that, correct isolation and purification of the presumed virus has to be executed. This has not ever been done.
And on you next post we won’t agree either. The ncbi.nlm.nih.gov/pmc/articles/PMC8134822/ again refers to a culture. A supernatant of a conglomeration of cells no Isolation and Purification or Characterisation either. The accepted scientific methodology has not been fulfilled. I referred to that in the SOVI post. Its the same argument .
So the existence of SARS, MERS or whatever is a spook story about a phantom. A long latin waffle name means nothing. A mere assignment of names and code numbers like covid or sars cov2 is being used to convince or scare people BullSh*t baffels brains simply put.
Jon Rappoport takes the CDC where this originates apart. https://blog.nomorefakenews.com/2020/10/22/the-virus-that-isnt-there-genetic-sequencing-and-the-magic-trick/- So, lets leave it there.
You cannot have a high viral load and not be infectious. You can have a high RNA load and not be infectious. Two different things.
Viral isolation…
“In order to observe virus particles, Vero cell monolayer showing the cytopathic effects was fixed as previously described.7 It was cut on ultramicrotome (RMC MT-XL; RMC Boeckeler, Tucson, AZ, USA) at 65 nm. Ultrathin sections were stained with saturated 4% uranyl acetate and 1% lead citrate before examination with a transmission electron microscope (JEM-1400; JEOL USA Inc., Peabody, MA, USA) at 80 kV. Spherical particles with crown-like spikes ranging 66 to 81 nm in diameter were observed within the cytoplasmic vesicles and in the extracellular space adjacent to cell membrane (Fig. 1C and D).”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7036342/
Your source is mistaken.
Its astounding that you reference articles that have been found to be unsupported by well known scientific methodology including referenced material that has been questioned regarding to interpretation of EM that was incorrect.
Specifically this one that was questioned by BPA Pathology Dr. Stoyan Alexov.
Study 3: Wan Beom Park et al. “Virus Isolation from the First Patient with SARS-CoV-2 in Korea”, Journal of Korean Medical Science, February 24, 2020
Replying Author: Wan Beom Park Date: March 19, 2020
Answer: “We did not obtain an electron micrograph showing the degree of purification.”
The ref in this was 3. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020 doi: 10.1056/NEJMoa2001017. This was questioned by Dr Alexov.
Study 4: Na Zhu et al., “A Novel Coronavirus from Patients with Pneumonia in China”,
2019, New England Journal of Medicine, February 20, 2020
Replying Author: Wenjie Tan Date: March 18, 2020
Answer: “[We show] an image of sedimented virus particles, not purified ones.”None of these studies fulfilled Koch’s postulates.
Koch’s Postulates are unnecessary to prove viral disease. They only apply to bacterial infections.
What happens when a pathogen causes disease in some organisms but not in others?
https://en.wikipedia.org/wiki/Koch%27s_postulates
Wiki a reference. Oh dear. Now I understand your delusion.
Wiki is not a problem if the topic is not controversial.
I am finished discussing with you.
Looks like your Rappoport link is busted.
Link works here.https://blog.nomorefakenews.com/2020/10/22/the-virus-that-isnt-there-genetic-sequencing-and-the-magic-trick/
You can search it too by the month
October 2020
I searched for “ACE2” in your link and the Cowan link. I think that Cowan is a fraud and maybe Rappoport as well.
“I am finished discussing with you. Wiki is not a problem if the topic is not controversial.”
Please put on a mask and a blindfold too.
Wiki is one of the big censors of fact and free speech. Any censorship based on ideology is unacceptable.
https://quadrant.org.au/opinion/free-speech/2021/03/wikipedia-they-should-call-it-wicked-pedia/
Even the co-founder thinks so.
A few years ago, a senior Consultant told me my father-in-law should not have an operation because there was a 20% chance he would die from it. I responded with what I thought was the obvious question: what chance has he of dying if he *doesn’t* have the operation?
Blank out.
He seemed not even to understand the question.