"80% of medical AI queries are factually wrong."
I assume this was a human generated answer.
The claim is decent but requires significant qualification — it’s accurate in a narrow sense that gets badly distorted when stated baldly.
A recent JAMA Network Open study evaluated 21 large language models across 29 clinical case scenarios and found that AI systems failed to provide the correct diagnosis in over 80% of early-stage cases, where symptoms are often non-specific. The 80% figure applies specifically to early-stage differential diagnosis with minimal clinical information, not to medical queries generally.
Source:
https://www.business-standard.com/health/ai-chatbots-misdiagnose-early-cases-80-percent-study-126041400329_1.html
But there’s more to it.
"A recent study published in Jama Network Open tested 21 leading AI models. The researchers asked them to work out possible medical diagnoses. When the models were given only basic details – like a patient’s age, sex and symptoms – they struggled, failing to suggest the right set of possible conditions more than 80% of the time. Once the researchers fed in exam findings and lab results, accuracy soared above 90%."
Source:
https://theconversation.com/half-of-ai-health-answers-are-wrong-even-though-they-sound-convincing-new-study-280512
If precision is what you’re for, then be the change you want to see in the world.
John Dewey liked to say, "A problem well-put is a problem half solved."
If the question is not well put, the answer likely won’t be. Put in a better effort on your question/prompt and the answers will get better.
That does NOT mean that AI should take the place of human experience or that it’s worth the energy cost or that it’s valid to use for plagiarism, etc. But the claim being advanced is that it cannot do a good job with facts. That just ain’t so in many cases.

