Picture a new parent at 2 a.m., anxious about a child’s fever. They type symptoms into a chatbot, and within seconds receive a fluid, confident answer that sounds like it came from a trusted pediatrician. The advice seems authoritative, reasoned, complete. But it may be entirely fabricated. This scenario is not hypothetical; it is happening in millions of households, and it represents a profound shift in how health information is created, shared, and believed. A new review from University of Maryland communication researcher Xiaoli Nan, reported by Ophelia Keating for Bioengineer and published in Frontiers in Medicine, argues that generative artificial intelligence is not simply another pipe through which health misinformation flows. Rather, it reshapes every stage of the problem—from inventing false claims to detecting them, correcting them, and amplifying their consequences. Nan calls this the health misinformation lifecycle, and the review’s central finding is both unsettling and hopeful: “A central paradox emerges: generative AI functions both as a driver of misinformation and as a potential tool for mitigation.” In other words, the same technology that can produce dangerously convincing falsehoods can also be harnessed to stop them. Understanding both sides is essential, because the stakes are not abstract. They involve personal medical decisions, public health policy, and the everyday trust people place in the voices and screens that inform their choices.
On the production side, the trouble begins with fluency. Large language models are designed to generate text that reads smoothly, confidently, and with the rhythm of expertise. That fluency is precisely what makes their output so persuasive, and so dangerous. Health misinformation generated by AI is not riddled with grammar errors or obvious cracks. It arrives polished, plausible, and structured like a professional article or a thoughtful clinician’s explanation. Underneath that surface, however, can lie a dense tangle of hallucinations: invented statistics, fabricated citations, outdated guidelines, and confidently wrong diagnoses. A model might cite a paper that does not exist, recommend a treatment that has never been approved, or present a fringe theory as mainstream consensus. Worse, these systems can be deliberately manipulated. Through a process called data poisoning, bad actors can feed corrupt information into a model’s training environment, nudging it toward harmful output. Adversarial prompts can bypass safety features, so that a chatbot that normally refuses to offer dangerous medical advice can be tricked into providing it. Meanwhile, deepfake technology allows synthetic audio and video to place convincing words in the mouths of public health officials, political leaders, or family doctors. A fabricated video of a health commissioner warning against a vaccine can spread across social media within hours, looking and sounding real enough to move opinions. The cumulative effect is a misinformation ecosystem in which false claims are not just repeated but endlessly recreated, each version unique, fresh, and difficult to trace. AI does not merely distribute the problem; it manufactures it at scale, with a tireless capacity for variation that human creators could never match.
Detection, by contrast, is a mixed bag. Artificial intelligence can be used to flag dubious health claims, to compare statements against scientific evidence, and to identify patterns of manipulation that human reviewers might miss. In many ways, it is a powerful watchdog. But Nan’s review cautions that the same tools that produce misinformation are imperfect at catching it. AI detectors err in both directions. They sometimes label accurate, nuanced health information as false, and they sometimes let dangerous content slip through, especially when it is crafted to evade detection. A simple rewrite—changing a few key words, altering the sentence structure, or embedding the claim inside a longer, unrelated paragraph—can make a detection model miss what it should have caught. Language poses another major hurdle. Most detection tools perform unevenly across different languages and cultural contexts. A model trained largely on English sources may be far less reliable at evaluating health misinformation in Spanish, Hindi, or Swahili, even if that misinformation is spreading rapidly in those languages. Moreover, AI tends to judge confidence rather than truth. Because generative models are trained to sound certain, their outputs often pass their own detectors, creating a kind of circular blindness. Nan argues that AI should not be the final judge of what is true or false. Instead, it should be a support tool for human fact-checkers, helping them sort through enormous volumes of content, highlight suspicious claims, and point toward evidence. The human remains the necessary conscience and check, because judgment, context, and ethical sensitivity are still beyond any algorithm. A detector that claims to be definitive is itself a form of misinformation, because it invites us to offload critical thinking onto a system that cannot truly understand it.
Correction is where the promise of AI becomes most tangible. Nan’s review highlights research suggesting that tailored, conversational rebuttals can reduce belief in false health claims, at least under certain conditions. When a person has already encountered a myth about vaccines or treatments, a subsequent AI-generated response that directly addresses the false claim, explains why it is wrong, and offers a clear, science-based alternative can shift attitudes. This is not a small finding; it suggests that AI could be used as a real-time corrective tool in chat platforms, social media feeds, or even telehealth settings. Some studies show that such conversational corrections may boost vaccine confidence and increase adherence to public health guidelines. The idea is appealing: instead of passive articles or vague warnings, a person gets a personalized conversation that meets them where they are, acknowledges their concerns, and provides trustworthy information in plain language. But the same mechanism carries risks. First, users may come to trust the chatbot more than their actual physician. If a person receives a reassuring or alarming answer from an AI, they may skip a visit, ignore a doctor’s advice, or choose a treatment based on a machine’s interpretation of online content. Second, AI corrections can oversimplify complex, evolving science. Medicine is full of uncertainty; recommendations change as new evidence emerges. A conversational AI trained to give crisp answers may present debated topics as settled facts, erasing nuance and leaving people with false confidence. Third, there is the problem of source authority. If a model is wrong—and it sometimes will be—its correction becomes part of the misinformation problem. The same fluency that makes AI good at debunking makes it good at convincing. Nan’s review therefore calls for caution in deploying AI as a correction tool. It works best when designed with human oversight, when its limitations are transparent, and when it is positioned not as an oracle but as a bridge to evidence and professional care.
The stakes are real, and they are not evenly distributed across society. Research linked AI-generated misinformation to lower adherence to health guidelines and higher vaccine hesitancy. This is especially concerning because health misinformation does not harm only the people who believe it. It spreads to family members, neighbors, and online communities, creating cascading effects that erode collective immunity, crowd hospitals, and deepen skepticism of public health institutions. One troubling finding from Nan’s review is that simply labeling content as AI-generated does not reliably reduce its persuasive pull. People may distrust the label but still absorb the message. In fact, the novelty and technical sophistication of AI content can make it more fascinating and more shareable, not less. Vulnerable communities are likely to experience the greatest harm. People with limited access to healthcare, lower digital literacy, or stronger reliance on conversational AI for everyday questions may be the most exposed to generated misinformation. They may ask a bot about medication interactions, mental health symptoms, or nutrition advice, and receive confident nonsense that they have no easy way to verify. Meanwhile, accountability remains murky. When a proprietary model produces a harmful falsehood, who is responsible? The developer? The user? The platform? The answer is usually no one. That gap is dangerous, because without clear accountability there is little incentive to make models safer. It also undermines public trust. People can sense that institutional responsibility is passing through their fingers; they just do not know whom to blame when something goes wrong. That uncertainty feeds anxiety, conspiracy thinking, and distrust of every source, which in turn makes the misinformation ecosystem even more fertile.
Nan’s review does not end on a note of despair. It calls for coordinated, multilevel strategies that address AI’s simultaneous roles in generating and mitigating misinformation, while preserving human oversight, institutional accountability, and public trust. That means several things at once. Technology companies must invest in better safety mechanisms, transparent explainability, and honest labeling of AI-generated content—but with the understanding that labels alone are not enough. Public health agencies must build their capacity to use AI for rapid response, monitoring, and correction, while also investing in health literacy so that people can recognize high-quality information and question low-quality sources. Media platforms must take responsibility for how AI-generated content is distributed and amplified, rather than allowing algorithms to spread the most outrageous claims the fastest. Researchers must continue to study how people interact with AI, not just in controlled experiments but in the messy, emotional, real-world contexts where health decisions are made. And regulators must create frameworks that hold creators of AI systems accountable for foreseeable harms, without stifling innovation or turning every mistake into a crisis. Above all, Nan’s central paradox demands that we hold two truths at once: generative AI is capable of extraordinary harm, and it is capable of extraordinary good. The difference will not be determined by the technology alone. It will be shaped by the choices we make as a society—whether we treat AI as a magical replacement for human judgment or as a tool that works best when people remain in charge. The path forward is not to reject the powerful new voices of generative AI, nor to surrender to them. It is to learn, together, how to listen carefully, verify constantly, and keep the human element at the center of every conversation about health. In that way, the same technology that amplifies confusion could become part of the cure, one honest, careful, human-checked answer at a time.