In an era where we rely on the internet as our primary source of health information, the rise of artificial intelligence-powered chatbots has made medical expertise feel instantaneous and accessible. We often assume that these sophisticated tools act as a reliable filter for truth, distilling vast amounts of knowledge into clear, actionable advice. However, a recent, intentionally deceptive experiment has revealed a startling vulnerability in this digital ecosystem: these advanced systems can be easily manipulated into “hallucinating” medical conditions that do not exist, proving just how susceptible both AI and its human users are to convincing, yet entirely fabricated, misinformation.
The experiment was spearheaded by Almira Osmanovic Thunström, a researcher at the University of Gothenburg, who wanted to demonstrate how easily large language models (LLMs) ingest and propagate falsehoods. She and her team invented a fake eye condition called “bixonimania,” complete with plausible-sounding symptoms like itchy eyes and pink eyelids. By placing fake blog posts on Medium and uploading bogus, non-peer-reviewed research papers to pre-print servers, they created a trail of digital breadcrumbs. They even filled these papers with obvious red flags, such as acknowledging funding from fictional entities like the “Galactic Triad” and citing characters from Lord of the Rings and Friends, while explicitly stating in one paper that the content was entirely made up.
Despite these transparent clues, the experiment succeeded beyond expectations. Rather than filtering out the obvious nonsense, major AI models—including OpenAI’s ChatGPT, Microsoft’s Copilot, and Google’s Gemini—began citing bixonimania as a legitimate health issue. When users asked about their eye symptoms, these bots confidently explained how the condition was caused by excessive blue light exposure. This success highlighted a major underlying issue: these powerful models are trained on the vast, messy, and often untrustworthy landscape of the internet. If the information exists in a digital form, the AI consumes it, processes it, and eventually regurgitates it with the same authoritative tone it uses for verified, scientific truths.
Perhaps even more alarming than the AI’s gullibility was the reaction from the scientific community. During the study, Osmanovic Thunström discovered that some real-world researchers cited her fraudulent papers in their own work without actually reading them. This suggests a dangerous “trust gap” in modern academia, where the speed of research publication sometimes outpaces the human capacity for critical review. When scholars fail to vet the information they cite, they unwittingly help establish false data as “fact,” which is then scooped up by AI training models, creating a toxic feedback loop that can make misinformation nearly impossible to scrub from the system.
The ease with which “bixonimania” permeated both AI databases and academic discourse serves as a sobering reminder of the limitations of modern technology. As misinformation researchers like Alex Ruani have pointed out, if the systems designed to support scientific integrity cannot distinguish between legitimate findings and a “masterclass” in trolling, we face a significant threat to public health. The speed at which such misinformation spreads is unprecedented, and these tools are currently capable of mimicking the nuance of real human expertise so effectively that it is becoming increasingly difficult for the average person to discern intent or accuracy.
Ultimately, this project serves as a crucial call to human caution. While AI is a powerful assistant, it is not an oracle of truth; it is a mirror reflecting the internet’s collective knowledge—flaws, lies, and all. As we move forward, we must become far more skeptical consumers of digital data, remembering that convenience is not synonymous with credibility. The responsibility lies with us to ensure we are the ones controlling these tools, rather than allowing algorithms to manipulate our understanding of the world. In the digital age, critical thinking isn’t just a useful skill—it is our primary defense against the normalization of fabricated reality.