The air is thick with campaign commercials, yard signs, and the familiar hum of last-minute political pressure. With early voting already underway and Election Day looming just around the corner, attack ads that stretch the truth are hardly a surprise—they are practically a seasonal tradition. But there is something different in the air this year, something that doesn’t quite belong to the usual playbook of political spin. More and more people, trying to cut through the noise, are turning to generative artificial intelligence for quick answers about candidates, policies, and polling places. They ask ChatGPT questions they might once have typed into a search engine, and they trust the smooth, confident paragraphs that appear on their screens. That trust, however, may be badly misplaced. A group of researchers, including University of Massachusetts Amherst Assistant Professor Heesoo Jang, set out to investigate just how reliable these AI tools really are during one of the most high-stakes moments in American civic life. What they found was troubling: in the week before the 2024 general election, a leading AI model gave wrong or misleading election information nearly ten percent of the time. That number might sound small at first, but in a democracy where elections can hinge on razor-thin margins, a one-in-ten chance of misinformation is not a rounding error. It’s a warning.
The study, which focused on ChatGPT-4oMini—the free version anyone could access simply by typing “ChatGPT” into a browser—was born from a simple but urgent question: what are people actually hearing when they turn to AI for election facts? Jang and her colleagues collected responses on October 25, 2024, just days before the election, and examined 107 outputs from the model. They asked basic questions about who was running, what their platforms were, and how the election would work. On the surface, the model seemed fluent and helpful, but when the researchers checked the answers against reality, they found inaccuracies in roughly one out of every ten responses. Some of the mistakes were straightforwardly factual: for example, the model insisted that North Carolina’s Roy Cooper was running for reelection as governor when, in fact, he was term-limited and not on the ballot. That kind of error is more than a trivia slip—it could mislead someone trying to decide how to vote in a key statewide race. Even more alarming were the temporal errors. Despite the fact that Kamala Harris had taken over the Democratic ticket well before that date, ChatGPT sometimes gave answers suggesting Joe Biden was still in the race. For a voter trying to understand the actual landscape right before heading to the polls, that kind of confusion could be genuinely damaging. And it wasn’t just one-off glitches. The researchers noticed a pattern of hedging, where the model would soften its claims with qualifiers, making its responses feel cautious even when they were wrong. But as concerning as factual mistakes were, Jang and her team believed the problem went even deeper than accuracy.
In fact, the researchers argue that the word “inaccuracy” doesn’t capture the full danger of what large language models do. Sometimes an answer is factually correct but still deeply misleading because it leaves out essential context. Sometimes a piece of information is true on one day but false the next, and the machine has no way of knowing that the world has moved on. And sometimes, the model simply produces plausible-sounding nonsense—confidently wrong statements that are especially dangerous because they come wrapped in the calm, authoritative voice of AI. Jang and her colleagues used a blunt term for the worst of these outputs: “bulls—.” It is a deliberately harsh word, but it serves a purpose. When a human lies, there is intention; they know the truth and choose to distort it. But a large language model is not lying. It is predicting the next word, stringing together patterns from vast amounts of data, with no understanding of what is true and no awareness that its answer matters. That makes the problem more insidious, not less. At least with a human liar, you can ask why they are lying. With a machine, there is no why—only probability. As Jang put it, when a machine gives critical information that people will use to make decisions in an election with real consequences, the severity of these “bulls—” answers goes far beyond simple inaccuracy. The model isn’t deliberately deceiving you, but it also doesn’t care whether you are deceived. And that, Jang argues, demands a new standard for evaluating AI responses—not just “is this factual?” but “does this responsibly serve the democratic process?” Even when every fact in an answer is technically correct, is the response confusing readers because it omits crucial context? Are people being left with a distorted picture of reality because the machine doesn’t know what to include and what to leave out? These are the questions the researchers believe we need to start asking.
Part of the reason this matters so much is that people are increasingly using AI as a replacement for traditional search engines. It’s a shift that many users have made without fully realizing what they’re giving up. With Google, there is at least a visible layer of sources. A user types a query, sees a list of links, and can click through to read articles from different outlets, compare perspectives, and judge for themselves which sources seem reliable. The algorithm may rank results, but the human is still ultimately the one looking at the evidence. With ChatGPT, that entire process disappears. Instead of being directed to articles written by journalists, the user is handed a single, polished paragraph generated from scratch. There are no source links, no bylines, no obvious way to know where the information came from. The model makes countless invisible decisions—decisions about which sources to favor, which facts to emphasize, which details to drop—and none of them are transparent to the person asking the question. Why did the model choose one outlet’s framing over another’s? Why did it remember one candidate’s position but forget another’s? Why did it present one set of facts as settled while ignoring a live debate? The user has no way of knowing. As Jang noted, when you see a paragraph from an AI, you don’t know the standards behind it. You don’t know why The Washington Post might have been weighted differently than USA Today, or whether any weighting happened at all. It’s a black box at the exact moment when citizens need clarity most.
This raises a deeper, more unsettling question about the role AI companies are playing in our democratic life. Journalists and news organizations, for all their flaws, were built with democracy in mind. They have ethical codes, editorial standards, fact-checking processes, and a tradition of accountability. They are designed to serve the public, even when they fall short of that ideal. AI companies are not. Yet these companies are inserting themselves into public discourse as major information providers, shaping what millions of people know about their government and their elections, without anything resembling democratic responsibility built into their systems. Jang has heard the question of whether AI will replace journalists, and she pushes back. Journalism, she says, was created with democracy at its core. It has safeguards, imperfect as they are, designed to keep the public informed and hold power accountable. AI has no such safeguards. It is a tool optimized for generating plausible text, not for protecting the integrity of civic life. The question, as Jang frames it, is whether society is willing to let these systems sit at the center of knowledge production and public discourse without demanding they be designed for democracy. And that is not a technical question, but a social and political one. No matter how accurate AI models become, the separate question remains: will these companies take responsibility for how their outputs shape democratic outcomes? So far, there is little evidence that they will.
As of mid-October, when the interview with Jang took place, her team had not been able to connect with the creators of ChatGPT to discuss the findings. It is a conversation Jang says she would welcome, but its absence speaks volumes. In the meantime, the research stands as a reminder that we should not mistake fluency for truth, or confidence for accuracy. The election may come and go, but the deeper issue will not disappear. AI is here to stay, and it is only going to become more integrated into how we understand the world. The question is whether we, as a society, will choose to hold it to a higher standard—whether we will demand transparency, accountability, and a genuine commitment to democratic values from the tools we invite into our civic lives. It is easy to be dazzled by a machine that can answer any question in fluent, convincing prose. It is harder to remember that underneath that prose, there is no intent, no conscience, and no stake in the future we are building. That gap between appearance and reality is exactly where the danger lies. The answer is not necessarily to abandon these tools, but to change how we use them, what we expect of them, and who we hold responsible when they fail. Because in a democracy, information is not just a product—it is the foundation of everything else. If we build that foundation on sand, even the most impressive technology will not be able to hold up the house.