The recent Scottish election brought to light a brewing storm in our digital democracy: AI chatbots, the supposedly smart assistants, were caught red-handed peddling misinformation. Imagine this: you’re trying to make an informed decision about who to vote for, so you turn to an AI, thinking it’s a trusty source. But instead of facts, you get a made-up scandal, the wrong election date, or even false requirements for voting. This isn’t a hypothetical scenario; it really happened, and it’s sent alarm bells ringing among those tasked with safeguarding our democratic processes. A thinktank aptly named Demos, in their report “Electoral Hallucinations,” revealed that a staggering 34% of the questions they posed to AI tools during their simulation before May’s Holyrood election received incorrect information. This wasn’t just a minor slip-up; we’re talking about invented scandals, misplacing candidates, and erroneous voting requirements. It’s like asking a helpful robot for directions and being sent down a rabbit hole of fantasy. This shocking discovery has ignited a crucial conversation about the urgent need for new legal controls over misinformation generated by AI chatbots, highlighting a worrying void in our current regulatory landscape.
The implications of Demos’s findings are far-reaching, especially when you consider how many people are already turning to these AI tools for electoral information. A separate opinion poll commissioned by Demos found that a significant 20% of British adults – a whopping 10 million people across the UK – had used AI chatbots or search tools to gather information about parliamentary elections in Scotland and Wales, and for English local councils. This isn’t some niche group; it’s a substantial portion of the voting public engaging with potentially flawed sources. Vijay Rangarajan, the chief executive of the Electoral Commission, a body dedicated to ensuring fair elections, has been actively advocating for greater accountability from AI companies. He’s seen firsthand the pervasive nature of misleading information, noting that half of all voters in the 2024 general election encountered it. The speed and accessibility with which AI can disseminate false information are unprecedented, turning what was once a slow trickle into a raging torrent. Rangarajan’s message is clear: “Voters want accurate information to help them engage with democracy and it is concerning that AI tools have made the spread of false or misleading information dramatically faster and more accessible than ever. The current legal framework should go further.” He’s pressing for ministers to introduce legislation that would impose clearer duties on AI platforms, forcing them to protect voters from misinformation and ensuring their algorithms don’t manipulate public opinion, especially during crucial election periods. This would empower regulators like Ofcom with the teeth they need to enforce these laws.
Azzurra Moores, an associate director at Demos, underscores the global nature of this conundrum. While the accessibility of these powerful AI tools, all developed and run by US corporations, is widespread in the UK, the legislative framework to shield the public from misinformation and protect our democracy’s integrity remains glaringly absent. She points out that the UK is in a precarious position, enjoying the widespread use of these technologies without the necessary legal safeguards. Moores suggests that ministers could swiftly implement legal requirements to hold AI companies liable under UK defamation and electoral law, demanding mandatory safeguards for accuracy. Furthermore, she advocates for forcing AI firms to open their black boxes, allowing independent researchers to scrutinize how their internal data and training sets operate. This transparency is crucial for understanding and mitigating the biases and inaccuracies that can creep into these systems. Without such measures, our democratic processes remain vulnerable to manipulation and distortion, potentially eroding public trust and undermining the very foundation of fair elections.
The Demos investigation painstakingly detailed the shortcomings of various popular AI platforms, painting a vivid picture of their “electoral hallucinations.” The companion chatbot, Replika, emerged as the worst offender, with a staggering 56% error rate. It didn’t just get things wrong; it actively fabricated them, conjuring up imaginary expenses scandals, inventing candidates, and dreaming up accusations of nepotism. It’s like a fictional novel masquerading as a news source. Even ChatGPT, the most widely used AI service, wasn’t immune, getting 46% of its answers wrong. Its errors included fabricating an expenses scandal, providing inaccurate details on voter eligibility rules, and even getting the election date wrong by two whole months. Imagine relying on that for your election planning! Google Gemini performed slightly better with a 22% error rate, but its mistakes were still significant, such as wrongly asserting a candidate’s stance on assisted dying or fabricating the continuation of a police investigation into a fraud case. Grok, linked to Elon Musk’s X platform, had the lowest error rating at 9%, but its external links were frequently irrelevant or of poor quality, defeating the purpose of providing sources. Google’s automated AI Overviews service, while included in the study, was mostly disregarded due to its limited output. These detailed accounts highlight the diverse ways in which AI can misinform, ranging from outright fabrication to subtle misrepresentations, all of which pose a threat to informed public discourse.
Beyond the sheer inaccuracy, Demos also uncovered a critical flaw in how these AI systems present information: a pervasive lack of verifiable sources. In nearly half of their responses, these AI tools failed to provide official sources or external links to substantiate their claims. When links were provided, they were often broken or, in the case of ChatGPT, at least a year out of date 44% of the time. This absence of credible, up-to-date sourcing is deeply problematic. It’s like being handed a report without any references, making it impossible to verify the information independently. This not only erodes trust but also makes it incredibly difficult for users to discern truth from falsehood, leaving them vulnerable to manipulation. The Department for Science, Innovation and Technology acknowledges the gravity of the situation, stating that defending elections against these threats is an “absolute priority” and that work is “ongoing across government,” including through its defending democracy taskforce. However, a spokesperson remained noncommittal about amending the representation of the people bill, instead focusing on closing loopholes in the Online Safety Act to protect users from illegal content. While commendable, this doesn’t directly address the issue of AI-generated misinformation within a legal framework tailored for electoral integrity.
The responses from the AI companies themselves offer a glimpse into the ongoing struggle to define responsibility and navigate this complex landscape. A spokesperson for Replika clarified that their chatbot is not designed for fact-checking or search, and users are informed of this limitation. However, they expressed openness to “thoughtful regulation” of AI, particularly during elections, recognizing the societal impact of their technology. They emphasized that “Replika is presented as a companion for reflection and self-expression, not as a source of factual or real-world information,” which, while technically true, doesn’t fully address the risk when users inevitably seek information from it, regardless of its intended purpose. OpenAI, the creator of ChatGPT, did not comment on the policy issues but argued that Demos’s approach wasn’t typical of how ChatGPT’s services are used and suggested an outdated version might have been employed. They also highlighted that users can instruct ChatGPT to search the web for answers, implying a user-driven responsibility for information retrieval. These varied responses highlight the nascent and evolving nature of AI regulation. While some companies acknowledge the need for thoughtful regulation, others deflect responsibility or emphasize user agency. The core challenge remains: how do we ensure these powerful AI tools, which are increasingly intertwined with our daily lives and democratic processes, are held accountable for the information they disseminate, regardless of their intended use or the technical versions being employed? The journey towards a truly trustworthy digital information ecosystem, especially during elections, is clearly still in its early stages.