In a country renowned for its lightning-fast adoption of new technologies, South Korea finds itself at the forefront of a modern-day battle: the fight against AI-generated disinformation in elections. It’s a story of rapid technological advancement meeting the bedrock principles of democracy, where dedicated individuals are standing guard on the digital frontier. In airy offices just south of Seoul, teams of social media monitors are working tirelessly, sifting through the ever-growing deluge of online content, trying to distinguish genuine political discourse from cleverly crafted AI fabrications designed to sway public opinion. This isn’t just a technical challenge; it’s a human one, fraught with the immense pressure of protecting electoral integrity in an era where the lines between truth and illusion are blurring at an alarming rate.
The stakes are incredibly high, as experts globally warn that the increasing sophistication and affordability of AI models are accelerating the spread of online deception. For South Korea, a nation that has embraced AI with unparalleled speed, this issue is particularly acute. Recognizing the profound threat, the government took decisive action in 2023, strengthening electoral laws specifically to counter the misuse of AI. This legislative muscle has been backed by a significant human investment: hundreds of staff have been hired, charged with the crucial task of tracking and neutralizing manipulated content ahead of local polls scheduled for June 3rd. Yet, despite these efforts, those on the front lines openly admit that it often feels like an uphill battle, a relentless struggle against an omnipresent and rapidly evolving adversary.
Take Choi Ji-hee, a disinformation monitor at the National Election Commission (NEC) headquarters in Gwacheon. Her voice carries the weight of someone intimately familiar with the challenge. She describes how the technology’s evolution is almost palpable: “We can literally see how fast this technology evolves — like how each new version of AI makes videos and audio look and sound even more convincing.” This constant upward trajectory of AI’s capabilities means her job, and that of her 18 colleagues, is perpetually getting “harder and harder.” Their daily routine involves a meticulous deep dive into the vast ocean of online platforms – Instagram, YouTube, chatrooms, even the dedicated “fan clubs” of local politicians – all in pursuit of content concocted by AI. Their recent discoveries paint a vivid picture of the ingenuity of these fabrications: a fake TV news report that deceptively claimed a mayoral candidate had made Time magazine’s prestigious list of rising political leaders, and a slick, AI-produced K-pop song that not only praised one politician but cunningly mocked his rivals. Once such content is definitively identified as AI-generated, authorities have the power to demand its removal, with severe consequences, including potential jail time in the most egregious cases.
In the NEC offices, the scene is one of focused determination. In one corner, a group of workers intensely discusses a suspicious video, meticulously considering whether to dissect its various components: the audio, key frames, facial images, and background footage, each piece a potential clue. Nearby, data analyst Kim Ma-ru is engaged in a different but equally crucial task. He’s mapping out the distribution networks of fake materials – precisely where, when, and by whom they were spread. This strategic intelligence helps Choi’s team to more quickly identify and respond to dubious content, acting as an early warning system against the spread of misinformation. He refers to their work as an “exhausting job that can feel like a (game of) whack-a-mole,” a description that perfectly encapsulates the endless, repetitive nature of stamping out one piece of misinformation only for another to pop up elsewhere. Yet, despite the inherent frustration, Kim’s dedication shines through, driven by a profound sense of civic responsibility. He knows the importance of their work, recognizing that the integrity of their elections, and ultimately their democracy, depends on it.
Indeed, the local polls in South Korea represent the third major election since the landmark 2023 law was enacted to combat AI-fueled electoral falsehoods. The scale of AI adoption in South Korea is staggering: over 45% of its population uses generative AI, and OpenAI, the creator of ChatGPT, reports that South Korea has the highest number of paid subscribers outside the United States. This rapid embrace, however, comes with a downside. South Koreans also consume more “AI slop”—low-quality, AI-generated content—than any other nation. Disturbingly, reports of false AI-created content surged 27-fold between the 2024 general election and the following year’s presidential campaign, painting a clear picture of an escalating problem. Beyond simple misinformation, the nation has grappled with deeply damaging conspiracy theories, such as those alleging vote-rigging, which have severely eroded public trust in the electoral process. These anxieties manifested dramatically when jailed ex-president Yoon Suk Yeol, during a short-lived attempt to impose martial law in late 2024, sent armed troops to the NEC, echoing widely debunked far-right claims of vote hacking. The lingering impact of these events is visible even today, with pro-Yoon protesters maintaining a banner outside the NEC offices demanding an investigation into “rigged elections.” The human cost of these battles is also evident: both Choi and Kim, fearing growing threats and online bullying, declined to be photographed or filmed, a stark reminder of the personal risks faced by those protecting democratic institutions.
Jung Hui-hun, a digital forensic specialist in the NEC’s cyber investigations unit, candidly admits the rapidly closing gap between reality and AI-generated deception. “In such a short time, it has become so difficult for voters to tell what is real and what is not,” he observes, as he meticulously runs suspicious videos through state-developed software designed to detect AI imagery. These sophisticated programs boast an accuracy rate of approximately 92%, with human experts stepping in to analyze the most intricate and convincing fakes. Once a violation is confirmed, authorities compel either the content creator or the platform to remove the material, in accordance with the 2023 law. This stringent legislation bans AI-generated content involving candidates that appears realistic enough to confuse voters within the critical three-month period before an election. The penalties are severe: repeat offenders, or those who create particularly harmful content, could face up to seven years in jail or a hefty fine of 50 million won (approximately US$34,000). Kim Myuhng-joo, director of the Korea AI Safety Institute, acknowledges that such strict regulations might appear excessive to outsiders, particularly in countries like the US that prioritize freedom of expression. However, he explains that South Koreans, having quickly embraced AI, also swiftly became aware of its inherent dangers. He points to the election conspiracy theories and a public scandal involving deepfake pornography targeting women and girls as catalysts that shaped public opinion. “Public consensus has formed that we need tough regulations over the use of AI when it comes to election transparency,” Kim asserts. This sentiment is strongly supported by the public, with a substantial 75% of South Koreans in a recent survey believing that AI-generated content could sway election results, and nearly 80% advocating for stronger efforts to detect and punish its misuse. While Jung, the digital forensic specialist, recognizes the limitations of the current response, he remains hopeful that South Korea’s pioneering efforts will spark a broader, global conversation on how to effectively address the challenges of AI-fueled disinformation. “We’re still trying to figure out what is the best solution… but I think we are moving forward—slowly but surely,” he concludes, encapsulating the measured optimism and unwavering commitment of those battling on the digital front lines.