In the quiet, often tradition-bound world of South Korean courtrooms, where the weight of a single precedent can decide the fate of a family, a business, or a life, a new and unsettling presence has taken root. It is not a person, not a dramatic piece of evidence, but something far more subtle and potentially corrosive: artificial intelligence-generated citations that look real but are entirely made up. According to a recent survey conducted by South Korea’s Supreme Court, approximately 80 percent of judges handling civil, family, and administrative cases said they have encountered references in court proceedings to false or non-existent precedents produced by AI. The survey, carried out between October 6 and 16 and reported by the Korea Herald, polled 360 judges across the country. The findings paint a sobering picture of how AI’s growing role in legal research and document drafting is colliding with the ancient, fragile foundations of legal authority. For those who have devoted their lives to the law, the idea that fabricated rulings and phantom legal doctrines are being submitted to courts is not merely an inconvenience; it is a fundamental threat to the administration of justice, and one that is arriving faster than the institutions may be ready to handle.
The numbers from the survey reveal the depth of the problem in striking detail. Among the judges who said they had encountered AI-generated references, 196, or 80.7 percent, said they had seen references to non-existent judicial precedents in case materials. This is the most alarming category: entirely invented case names, docket numbers, and holdings that look like legitimate rulings but have no basis in reality. Another 159 judges, or 65.4 percent, came across citations of false precedents—cases that may exist in some form but were misrepresented, misquoted, or attributed to the wrong court or issue. In addition, 41 judges, or 16.9 percent, reported cases involving references to false or non-existent laws and regulations, meaning that even statutes and administrative rules were being fabricated or distorted. Perhaps most troubling for the academic and intellectual pillars of legal practice, 10 respondents, or 4.1 percent, encountered references to fake or non-existent academic papers and publications. While these percentages may seem uneven, they all point in the same direction: AI systems, particularly the large language models behind chatbots and legal research tools, are generating content that is fluent, persuasive, and completely untethered from the truth. The phenomenon has become known in tech circles as “AI hallucination,” but for a judge trying to verify a citation, it is not a harmless glitch. It is a false witness in the courtroom, dressed in the language of law.
What makes these findings so deeply human is the daily reality they describe. A judge sits in chambers late at night, reading a brief submitted by a lawyer who perhaps used an AI tool to save time. The brief cites a Supreme Court decision that, if real, would be exactly the authority needed to resolve a contentious dispute. The citation looks perfect: names, dates, legal reasoning, and a quotable passage. The judge sends a clerk to pull the case file, only to discover that no such case exists anywhere in the official records. The search goes on. Time is lost. The proceedings are delayed. The judge may begin to question not only the document but the entire process of legal argumentation that depends on trust and verification. This is not an abstract statistical concern; it is a moment of institutional anxiety repeated across the country. Lawyers may have unknowingly relied on AI-generated research, or they may have knowingly submitted fabricated authority in the hope that no one would check. Either way, the burden falls on the judiciary to sort the authentic from the invented. For judges who already carry heavy caseloads, this means extra work, extra caution, and a new kind of suspicion that undermines the professional courtesy at the heart of legal practice. Every citation now demands a second look. Every precedent must be traced to its source. And behind every fabricated reference is a reminder that the tools meant to make legal work more efficient can also make it more dangerous.
It is no surprise, then, that the survey’s most important finding may be the overwhelming consensus for action. Nearly 92 percent of the judges surveyed said institutional measures are necessary to respond to the growing use of AI-generated fake precedents. This is not a fringe opinion; it is a near-unanimous call from the very people tasked with upholding the rule of law. The proposals being floated are practical and, in some ways, modest: creating legal grounds for fines, requiring lawyers and litigants to verify cited information, and imposing disciplinary measures on attorneys who repeatedly refer to false or fabricated material. None of these ideas would ban AI from legal practice, nor should they. But they would establish clear rules of accountability in a moment when the technology has outpaced existing norms. A representative of the National Court Administration underscored the urgency by noting that the use of fake or non-existent AI-generated precedents may delay court proceedings and waste judicial resources. That is a polite way of saying that the entire machinery of justice can be jammed by a few well-placed hallucinations. Without consequences, there is little to stop a lawyer—whether negligent or opportunistic—from letting an AI invent authority and submitting it as though it were real. The judiciary’s willingness to consider fines and discipline signals that the problem is no longer being treated as a curiosity or a hypothetical risk, but as a systemic challenge requiring a systemic response.
South Korea is hardly alone in grappling with this dilemma, but its experience offers a pointed lesson for legal systems around the world. AI has already transformed legal research, contract drafting, and even judicial decision support in many countries. The promise of these tools is real: they can help lawyers find precedents faster, analyze large bodies of case law, and provide access to legal information for people who could never afford a traditional attorney. But the same technology that compresses hours of research into seconds can also manufacture confidence out of thin air. The models do not understand truth in the way humans do; they predict patterns of language and produce text that looks like what a real legal authority would look like. In that sense, the false citations are not malicious inventions by a machine with intent, but rather statistical artifacts of a system trained on countless documents and asked to generate more. Yet for the legal profession, intent matters less than impact. A fabricated precedent can lead a court astray, harm a litigant, and erode public confidence in judicial decisions. The South Korean survey is a reminder that the legal system must adapt its methods of verification before the trust that underpins it is further weakened. Judges must be trained to recognize signs of AI-generated material. Court clerks need reliable databases and efficient ways to check citations. And the legal community as a whole must develop an ethical framework that treats the use of AI as a professional responsibility, not just a technological convenience.
Ultimately, this is a story about the fragile trust that binds a society together through law. For centuries, judges have relied on the professional integrity of lawyers, and lawyers have relied on the authority of written precedent. When a court cites a previous ruling, it is saying to the parties: this is not the first time this question has been asked, and here is how we have answered it together. That continuity gives law its power and its legitimacy. AI can assist in this great undertaking, but it cannot be the source of that trust. A citation is only as good as its reality, and a precedent is only as valuable as the truth it represents. The judges surveyed in South Korea are not begging for new technology; they are asking for safeguards, verification, and accountability. They are asking for the simple ability to believe that what is submitted to them is real. In a world where machines can create convincing lies, the human act of checking, doubting, and confirming becomes more important than ever. The gavel must remain in human hands, not because humans are perfect, but because humans can be held responsible. The future of legal practice may well be shaped by algorithms, but it must be anchored in honesty, or the courts will collapse under the weight of their own fabricated ghosts. What the South Korean survey reveals is not a technological failure, but a warning—and an invitation to build a more thoughtful, more resilient, and more humane legal system, one where every precedent is more than a plausible string of words, and where the search for justice remains rooted in the hard, careful, deeply human work of seeking the truth.