AI Hallucinations in Court: How to Respond Whether You Found One or Filed One

AI hallucinations in legal filings: Your obligations when you’ve filed one

Let’s start with the situation nobody wants to be in. You’ve submitted a filing, and it contains citations to cases that don’t exist or cases that don’t say what you’ve cited them for. The legal liability for AI hallucinations falls squarely on the signing attorney, whether the error is in a brief, a motion, or any other paper submitted to the court. That’s true under the ethical duties that already apply to AI in legal practice, and courts are enforcing it.

Before you even get to the filing stage, you have Rule 1.1, the duty of competence. Rule 3.3: a duty of candor toward the tribunal. Rule 5.1, the duty of supervision. And Rule 5.3, the duty of supervision for everyone on your staff who is not licensed, including AI. You need to make sure that you have read the cases and verified them before you submit any filing.

If you’ve submitted a filing with hallucinations, the requirement is that you amend it. You have a duty to notify the court. Sometimes it might require the court’s permission to amend, but you cannot leave it and hope that nobody finds it. Many courts have safe harbor provisions built into their rules so that you can issue corrections.

It’s good practice to apologize to the court and to show how you will prevent it from happening again.

The Sullivan & Cromwell model

Sullivan & Cromwell is one of the best examples of how to handle this. When a filing from their team included hallucinated citations, the attorney who signed it took full ownership. He didn’t blame it on an associate, or a tool, or anything. He said he was the signer of the document. It was ultimately his responsibility. He apologized to the court and talked about what steps the firm would be taking to reduce the likelihood of this happening again.

That really was the model approach. The story was newsworthy for a day, and then it was over.

What happens when you don’t own it

By contrast, there are several cases where the court identifies hallucinated citations and the lawyer, rather than owning it, tries to artfully dodge responsibility. There’s a case in the New York Supreme Court where a lawyer tried to deflect, arguing that the real issue wasn’t the veracity of the citations but the conduct of the defendant. Others try a more creative defense: Yes, the case may not exist, but the underlying legal principle is correct, so the hallucination is harmless. Courts have not been receptive to this argument.

Every time that happens, the judges bring the hammer down. When law firms own the mistake and correct it promptly, the courts are far more lenient. When they deny responsibility or dodge it, the consequences escalate sharply.

Court sanctions for AI hallucinations: What judges look for and what makes it worse

Courts are starting to look beyond the hallucination itself. The legal implications of AI hallucinations now extend well past the citation error. Judges want evidence that you understood the tools you were using and that you had systems in place to catch errors.

Specifically, judges are looking for evidence of two things. First, knowledge of the strengths and weaknesses of these tools. Did you take CLE on this? Second, does your firm have a policy around AI, and did your conduct comply with that policy?

Part of taking responsibility is showing that you understood the risk, how the error could have been prevented, and how it will be prevented in the future. Judges want to see lawyers acting responsibly, and that’s the evidence they’re seeking.

We haven’t really seen it yet at scale, but courts will certainly start asking lawyers to show that they did due diligence on their tools and that those tools are grounded in real law. When a hallucination surfaces, you want to be able to demonstrate that you used tools grounded in the law of the jurisdiction you’re practicing in, instead of relying on general-purpose AI to generate citations it can’t verify.

Withers v. City of Aberdeen: What happens when nobody takes responsibility

A recent case out of the Northern District of Mississippi shows what the worst-case scenario looks like. A law firm in Texas and a law firm in Louisiana were litigating in federal court in Mississippi, both represented by outside counsel with local counsel in Mississippi.

One party made a filing to the court, prepared by their out-of-state counsel. Local counsel signed off on it and submitted it. The filing contained hallucinated citations. The opposing party, also based outside Mississippi, drafted an opposition with hallucinations of its own. Their local counsel signed off on it too.

When the court reviewed the filings, it found hallucinations on both sides. The judge didn’t dismiss the case, but she dismissed every lawyer attached to it, removing all four and ordering the clients to find new counsel.

The pro hac vice lawyers were fined and banned from appearing in that district for two years. The local counsel weren’t fined but were referred for disciplinary action.

The sanctions in this case weren’t uniform, and the differences reveal what courts treat as aggravating factors:

• Using a tool you knew was wrong for the job. One pro hac vice lawyer was using a tool made for in-house counsel that was trained on Texas law, not Mississippi law. The judge sanctioned her in part because she knew the tool was inappropriate for the case and went ahead with it anyway.
• A pattern of repeatedly filing hallucinations. The other pro hac vice lawyer used a tool called First Draft. This wasn’t the only case in which she had submitted hallucinations. The judge pointed to a separate bankruptcy court matter where this lawyer was being asked to show cause for hallucinations in her filings, after she had already been notified about the hallucinations in the Mississippi case. The judge characterized this as either a pattern of abuse or a willingness to accept the risk without doing due diligence. She made an empty apology to one court and then did the same thing several weeks later in another. That’s someone who hasn’t learned the lesson. She received the highest fine of all four.
• Courts catching it before you do. In the Mississippi case, it was the judge herself who flagged the suspect citations, not opposing counsel. A startup called Learned Hand, now being used by several federal and state judges in Texas and California, evaluates filings for hallucinations before they reach the courtroom. Courts are also using page limit orders as a blunt check on AI-generated filings, capping submissions at 10 pages when they know AI can produce 200 without effort. The window between filing a hallucinated citation and getting caught is shrinking.

What to do when you find AI hallucinations in opposing counsel’s filing

Now flip the scenario. You’re a diligent lawyer, you’re using your tools responsibly, and you receive a filing from opposing counsel that contains hallucinated citations.

A lot of litigators would look at this as a gift-wrapped mistake. An opportunity to gain ground in front of the judge. But more often than not, taking that approach is a mistake.

Here’s why. What you’re doing is diverting the litigation from the actual cause of action. Instead of arguing the substance of the case, you’re arguing about tool usage and citations. Unless your litigation strategy is to delay proceedings and run up a bill, there’s no reason to throw the case off the rails over a hallucinated citation.

Notify opposing counsel first

The better approach is to notify opposing counsel. Reach out, identify the citations that don’t check out, and give them the opportunity to file an amended document. There are many reasons errors end up in filings, and not all of them are AI-related. This is in line with court rules that encourage lawyers to resolve procedural issues without involving the judge. Civility is not a breach of your duty of zealous advocacy, and courts treat parties better when they treat each other with respect.

If you try to exploit the mistake by hammering opposing counsel before the court first, the judge’s first question will be: Why are you bringing this to me? Have you talked to them about it? If the answer is no, the court is likely to do what a phone call would have accomplished: order them to amend.

We heard from a president of a bar association who identified hallucinated citations in an opposing filing and reached out as a courtesy. The other lawyer reacted poorly, denied using AI, and accused her of acting in bad faith. She held her ground and suggested he verify for himself. A day later, he confirmed the citations were hallucinated and refiled the brief, without acknowledgment. Even in that interaction, courtesy was the right call. The legal profession is a small community, and civility is how lawyers continue to work together when they move on to the next case.

Even court rules support notifying opposing counsel first. FRCP Rule 11(c)(2) states that a motion for sanctions due to misrepresentation to the court, i.e. hallucinations, must be served on the offending party. Filing the motion with the court is delayed. The served party has a 21-day grace period to either withdraw or appropriately correct an erroneous submission. California’s Code of Civil Procedure has a similar rule. Florida’s rules include a duty to confer with opposing parties prior to filing non-dispositive motions. 

When it’s a pro se litigant

The situation gets harder when the hallucinated citations come from a self-represented litigant who may not know how to amend properly or recognize hallucinations in future work. You probably need to raise this before the court, but as a heads-up rather than a motion, asking whether the court can provide instruction on appropriate materials for this litigant to use. The first response should be education, not punishment, unless there’s a pattern of continued abuse after notice.

Don’t wait to build your process

As AI hallucination legal cases increase, judges are also expecting you to look for hallucinations in the material you’re receiving from opposing counsel. When it comes to determining whether attorney’s fees should be awarded, courts are looking for the other side to have raised these issues.

Document now that you have a procedure in place to identify hallucinations in filings you receive. Document that you took steps to notify and correct. If the judge raises a sua sponte motion or order to show cause, you want to have that email you sent to opposing counsel confirming you noticed the issue and gave them a heads-up. That record allows you to argue for attorney’s fees, even if you didn’t file a motion to show cause.

The legal implications of AI hallucinations for your firm

Whether you filed the hallucination or found one in opposing counsel’s brief, the through line is the same. Before you file a paper, there is a step where you verify and validate all the citations. This obligation has always existed. AI is making it more important than ever.

In the past, we moved this supervision step into hiring. We hired people from good law schools, or who had credentials that told us they weren’t going to cite things incorrectly, and we didn’t supervise that final step in quite the same way. AI is forcing us to recenter that obligation at the end of the process, where it belongs.

Maybe this is a proofreading clerk that firms hire to do exactly this. Maybe it’s something that senior partners or senior associates need to reacquaint themselves with. But every citation in every paper, 100% of the time, needs to be accurate, true, and verified to exist in the real world. And always include a hyperlink. Hyperlinks, or it didn’t happen.

That’s why we designed Clio Work and Vincent by Clio, for Enterprise, and the way we did. They’re designed to be highly verifiable. All the authorities are hyperlinked. You can see in the side panel where they came from, pull out the most relevant paragraph and the full text that was actually cited. With one click, you can read it and validate it for yourself.

The goal is to make validation as frictionless as possible, so lawyers can be as thorough as the rules require. The legal liability for AI hallucinations isn’t going away, on either side of the v. But the answers start in the same place. Verify, take responsibility, and make sure the tools you’re using are worthy of the trust you’re placing in them.