You type a quick prompt, hit enter, and wait for the machine to do the heavy lifting. Lawyers across the country are learning the hard way that treating an AI text predictor like a competent junior associate is a fast track to professional ruin. The New Mexico Supreme Court recently handed down a five-thousand-dollar fine and a contempt citation to defense attorney Stephen Aarons. His crime? Submitting a high-stakes criminal appeal brief loaded with police testimony and witnesses that OpenAI's ChatGPT completely invented out of thin air.
If you think this is an isolated incident, you haven't been paying attention. Legal professionals keep trying to cut corners with artificial intelligence, and the court system keeps making public examples of them. But looking at this purely as a technology failure misses the point entirely. This is an indictment of professional laziness and a basic misunderstanding of how language models actually function.
The Mechanics of a Complete Fabrication
Aarons took on the murder appeal of Oscar Renee Sandoval, a man serving a life sentence for killing the mother of his children. Instead of slogging through dense trial transcripts like attorneys have done for centuries, Aarons fed computer-generated records into ChatGPT. He expected a clean, bulletproof summary of the proceedings.
What he got instead was creative writing disguised as jurisprudence.
The brief submitted to the state supreme court contained fictional statements, including fabricated details about what the shooter wore, alongside entirely imaginary witnesses. When the court caught the discrepancy and hauled Aarons in for a hearing, his defense boiled down to calling it an honest mistake and claiming he did not know AI could hallucinate.
Justice C. Shannon Bacon didn't buy the excuse. During the August hearing, she asked if he watched the news or read anything about the world, noting that lawyers relying on fabricated AI outputs has been a major headline for years. The justices rightly pointed out that his defense demonstrated a shocking lack of remorse and indifference toward his client's freedom.
Why the Legal Profession Keeps Walking Into the Trap
Lawyers are overworked, under immense billing pressure, and drowning in paperwork. The temptation to outsource tedious drafting to an instant generator makes logical sense on paper. You want to save time, reduce billable hour stress, and clear out your docket.
The problem is that ChatGPT does not know what is true. It does not know the law. It does not look up real cases or verify facts against a database unless specifically hooked into a verified retrieval tool. It predicts the next most likely token in a sentence based on statistical probability. If it needs a witness name or a specific quote to make your summary sound polished, it will invent one that sounds plausible. It sounds authoritative even when it is lying directly to your face.
Attorneys who skip human verification are violating basic professional competence rules. You wouldn't trust a random person off the street to draft your court filings without checking their work. Yet, professionals put blind faith in a glorified autocomplete because the output looks neat and professional.
How to Protect Your Practice From AI Disasters
If you practice law or work in any high-stakes professional field where factual accuracy dictates outcomes, you need strict guardrails. Do not treat modern AI models as truth engines. Treat them as unhinged brainstormers that require aggressive fact-checking.
- Never submit unverified text. If an AI writes a summary, a sentence, or a citation, you must manually check every single word against the primary source document.
- Understand hallucinations. Recognize that models fill gaps with fiction when they lack data. If you feed messy transcripts into a chat interface, expect the model to invent details to smooth out the narrative.
- Implement firm-wide policies. Every law firm needs explicit rules regarding where generative tools are allowed in the workflow. Drafting final filings or appeals without human verification should be grounds for termination.
The New Mexico Supreme Court referred Aarons to an attorney disciplinary board for further investigation. His client's appeal sits in limbo, delayed because an attorney wanted to take a shortcut. Stop treating AI shortcuts as harmless productivity hacks. Verify every fact, read every transcript yourself, and accept that technology will never replace the grinding work of actual legal analysis.
