ChatGPT invented fake police testimony in murder appeal, New Mexico high court says

WASHINGTON, Sept 11 : A defense lawyer appealing his client's murder conviction submitted a brief containing made-up police testimony and witnesses fabricated by OpenAI's ChatGPT, New Mexico's highest court said.

The New Mexico Supreme Court on Wednesday fined the attorney, Stephen Aarons, and held him in contempt for failing to verify the accuracy of the court filing, which Aarons said he prepared with help from the AI program.

The filing “contained false testimony from wholly fabricated witnesses,” the court said.

The panel also said Aarons had “demonstrated a lack of remorse and a lack of concern for his client.” The justices fined Aarons $5,000 and said they will refer him to an attorney disciplinary board for investigation.

Aarons in a statement to Reuters said he had used ChatGPT to summarize the trial proceedings when he agreed to take up the defendant's appeal last year, and did not understand the degree to which AI could "hallucinate" facts.

"I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake," he said. "It is a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology."

OpenAI did not immediately respond to a request for comment.

The court’s sanction is the latest in a growing number of cases in which state and federal judges have disciplined lawyers for submitting court documents generated by AI tools without adequately vetting them.

Dozens of lawyers have been sanctioned for filing briefs where AI made up case ​citations or misquoted the law. Aarons’ filing appears to have gone further, containing fabricated witness testimony in a criminal appeal.

Aarons, a private attorney based in Sante Fe, was handling the appeal of Oscar Renee Sandoval, who pleaded not guilty to the murder of the mother of his children before he was convicted and sentenced last year to life in prison.

The appeal is still pending and was assigned on September 2 to Kim Chavez Cook, a New Mexico public defender. Cook declined to comment.

The district attorney's office for Doña Ana County also declined to comment.

The state supreme court last month ordered Aarons to explain how the fabricated material, which it said appeared to include "fictional statements that the shooter was wearing dark pants and a white shirt," was included in his primary brief in the appeal. 

Aarons told the court at an August 21 hearing that he fed a computer-generated transcript and other case materials to ChatGPT, presuming it would generate "a bulletproof summary." 

The justices sounded incredulous that Aarons was not fully aware of how AI can make mistakes.

"Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?" Justice C. Shannon Bacon said at the hearing. "Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”