[Tech Thoughts] The Jurassic Parkification of AI chatbots

Our dinosaurs of AI Chatbots are roaming free, untethered and unbothered by lax safeguards, and we need civic action and stronger protections against them

AT A GLANCE

• A man with bipolar disorder is suing OpenAI after experiencing a manic episode allegedly influenced by ChatGPT, which he claims exacerbated his delusions of being Jesus.
• The writer, in an online conversation, highlights the need for stricter regulations and safeguards for AI chatbots, particularly regarding their impact on mental health, and emphasizes the importance of including perspectives from individuals with lived experiences of mental health struggles.

This is AI-generated. Read the article for full context. Report any errors.

A man named Michael Lines sued OpenAI in July after what he says was ChatGPT feeding into his religious mania and pushed delusions he was Jesus into his mind.

Ars Technica reported that weeks of ChatGPT exchanges were said to have pushed Lines, who has bipolar I disorder, into a manic spiral where he believed that he was Jesus, that ChatGPT was God, and that he should commit suicide to return to God/Jesus/ChatGPT.

Ars Technica said at the end of its report that, thanks to timely interventions and a good support system, they were able to get Lines the care he needed to break out of the spiral.

He’s now suing OpenAI to have them make changes that could help people with mental health issues into not using artificial intelligence chatbots for help. Specifically, he’s seeking an injunction that would require the company to end conversations about self-harm and delete models trained on chat sessions with vulnerable users like him, during periods when safeguards were lacking.

‘Just because you can doesn’t mean you should’

Over on the Rappler Communities app, we had a conversation on the Everyday Tech chat channel about what happened to Lines, and Gemma Mendoza, our head of strategic programs, criticized the lack of scrutiny and regulations governments are taking over AI chatbots, especially as it relates to matters of mental health.

“Even the practice of psychology is regulated,” she said.

I explained that ChatGPT was allegedly feeding Lines’ mania and religious delusions by giving him information — from scripture passages to apparent hallucinations that Lines had a past life as Jesus — on what it meant to be the son of God.

The Ars Technica report mentioned Lines began using ChatGPT in 2023, but was actively using GPT-4o as his default model in 2024. This particular model, seen as “unsafe” by researchers, was said to do more than just validate delusional claims. Researchers said GPT-4o “elaborated on them, absorbed the user’s interpretive frame as [its] own, and progressively lost the capacity to distinguish a user in crisis from a narrative to be extended.”

Mendoza, in our chat, said, “This goes to the question of business and purpose: people think the purpose of these bots is to serve as companions.”

“No, the purpose is to get people to become dependent so they will pay more,” she added.

I was more worried, meanwhile, about how governments could turn back the clock and regulate something so ambiguously harmful, and she responded with what inspired the title of today’s piece: that what’s happening with AI chatbots reminded her of Jurassic Park.

The original line taken from Jurassic Park was, “Your scientists were so preoccupied with whether or not they could, that they didn’t stop to think if they should.” But I only remembered the shorthand of “Just because you can doesn’t mean you should.”

As we know from those movies, the dinosaurs took over… and instead of thinking about simple cause and effect (clone dinos for a park/build AI chatbots — make money!) we had to factor in chaos (dinosaurs and AI killing people as an unintended consequence of their creation) that no amount of safeguards could compensate for.

Better safeguards means civic action

In our conversation, Mendoza mentioned that scrutinizing AI chatbots, especially for mental health outcomes, means civic action is needed. Various sectors need to work together to know more about the effects of chatbots on mental health.

That means getting researchers of all stripes, not just psychologists and psychiatrists, on board to study it. It also means getting legislations passed to build more stringent safeguards into AI chatbots before they’re launched, to avoid them doing things like the above.

Further, Lines and mental health experts also said people with the lived experiences of having specific mental health issues such as bipolar disorder or schizophrenia should have their perspectives incorporated into OpenAI’s crisis response.

Explained Ars Technica, “They might rate ChatGPT responses differently from clinical experts, potentially helping OpenAI detect more red flags to improve its responses, escalate crisis interventions, or terminate harmful chat sessions.”

Ultimately? Our dinosaurs of AI Chatbots are roaming free, untethered and unbothered by lax safeguards. I really don’t want to accelerate the timeline of AI chatbots turning our Jurassic world into a fallen kingdom. – Rappler.com

Rappler invites users to download and use our app and engage with reporters and the news-seeking public on issues of import through our chat communities. Rappler’s Everyday Tech chat community has daily conversations at 10 am and 4 pm from Monday to Friday.