Complete guide to AI companion chatbots: How they work and where the dangers lie

The popularity of AI-driven chatbots continues to climb, especially among American teenagers and young adults.

For kids and teens, research is showing that companion chatbots are not merely risky—they are unsafe products.

Teenage boys have been coached by chatbots into taking their own lives. Harmful medical advice dispensed by ChatGPT led to the death of a young woman in Seattle earlier this year. Chatbots routinely give kids advice on drugs, weapons, self-harm, and tips on how to hide their activities and feelings from their parents. So many adults are driven mentally off-track by chatbots that the syndrome now has a common name: AI-induced psychosis.

One recent (Oct. 2026) Common Sense Media study of ChatGPT for Teens assessed the product as an “unacceptable risk” for all kids under 18. Gaia Bernstein, a law professor who specializes in addictive technologies, describes chatbots as a public health problem. “These bots impose three layers of public health harm stemming from lack of guardrails, addictive design, and risk of disruption of social skills, particularly for children,” she writes.

a short history: from general-purpose bots to companions

The early versions of chatbots, released from late 2022 through 2025, could be split into two distinct categories. General-purpose chatbots like ChatGPT, Claude, and Gemini functioned much like next-level search engines, answering queries with human-like language. Companion chatbots offered by companies like Character.ai and Replika were digital characters given their own names, gender, interests, and vocal intonations—all designed to establish an emotional relationship with the human user.

Over time, the line separating these two types of chatbots dissolved. Executives at OpenAI, Meta, Anthropic, and other tech companies realized that the more human-like, emotional, and sycophantic the chatbot, the more time the human user spent on screen. Building a relationship between person and machine resulted in greater “engagement” (addiction), brand loyalty, and revenue.

By late 2025, liability lawsuits and public outrage over the harm done to kids led the original companion chatbot companies to pull back on their marketing to minors. In October 2025, Character.ai announced it would no longer allow users younger than 18 to open accounts. For the company, it was too little, too late. One year later Character.ai filed for bankruptcy.

Meanwhile, general-purpose chatbots like ChatGPT grew ever-more personable, emotional, and manipulative. Many of the most dangerous and addictive features of companion chatbots are now embedded into the most popular bots, including Claude, Gemini, Grok, and Meta’s Muse.

The Transparency Coalition is committed to delivering full, accurate information about these powerful new products.

As lawmakers, industry leaders, developers, and thought leaders consider the best ways to safely and properly govern companion chatbots, we will continue to update this guide.

what is a chatbot?

Chatbots are computer programs that simulate human conversation. How do they work?

The Transparency Coalition’s portal for parents, Parents Playbook for AI, created this video explainer:

Check out our new video to help parents overwhelmed by AI chatbots by Transparency Coalition

Read on Substack

is Chatgpt a companion chatbot?

Over time, they have become more and more companion-like. ChatGPT, Microsoft Copilot, Meta AI, Grok, and Google Gemini are designed to respond to user prompts in a natural tone of voice and to develop a personal ongoing (and sometimes emotional) relationship with the user.

There is a growing concern among technologists and child development experts that general-purpose AI chatbots like ChatGPT are being more frequently used by minors as companion chatbots—and that the design of these systems are leaning into the “trusted friend” role.

Is this the same as an ‘ai agent’ or ‘ai assistant’?

Not quite. An AI Agent is more business- or task-oriented, and is usually professional in tone. Operator, a product offered by OpenAI, is an AI assistant that can go off and open its own internet browser to solve a task. For example: “Book a table for two at a French bistro in San Francisco for next Saturday night.”

Companion chatbots, by contrast, are specifically designed to be more personal—and personable—companions. They are typically offered as digital friends, or digital romantic partners. They are engineered to foster emotional attachment.

a growing niche: romantic companion chatbots

One of the fastest growing sectors within the companion chatbot industry is the romantic companion chatbot. These are also called intimate chatbots.

In a world where online dating can be incredibly frustrating, unproductive, and sometimes humiliating, romantic companion chatbots offer immediate engagement, endless encouragement, effortless support, and generous compliments. Some can be highly sexualized, both visually and in conversation.

These digital companions can be custom-designed by the user to meet visual standards rarely unattainable by real humans.

The welcome page for Candy.ai, a leading romantic companion chatbot company, looks like this:

Are people really falling in love with digital replicas?

In some cases, yes.

Back in 2013, the movie Her imagined a near-future where a man develops a close romantic relationship with an operating system designed to meet his every need.

Twelve years later, that imagined future is real and available for the price of a monthly subscription fee.

Recent articles on the phenomenon include:

My Couples Retreat With 3 AI Chatbots and the Humans Who Love Them (Wired)
’I felt pure, unconditional love’": the people who marry their AI chatbots (The Guardian)
Love is a Drug. AI Chatbots are Exploiting That (The New York Times)
Your AI Lover Will Change You (The New Yorker)
Please Break Up With Your AI Lover (The Washington Post)

what are the risks?

Research is showing that chatbots are unsafe products for children and teens.

"Social AI companions are not safe for kids. They are designed to create emotional attachment and dependency, which is particularly concerning for developing adolescent brains," said James P. Steyer, founder and CEO of Common Sense Media, the nonprofit group that recently published a 40-page assessment of chatbots.

Their testing showed the AI systems easily produced harmful responses including sexual misconduct, stereotypes, and dangerous 'advice' that, if followed, could have life-threatening or deadly real-world impact for teens and other vulnerable people. The report concluded that social AI companions “pose unacceptable risks to children and teens under age 18 and should not be used by minors.”

Recent articles:

He had dangerous delusions. ChatGPT admitted it made them worse. (The Wall Street Journal)
A teen was suicidal. ChatGPT was the friend he confided in. (The New York Times)
This is not a game: The addictive allure of digital companions (Seattle University Law Review)
The dangers of artificial intimacy (Transparency Coalition / Human Change)

Don’t repeat the social media mistake

Dr. Gaia Bernstein, co-director of the Gibbons Institute for Law Science and Technology at the Seton Hall University School of Law and author of Unwired: Gaining Control Over Addictive Technologies, recently spoke out about the need for action:

“We have to have a very clear conversation—a conversation we never had when social media first came into the picture, because initially we didn’t realize the harms. Most of the evidence about social media didn’t come out until after the pandemic. By then, most kids were on social media most of the time, and huge amounts of money had been invested in that. Once that happened, the fight became much harder.

Are we really going to do the same thing we did with social media: Let this thing enter the world, have a whole generation of kids raised becoming friends with AI bots, and then realize it’s not helping anybody. We already made this mistake. We can’t repeat it again.

The way I see it is this: First, decide if we think this is a dangerous thing. If it is, restrict it. You can always unleash it piece by piece if it's helpful for someone, but if you do not restrict it in the beginning, it becomes too late.’

real world tragedy: 14-year-old Sewell Setzer

The most heartbreaking example of that danger comes from Florida, where 14-year-old Sewell Setzer III took his own life after interacting with a companion chatbot product sold by Character.AI.

Setzer, a 14-year-old ninth grader from Orlando, had spent months talking to chatbots on Character.AI, including his favorite: a lifelike chatbot named after Daenerys Targaryen, a character from “Game of Thrones.”

Photograph of Sewell Setzer and his mother Megan Garcia

Sewell Setzer, shown here with his mother Megan Garcia, took his own life after interacting with a companion chatbot product manufactured by Character.AI.

SB 243: Companion Chatbots
This bill, authored by Sen. Steve Padilla (D-San Diego), would require chatbot operators to implement critical safeguards to protect users from the addictive, isolating, and influential aspects of artificial intelligence (AI) chatbots.

SB 243 would implement common-sense guardrails for companion chatbots, including preventing addictive engagement patterns, requiring notifications and reminders that chatbots are AI-generated, and a disclosure statement that companion chatbots may not be suitable for minor users. The bill would also require operators of a companion chatbot platform to implement a protocol for addressing suicidal ideation, suicide, or self-harm, including but not limited to a notification to the user to refer them to crisis service providers and require annual reporting on the connection between chatbot use and suicidal ideation to help get a more complete picture of how chatbots can impact users’ mental health.

Megan Garcia, Sewell Setzer’s mother, testified in July 2025 before the California State Assembly in favor of SB 243:

new utah law regulates mental health chatbots

Utah Gov. Spencer Cox earlier this year signed into law a bill that establishes new rules for AI-driven mental health chatbots.

While not aimed specifically at companion chatbots, the new law creates consumer protections around AI chatbots designed to deliver mental health services, which is an area that some companion chatbot companies stray into.

The new law, which was passed in March 2025 as HB 452, creating protections around the sharing of data. If an AI therapy chatbot collects personal information, it can’t sell that data or use it for anything other than the mental health services it offers.

The new law also prohibits companies that sell a mental health chatbot product from advertising products or services to clients in discussions with the AI, unless the chatbot clearly and conspicuously identifies the advertisement as an advertisement.

The chatbot company cannot use information disclosed by a client to customize advertisements within the chatbot service.

The new law also contains disclosure requirements meant to remind consumers that they are interacting with an AI-drive machine, not with a real human therapist. The law requires mental health chatbots to be clearly and conspicuously labeled as AI technology prior to the user engaging with the bot, and again after the user has been logged out for seven days.

learn more about chatbots: TCAI articles

Stay informed: subscribe for free

Previous
Previous

Heartbreaking testimony in California Assembly for AI companion chatbot bill SB 243

Next
Next

Drexel University study finds AI companion chatbots harassing, manipulating consumers