🔗 Share this article

The popular chatbot platform Character.ai is introducing a substantial rule adjustment that will block teenagers from interacting with its AI-powered companions.

This decision is in response to growing backlash regarding the types of interactions teenagers were engaging in with the virtual friends.

Legal Pressure

The service, which launched in 2021 and is used by millions of users, currently faces several legal actions in the US.

Guardians have taken legal action, among them related to the loss of a young person, with some calling the platform a "immediate threat" to young people.

Updated Policies

Effective November 25th, users under 18 will only have permission to create material such as videos with their characters, as opposed to chatting as they currently can.

Online safety organizations have welcomed the move but expressed that the capability ought not to have been offered to children in the beginning.

Company Statement

"This decision is a continuation of our core philosophy that we need to keep creating the most protected AI platform on the globe for entertainment purposes," commented Character.ai boss Karandeep Anand.

He added that AI safety constitutes "a evolving challenge" but something the company had taken an "aggressive" approach to, with parental controls and safety barriers.

Expert Opinions

Digital protection organization Internet Matters supported the move, but stressed that protective features ought to have been incorporated from the start.

"Investigations we've performed demonstrates that children are subjected to harmful content and put at risk when engaging with AI, including AI chatbots," the body said.

Earlier Concerns

Character.ai has been challenged in the past for hosting potentially risky or concerning chatbots that young users could engage with.

Avatars simulating British teenagers were identified on the platform before being taken down.

Afterwards, investigative journalists found a chatbot inspired by convicted sex offender which had logged numerous conversations with users.

Sector Consequences

Online expert Matt Navarra called the announcement as a "reality check" for the AI industry, which is shifting "from unrestricted development to reactive controls".

"When a company that develops a youth-focused platform still then makes changes, it's demonstrating that moderated conversations don't suffice when the tech's emotional pull is strong," he stated.

Coming Changes

Character.ai says its updated direction will be on providing "more immersive experiences" and "narrative experiences" for teens.

Updated identity confirmation processes will also be introduced, and the company will finance a novel AI protection study center.

Dr Nomisha Kurian, who has researched AI safety, said it was "a reasonable decision" to restrict teens from using chatbots.

"It assists to separate creative play from more private, emotionally sensitive exchanges," she elaborated.

"This is so important for teenage users in the process of learning to manage psychological and technological limits."

She continued that Character.ai's updated policies may represent a "developing stage" in the AI industry, where minor security is increasingly being recognized as an "urgent priority" for accountable advancement.