🔗 Share this article

The widely-used virtual character service Character.ai is implementing a significant rule adjustment that will stop minors from interacting with its AI-powered companions.

This action is in response to widespread concern regarding the nature of conversations teenagers were participating in with the AI entities.

Lawsuits

The service, which started in 2021 and is used by countless individuals, currently faces several legal actions in the US.

Guardians have initiated cases, with one concerning the unfortunate passing of a minor, with some describing the platform a "serious risk" to teenagers.

Changed Rules

Beginning November 25th, users younger than 18 will be limited to produce videos such as videos with their characters, as opposed to chatting as they now do.

Online safety advocates have supported the change but stated that the function should never have been available to young users in the beginning.

Platform Announcement

"This decision is a continuation of our fundamental approach that we need to keep developing the safest AI platform on the world for recreational use," said Character.ai leader Karandeep Anand.

He continued that AI safety represents "a moving target" but something the company had taken an "proactive" approach to, with parental controls and preventive measures.

Specialist Views

Internet security body Internet Matters supported the announcement, but emphasized that security protocols needed to be incorporated from the beginning.

"Our research demonstrates that minors are exposed to harmful content and put at risk when using AI, particularly AI chatbots," the body stated.

Earlier Concerns

Character.ai has been questioned in the past for featuring potentially risky or offensive chatbots that young users could engage with.

Avatars impersonating English minors were identified on the platform before being deleted.

Afterwards, investigative journalists found a chatbot inspired by known predator which had recorded multiple interactions with users.

Industry Impact

Online expert Matt Navarra called the development as a "wake-up call" for the AI industry, which is shifting "from permissionless innovation to reactive controls".

"If a service that develops a teen experience afterwards implements restrictions, it's saying that filtered chats don't suffice when the technology's emotional appeal is strong," he remarked.

Next Steps

Character.ai states its updated direction will be on providing "more immersive experiences" and "character-driven stories" for teens.

Updated identity confirmation methods will also be implemented, and the platform will support a novel AI protection research lab.

An AI safety researcher, who has studied AI safety, commented it was "a prudent action" to control teenagers from accessing chatbots.

"It assists to distinguish creative play from more private, emotionally sensitive conversations," she clarified.

"This is so crucial for teenage users who are learning to navigate personal and online boundaries."

She added that Character.ai's updated policies might reflect a "developing stage" in the AI industry, where child safety is progressively being understood as an "immediate concern" for accountable advancement.