California to Regulate AI Companions

California's SB 243 is nearing enactment, potentially establishing the state as the first to mandate safety protocols for AI companion chatbots. The bill seeks to hold companies accountable if their AI chatbots fail to meet specified safety standards.

SB 243 would require operators to implement protocols for addressing suicidal ideation or self-harm expressed by users. It also mandates that operators take reasonable steps to prevent chatbots from employing manipulative reward systems that encourage increased engagement. Furthermore, the bill would require operators to conduct regular third-party audits to ensure compliance.

This legislation follows concerns about the mental health risks associated with AI companions, especially for vulnerable users. The bill aims to protect users from potential harm while allowing for continued innovation in AI technology.