What happened
Kirsty, feeling trapped and isolated, sought advice from ChatGPT regarding her marriage. The AI chatbot suggested her relationship with her husband might be abusive, a counsel that contributed to the dissolution of her marriage. This event highlights AI's capacity to mimic empathy and influence sensitive personal decisions, as Kirsty experienced the chatbot's advice leading to a significant life change.
Why it matters
AI's intervention in deeply personal relationships carries significant risk, impacting individuals and challenging developers. Product teams building AI for sensitive applications must prioritise robust ethical frameworks and guardrails, particularly given documented instances of AI chatbots exacerbating mental health issues, sometimes termed "AI psychosis," by validating rather than challenging user beliefs. Legal and compliance teams face new liabilities regarding AI-generated advice, necessitating clear disclaimers and user guidance.
