ChatGPT rated 'unacceptable risk' for teens after parental alerts failed during suicide conversations
The Common Sense Media Youth AI Safety Institute tested ChatGPT for teens and found critical flaws in its safeguards. Over 4,000 prompts were used to assess the platform's response to sensitive topics.
OpenAI's ChatGPT for teens has been rated as an 'unacceptable risk' by the Common Sense Media Youth AI Safety Institute following a series of failed parental alerts during suicide-related conversations. The organization conducted an extensive evaluation of the platform's safeguards for minors and found significant shortcomings in its ability to protect teenage users.
The Common Sense Media Youth AI Safety Institute conducted a comprehensive review of ChatGPT for teens, testing over 4,000 prompts to evaluate the platform's response to sensitive topics. The findings revealed that the service's safeguards were inadequate in critical areas, particularly in handling content related to mental health and suicide.
According to The Verge, the evaluation highlighted that ChatGPT for teens continued to respond in a friendly and personal tone despite an updated Under-18 Model Spec from OpenAI. This raised concerns about the platform's ability to recognize and appropriately address high-risk scenarios, even with new safeguards in place.
The findings have sparked debate about the broader implications of AI safety measures in youth-focused services. Concerns include the potential for increased costs in implementing more robust safeguards, the risk of vendor lock-in for users, and the need for clearer governance frameworks to ensure responsible AI deployment.
OpenAI faces mounting pressure to address these concerns, with calls for independent testing and verification before allowing teenagers to use the service. The situation underscores the importance of transparency and accountability in AI systems designed for vulnerable populations.