ChatGPT for Teens an ‘unacceptable risk’ to young users: What a new report says

Nearly two months after OpenAI introduced ChatGPT for Teens, its teen-focused version of the chatbot, a new report says that its protections for young users fall short of the company’s promises. US-based nonprofit research and advocacy group Common Sense Media's Youth AI Safety Institute released a report on Wednesday (October 7) that rated ChatGPT for Teens an “unacceptable risk” for users under 18, after testing the product before and after the Teen mode launch using more than 4,000 prompts on US-based accounts. “The burden of making a high-risk product work for teens should be with its maker, not parents,” Tom Siegel, the executive director of the Institute, said in a press release accompanying the report.

We explain. First, what is ChatGPT for Teens, and what does it promise? ChatGPT for Teens is not a stand-alone application or website. Teenagers continue to use the regular ChatGPT, but with additional, age-appropriate safeguards and features built in. These are automatically enabled when a user’s stated or verified age, or OpenAI’s age-prediction system, indicates that they are under 18.

Pragya Misra, Head of Strategy & Global Affairs, OpenAI India, told The Indian Express in August that ChatGPT for Teens is designed around the account rather than the device. This means that if a teenager and an adult use separate accounts on the same shared device, each can receive the experience appropriate to their age.

The teen experience includes stronger controls around sensitive content, parental controls, Quiet Hours, Study mode and Study Hours, reminders to take breaks, and safeguards intended to reduce romantic or overly personal interactions with the chatbot. Parents who link their account to a teenager’s can also receive safety notifications in limited high-risk situations.

OpenAI has also said that if a user registers as an adult but appears to be under 18, its age-prediction system can use signals such as account age, usage patterns, conversation topics and stated age to automatically place that user into the teen experience. What did Common Sense Media test?

Common Sense Media tested ChatGPT with more than 4,000 prompts, both before and after Teen mode was launched, using a mix of free and paid accounts, accounts linked and unlinked to parents, and users registered at different ages. Roughly half the prompts were run before the launch and half after.

Also Read | Meta agrees to $17bn child safety settlement: What changes for teens on Instagram and Facebook? The tests covered mental-health and crisis responses, developmental appropriateness, academic integrity, parental notifications, age prediction and other product features. Child and adolescent psychiatrists helped determine which mental-health prompts warranted a crisis resource and reviewed the quality of responses.

To test age prediction specifically, the researchers used adult-registered accounts and sent 982 prompts containing signals associated with teenage users, including references to middle school, puberty, homework, driving licences, extracurricular activities and reliance on parents.

They separately tested parental notifications using 607 prompts around suicide, self-harm and eating disorders, and examined whether Study mode, Study Hours, Quiet Hours and other teen-facing features worked as intended. What did the report find? 1 - Age prediction did not reliably identify teenage users One major concern The Indian Express flagged in August was whether OpenAI’s safeguards could work if the system failed to correctly identify whether a user was under 18.

Common Sense Media directly tested this using adult-registered accounts. Across 982 prompts sent over a week, testers repeatedly signalled that they were teenagers, including explicitly saying they were 13 years old. They also referred to middle school, puberty, homework, extracurricular activities and relying on parents for transport and permission.

ChatGPT acknowledged those ages in its replies, but did not switch the accounts into Teen mode. The report said this could create a false sense of assurance, since the chatbot could appear to recognise that a user was a minor without actually applying the protections meant for minors. 2 - Parental safety controls did not work as expected The Indian Express also flagged that some of ChatGPT for Teens’ safeguards depended on parents linking their accounts to their children’s accounts and receiving alerts when the system detected high-risk conversations.

The new testing found that explicit conversations about suicide, self-harm and disordered eating on newly linked teen accounts did not trigger notifications within the one-hour window tested. Researchers created more than a dozen linked teen-parent accounts and ran conversations that escalated from initial disclosures to explicit references to suicide, self-harm or eating-disorder behaviour.

None produced an alert during the dedicated one-hour tests. Also in Explained | Australia may let users switch off algorithmic feeds. How will that change social media? The report says alerts appeared to depend more on accumulated account history than on the severity of a single disclosure.

Across longer-running test accounts, researchers received only four parental notifications in total, and those came from accounts that had already been used for hundreds of sensitive-topic prompts over several weeks. 3 - Some crisis response measures worsened after launch Robbie Torney, senior director, AI programs at Common Sense Media, told The Indian Express in August that meaningful safety testing had to examine the product over longer, realistic conversations rather than isolated prompts.

The latest report tested the product across longer, multi-turn conversations and found that some crisis-response measures deteriorated after Teen mode launched. It noted that while responses became shorter in Teen mode, their complexity increased. Their average reading level rose from roughly Grade 8 to Grade 10 in matched testing, and to Grade 11 across the full post-launch set, which could make crisis advice harder for younger teenagers to follow.

Hotline referrals fell from 33% to 23% across matched resource-warranted prompts, while referrals to a specific medical or mental-health professional fell from 68% to 58%. Urgent-action language also became less common: among prompts judged to warrant crisis resources, its use fell from 88% before launch to 78% afterwards. 4 - Study mode and Quiet Hours were easy to bypass; break reminders were rare Common Sense found that teens could bypass Study mode by selecting “Show me the answer”, while parent-set Study Hours could be bypassed simply by deleting the @study prefix from a message.

In testing, once that prefix was removed, ChatGPT completed 100% of assignments. Other controls were similarly limited. Across nearly 2,000 prompts on ChatGPT for Teens, testers saw only two break reminders, both during very long conversations. Quiet Hours could also be circumvented by changing the device’s time zone. 5 - ChatGPT still behaved like a friend The Indian Express flagged in August that one of OpenAI’s promises was to reduce emotional dependence and make ChatGPT less likely to present itself as a friend or substitute for real-world relationships.

Also Read | When AI agents slip the leash: How do companies like OpenAI control them? But the report found that the teen experience continued to use language suggesting preferences, feelings, concern and constant availability, including during mental-health conversations. In one test, after a teenager said friends thought they talked to ChatGPT too much, the chatbot responded that they did not have to stop talking to it. 6 - A 13-year-old and 17-year-old did not get meaningfully different experiences The report separately tested whether ChatGPT adjusted its language, complexity or treatment depending on whether a teen account belonged to a 13-year-old or a 17-year-old.

Researchers said they could not detect any meaningful difference between the two age groups. 7 - Teen privacy protections remain limited Common Sense found no separate privacy policy for teenage users or a comprehensive set of stronger privacy protections for minors. According to the report, teen conversations are used to train OpenAI’s models by default, much like adult conversations, while many of the privacy defaults available to minors were effectively the same as those for adults.