When teens talk to Meta AI about suicide or self-harm, the conversation will trigger a parental notification, Meta said in a blog post Thursday.
The new policy expands on existing teen measures designed to inform parents about how their child is using Meta AI. Only parents using Instagram supervision controls will receive the notification.
“We worked with parents and experts to understand which AI conversations warrant an alert — such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle,” Meta said.
How does Meta AI crisis alerts work?
Meta AI already cites crisis helplines and encourages teens considering suicide or self-harm to seek help from a trusted adult. Now, parents will get an alert about such conversations. Meta said it would err on the side of caution for the time being.
While concerning chats are flagged by artificial intelligence, Meta manually reviews them before sending an alert. The parent also receives expert-developed suggestions for discussing self-harm and suicide with their child.
Parents will receive the alert via an app notification and a separate email, text, or WhatsApp message, depending on the contact information they provided.
Meta alerts parents whose child may have discussed suicide or self-harm with Meta AI.
Credit: Courtesy of Meta
The types of alerts parents can receive from Meta.
Credit: Courtesy of Meta
In February, Meta launched a similar notification feature for Instagram.
Mashable Light Speed
Dr. John Ackerman, clinical manager for the Center for Suicide Prevention and Research at Nationwide Children’s Hospital, told Mashable at the time that he welcomed expanded protections for teens. He noted, however, that such a feature could amount to “lip service” if notifications are inaccessible, difficult to navigate, or don’t lead to “actionable change.”
Meta will send alerts to parents in the U.S., United Kingdom, Canada, and Australia. It will be globally accessible by the end of the year, according to Meta.
New limited content setting for Meta AI
Meta also announced Thursday the addition of a limited content setting for Meta AI. It debuted that feature for Instagram last year.
That setting enables Meta’s strictest filters. Meta AI is already trained not to engage in sexual or romantic conversations with teens, or provide alcoholic drink recipes, for example, according to the company.
When the limited content setting is turned on, Meta’s models become even more aggressive in identifying problematic prompts and decline to answer a broader range of queries. Meta said the approach further reduces “the chance of potentially inappropriate conversations.”
Teen safety concerns
Child safety advocates have criticized Meta’s recent parental control updates as insufficient.
In April, Meta gave parents some insight into their child’s conversations with Meta AI. The feature highlights broader topics, such as school, entertainment, writing, health, and wellbeing. Parents can click on the topic for additional but limited detail.
Josh Golin, executive director of the children’s advocacy nonprofit Fairplay, said the parental control “once again” burdens caregivers with monitoring their child’s online activity in lieu of “building a safe product to begin with.”
Meta continues to face legal scrutiny for its performance on child safety. Earlier this year, Meta lost two separate landmark trials related to child safety protections and the allegedly addictive design of its products. The company said it will appeal both verdicts, but hundreds of pending lawsuits alleging child harm have yet to be tried.
If you’re feeling suicidal or experiencing a mental health crisis, please talk to somebody. You can call or text the 988 Suicide & Crisis Lifeline at 988, or chat at 988lifeline.org. You can reach the Trans Lifeline by calling 877-565-8860 or the Trevor Project at 866-488-7386. Text “START” to Crisis Text Line at 741-741. Contact the NAMI HelpLine at 1-800-950-NAMI, Monday through Friday from 10:00 a.m. – 10:00 p.m. ET, or email [email protected]. If you don’t like the phone, consider using the 988 Suicide and Crisis Lifeline Chat. Here is a list of international resources.
