OpenAI Child Exploitation Cases Skyrocket 80-Fold

Table of Contents

The Growing Risks of Generative AI and the Response from Tech Companies

As more countries begin to implement regulations targeting the negative impacts of social media on teenagers, concerns are also emerging regarding the risks posed by generative artificial intelligence (AI). These advanced systems, which enable users to create content with minimal effort, have led to a sharp increase in cases of misuse in criminal activities. In response, major tech companies are now working to introduce safety measures to address these growing concerns.

A Surge in AI-Related Crimes

According to reports from IT media outlet Wired, OpenAI has reported a significant rise in child exploitation cases involving its AI systems. In the first half of 2025, the company submitted 75,027 reports to the National Center for Missing & Exploited Children (NCMEC), marking an 80-fold increase compared to the 947 cases reported during the same period in 2024. Additionally, the number of child exploitation content items reported in the first half of this year reached 74,559, compared to just 3,252 in the previous year.

The NCMEC’s CyberTipline is a congressionally authorized institution that receives reports of child exploitation. When companies suspect such activity, they report it to the center, which then reviews the case and requests an investigation from law enforcement agencies. It is important to note that the current tally does not include reports related to 'Sora,' a video-generating AI released in September. If these reports are added, the number of cases is expected to rise further.

Legal Actions and Calls for Regulation

The consequences of AI's side effects are becoming increasingly serious. In the United States, a family of a teenage boy who died after interacting with ChatGPT has filed a lawsuit against OpenAI. While discussions about regulating social media for teenagers are gaining momentum in the U.S. and Europe, the debate around generative AI is still in its early stages.

In response, attorneys general from 42 U.S. states have sent letters to major tech companies such as Google and Meta, as well as leading AI startups including OpenAI, Anthropic, xAI, and Perplexity. They urged these companies to strengthen chatbot safety measures, stating, “Immediately mitigate the harm caused by flattering or delusional outputs from generative AI and introduce additional safety measures to protect children.” They also warned that failure to take action could result in violations of state laws.

Big Tech's Efforts to Implement Safety Measures

In response to these concerns, AI development companies are rolling out various safety measures. OpenAI is introducing an age prediction model for ChatGPT. This model analyzes the topics of conversations users have with ChatGPT and their primary time zones to determine if the user is under 18. Similarly, Anthropic is developing a system to detect subtle conversational cues that may indicate a user is a minor during interactions with its chatbot, Claude. Accounts identified as belonging to minors are deactivated, and users who disclose their minor status during conversations are flagged separately.

Meta has also introduced new features to help parents monitor their teenage children's interactions with AI characters. The company has implemented a method allowing parents to easily block one-on-one chats between their children and AI characters. Additionally, Meta is taking steps to design AI responses that are appropriate for teenagers, ensuring a safer online environment.

Ongoing Challenges and Future Steps

Despite these efforts, the rapid advancement of generative AI continues to pose challenges for both developers and regulators. As AI becomes more integrated into daily life, the need for robust safety measures and comprehensive regulations will only grow. The collaboration between tech companies, lawmakers, and advocacy groups will be crucial in addressing the potential harms of AI while ensuring its benefits are realized responsibly.