OpenAI today announced new child sexual abuse material (CSAM) fighting tools that tackle increasing safety issues with AI.
The update includes an image classification tool, warnings to ChatGPT users, guidance to developers, and tools for law enforcement.
| New Safety Initiative | Target Group | Key Action / Metric |
| CSAM Image Classifier | Select partner companies. | |
| Free API access; flags 48% fewer false positives | ||
| Warning Screens & Blocks | ChatGPT users | Displays deterrent alert, links to support, and triggers 24-hour suspension |
| Developer Guidelines | Third-party developers | Helps teams adopt safety tools and register to report to NCMEC |
| AI Investigative Suite | Law enforcement | Provides dedicated plug-ins, complex data analysis, and a field guide |
Free CSAM Image Classifier for Partners
OpenAI has launched a new image classifier to detect CSAM content such as AI-generated images. They are offering the tool to select partners for free through their API.
The image classifier identifies visual cues of abuse within never-seen-before images and scores them to help humans prioritize and manage potential CSAM content.
This Image Classifier was developed in partnership with the National Center for Missing and Exploited Children (NCMEC). They are also partnering with the Tech Coalition and ROOST to share access.
They have seen significant efficiency improvements using their Early Partners:
- 48% fewer images to review whilst finding the same number of abusive files.
- Images that have been flagged are twice as likely to be real CSAM, reducing the workload for reviewers.
ChatGPT Warning Screens and Account Blocks
OpenAI has introduced new warning banners within its ChatGPT app to prevent users from creating or viewing sexually explicit material involving children. Users asking for CSAM will now be warned via an alert.
The warning will lead users to specialist support resources to help prevent harmful behavior. It will also lead to a 24-hour suspension of their account.
Repeated offenses can lead to permanent removal of the account. The warnings have been created in collaboration with Protect Children, Lucy Faithfull Foundation, and Joel Scanlan, a researcher at the University of Tasmania.
Developer Guidelines and Law Enforcement Tools
The company is also releasing new guidelines that will help developers build with tools that can prevent, detect, and report CSAM.
The guidelines have encouraged a number of developers to implement child safety tools and sign up to report CSAM to NCMEC during early testing.
For law enforcement bodies, OpenAI is offering bespoke AI tools. These include specific plug-ins, investigative tools, and advanced data analysis performed by program administrators.
In addition, investigators will receive training, and there will be a field guide provided that demonstrates how to use AI responsibly.
Legal Pressure and Past Safety Updates
OpenAI is currently facing multiple legal claims that the tech is causing harm to users, particularly children. The safety-focused nonprofit organization Common Sense Media has also said OpenAI’s technology poses an ‘unacceptable risk’ for teens.
Other chatbots within the industry, such as Grok, have also been used to create non-consensual sexualized content involving children.
Within the last 12 months, OpenAI has made numerous improvements to enhance safety:
- Full safety blueprint
- Content policies updated to prevent inappropriate content
- Models have been fine-tuned to prevent harmful content
- A specific ChatGPT for teens has been launched






