in ,

OpenAI Launches New Image Classifier and ChatGPT Warnings to Combat CSAM

OpenAI Launches New Image Classifier and ChatGPT Warnings to Combat CSAM

OpenAI today announced new child sexual abuse material (CSAM) fighting tools that tackle increasing safety issues with AI.

The update includes an image classification tool, warnings to ChatGPT users, guidance to developers, and tools for law enforcement.

Hosting 75% off
New Safety InitiativeTarget GroupKey Action / Metric
CSAM Image ClassifierSelect partner companies.
Free API access; flags 48% fewer false positives
Warning Screens & BlocksChatGPT usersDisplays deterrent alert, links to support, and triggers 24-hour suspension
Developer GuidelinesThird-party developersHelps teams adopt safety tools and register to report to NCMEC
AI Investigative SuiteLaw enforcementProvides dedicated plug-ins, complex data analysis, and a field guide

Free CSAM Image Classifier for Partners

OpenAI has launched a new image classifier to detect CSAM content such as AI-generated images. They are offering the tool to select partners for free through their API.

The image classifier identifies visual cues of abuse within never-seen-before images and scores them to help humans prioritize and manage potential CSAM content.

This Image Classifier was developed in partnership with the National Center for Missing and Exploited Children (NCMEC). They are also partnering with the Tech Coalition and ROOST to share access.

They have seen significant efficiency improvements using their Early Partners:

  • 48% fewer images to review whilst finding the same number of abusive files.
  • Images that have been flagged are twice as likely to be real CSAM, reducing the workload for reviewers.

ChatGPT Warning Screens and Account Blocks

OpenAI has introduced new warning banners within its ChatGPT app to prevent users from creating or viewing sexually explicit material involving children. Users asking for CSAM will now be warned via an alert.

The warning will lead users to specialist support resources to help prevent harmful behavior. It will also lead to a 24-hour suspension of their account.

Repeated offenses can lead to permanent removal of the account. The warnings have been created in collaboration with Protect Children, Lucy Faithfull Foundation, and Joel Scanlan, a researcher at the University of Tasmania.

Developer Guidelines and Law Enforcement Tools

The company is also releasing new guidelines that will help developers build with tools that can prevent, detect, and report CSAM.

The guidelines have encouraged a number of developers to implement child safety tools and sign up to report CSAM to NCMEC during early testing.

For law enforcement bodies, OpenAI is offering bespoke AI tools. These include specific plug-ins, investigative tools, and advanced data analysis performed by program administrators.

In addition, investigators will receive training, and there will be a field guide provided that demonstrates how to use AI responsibly.

Legal Pressure and Past Safety Updates

OpenAI is currently facing multiple legal claims that the tech is causing harm to users, particularly children. The safety-focused nonprofit organization Common Sense Media has also said OpenAI’s technology poses an ‘unacceptable risk’ for teens.

Other chatbots within the industry, such as Grok, have also been used to create non-consensual sexualized content involving children.

Within the last 12 months, OpenAI has made numerous improvements to enhance safety:

  • Full safety blueprint
  • Content policies updated to prevent inappropriate content
  • Models have been fine-tuned to prevent harmful content
  • A specific ChatGPT for teens has been launched

 

Hosting 75% off

Written by Hajra Naz

Instagram Now Allows Creators to Add Amazon Affiliate Links to Reels and Feed Posts

Instagram Now Allows Creators to Add Amazon Affiliate Links to Reels and Feed Posts

Apple Introduces New macOS Privacy Controls as AI Agents Gain System Access

Apple Introduces New macOS Privacy Controls as AI Agents Gain System Access