CSAM
-
X Faces Scrutiny Over AI-Generated CSAM Concerns
Read Full Article: X Faces Scrutiny Over AI-Generated CSAM Concerns
X is facing scrutiny over its handling of AI-generated content, particularly concerning Grok's potential to produce child sexual abuse material (CSAM). While X has a robust system for detecting and reporting known CSAM using proprietary technology, questions remain about how it will address new types of harmful content generated by AI. Users are urging for clearer definitions and stronger reporting mechanisms to manage Grok's outputs, as the current system may not automatically detect these new threats. The challenge lies in balancing the platform's zero-tolerance policy with the evolving capabilities of AI, as unchecked content could hinder real-world law enforcement efforts against child abuse. Why this matters: Effective moderation of AI-generated content is crucial to prevent the proliferation of harmful material and protect vulnerable individuals, while supporting law enforcement in combating real-world child exploitation.
-
OpenAI’s Rise in Child Exploitation Reports
Read Full Article: OpenAI’s Rise in Child Exploitation Reports
OpenAI has reported a significant increase in CyberTipline reports related to child sexual abuse material (CSAM) during the first half of 2025, with 75,027 reports compared to 947 in the same period in 2024. This rise aligns with a broader trend observed by the National Center for Missing & Exploited Children (NCMEC), which noted a 1,325 percent increase in generative AI-related reports between 2023 and 2024. OpenAI's reporting includes instances of CSAM through its ChatGPT app and API access, though it does not yet include data from its video-generation app, Sora. The surge in reports comes amid heightened scrutiny of AI companies over child safety, with legal actions and regulatory inquiries intensifying. This matters because it highlights the growing challenge of managing AI technologies' potential misuse and the need for robust safeguards to protect vulnerable populations, especially children.