AI child abuse reports jump 8,400 percent in two years
Tech platforms reported over 400,000 AI-generated child abuse files in 2025, triggering strict new laws for model builders and hosts.
Policy · Source: StartupFortune
What happened
The AI industry has a massive abuse problem. According to the 2026 Into the Light Index from Childlight at the University of Edinburgh, tech companies sent more than 400,000 AI-linked child sexual abuse reports to the US National Center for Missing and Exploited Children in 2025. That is an 8,400 percent increase from just 4,700 reports in 2023. These figures were unveiled at a global summit of delegates from more than 100 governments focused on ending violence against children.
These numbers only count what platforms actually detect and report under United States law. NCMEC separately noted its CyberTipline received over one million generative-AI reports between January and September of 2025. The material includes deepfakes created with nudify apps. These tools place a real childs face onto explicit imagery within seconds. Bad actors then use these fake images to blackmail children for money or demand more images.
The threat is moving rapidly from still images to video. The UK Internet Watch Foundation found 3,440 AI-generated child abuse videos in 2025, up from just 13 the year before. Sixty five percent of those videos were graded in Category A, the most extreme grade of abuse. Childlight estimates that more than 300 million children a year experience technology-facilitated sexual exploitation, and one in four face online abuse before turning 18.
Key facts
- 400,000 — AI-linked abuse reports sent to NCMEC in 2025
- 8,400% — Increase in AI abuse reports from 2023 to 2025
- 1,000,000 — Generative-AI reports to NCMEC CyberTipline in first nine months of 2025
- 3,440 — AI abuse videos found by the IWF in 2025
- 5 — Maximum years in UK prison for creating abuse-generating AI tools
Why it matters
Governments are no longer waiting for the tech industry to self-regulate. The UK has moved the furthest by passing the Crime and Policing Act 2026. This law hands out up to five years in prison for possessing, creating, or distributing AI tools designed to generate child abuse material. It also explicitly criminalizes making or supplying nudification tools. This puts a massive legal target on the backs of developers who build generative models and the platforms that host them.
The compliance burden for hosting platforms is about to skyrocket. The UK offence turns on what a tool is designed to do. Platforms hosting open weights will have to police how users fine-tune those models before a prosecutor decides to test the law. Companies that do not actively scan for and report this material do not generate reports, creating a false sense of safety. The next set of industry figures will expose exactly which platforms are turning a blind eye.
For builders
Open weight hosts face severe legal risks
The UK now criminalizes supplying tools designed to generate abuse material. Platforms hosting open weights must figure out how to police fine-tunes, or they risk prison time for their executives.
Content scanning is mandatory infrastructure
The 400,000 reports came from companies actively scanning their networks. If you build an image or video generation app, you must pay for robust detection systems or lose your business to regulators.
Video models require strict safety guardrails
AI abuse videos jumped from 13 to over 3,400 in a single year. Founders building video models must invest heavily in safety filters because this is where lawmakers will strike next.
My take
I am tired of founders treating safety as a feature to build later. If your model makes it cheap and easy to generate child abuse material, you are building a weapon, not a product. Governments are finally stepping in with real prison sentences. If you get shut down or jailed for ignoring this reality, you deserve it.
Original reporting: StartupFortune. This is my rewrite and opinion.