Meta Faces San Francisco's Wrath Over AI-Generated Child Exploitation Ads

Meta Under Scrutiny for AI-Generated Child Exploitation Ads

San Francisco's City Attorney, David Chiu, has issued a formal cease-and-desist letter to Meta, demanding the company stop allowing paid advertisements that feature AI-generated child sexual abuse content. This move follows revelations from researchers at the Tech Transparency Project (TTP) that Meta had run over 350 such ads in recent months—some featuring images of real children, including members of a European royal family.

“Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them,” Chiu stated. “These ads are deeply disturbing and cause harm to real children.”

The findings come amid a broader wave of scrutiny over Meta's policies and systems for detecting harmful content on its platforms—Facebook, Instagram, and Threads. Despite the company's public stance against child exploitation, researchers found these ads were repeatedly published, often reaching thousands of accounts in the European Union and beyond.

Systemic Failures in Moderation

The letter from Chiu's office highlights a troubling pattern: not only did these ads evade detection initially, but Meta's response was also delayed. In some cases, ads were removed only after weeks of continued exposure, and even then, they weren't marked as containing child sexual abuse content.

• Ads featured AI-generated imagery that depicted real minors performing sexual acts
• These ads directed users to download nudify apps—tools used for creating nonconsensual intimate images
• The ads were shown in the US, Australia, India, and multiple EU countries
• Some of the same content was reuploaded multiple times, suggesting repeated violations

Meta claims these ads were not displayed in San Francisco, thereby falling outside its jurisdiction. However, the City Attorney's Office disputes this argument, stating that since San Francisco residents can access these ads online, the matter is within their purview.

Meta's Response and Industry-Wide Concerns

Meta has previously stated that most of the ads identified by TTP were removed before reporting, and they've claimed a total ad spend under $5,000 for the 300+ ads. But researchers argue these are only surface-level efforts, with systemic issues persisting beneath the surface.

The company's policies explicitly prohibit child sexual exploitation, nudity, and adult exploitation in advertisements. Their systems include both automated and manual reviews to ensure compliance. However, the evidence suggests that these safeguards failed when it comes to AI-generated content.

“Meta has claimed it takes a hard stance against child sexual abuse material, but since TTP's September 8 report published, more brand-new ads have run featuring some of the same children,” said Katie Paul, director of TTP.

This isn't an isolated case. Researchers continue to identify thousands of ads promoting nudify apps and other harmful platforms—despite Meta's efforts to take action. As of Wednesday, new ads were still appearing on Meta's platforms, indicating that current remediation strategies are insufficient.

The Legal and Ethical Implications

Chiu's letter raises significant legal questions for Meta. It suggests that the company's moderation systems may be inadequate in identifying AI-generated child exploitation content, which could have broader implications for corporate responsibility and platform governance.

The City Attorney's Office emphasized that its objective is not just to identify past failures but to ensure Meta implements robust systems to prevent such content from appearing on its platforms. The letter demands a response within 28 days, highlighting the urgency of the issue.

What This Means for Digital Platforms

This incident serves as a critical wake-up call for digital platforms worldwide. As AI-generated content becomes more sophisticated and pervasive, the responsibility of platforms like Meta to detect and prevent harmful material grows exponentially. We've seen a similar pattern with nudify apps—where despite takedown efforts, new ads continue to surface.

What's clear is that technology alone isn't enough. Human oversight, policy clarity, and rapid response mechanisms are essential components of an effective moderation system. Meta's current approach seems to fall short in these areas, especially with the rise of AI tools that blur the line between reality and artificial content.

A Call for Broader Reform

While this case centers on Meta, it reflects a broader issue facing social media platforms globally. The misuse of AI to generate harmful content isn't limited to one company—it's a symptom of larger systemic gaps in regulation and accountability. Governments, tech companies, and advocacy groups must work together to ensure that the digital economy doesn't come at the expense of vulnerable populations.

Ultimately, this is about more than just profits or policy compliance. It's about protecting children and ensuring that the platforms we rely on for communication, connection, and commerce don't become breeding grounds for exploitation. The question isn't whether companies can detect AI-generated child abuse content—it's whether they're willing to invest in systems that do so reliably and proactively.