S.F. threatens Apple, Google with legal action over fake nude apps

San Francisco City Attorney David Chiu is threatening legal action against Apple and Google over apps that can generate sexually explicit deepfakes of people without their consent, alleging the companies profit by processing payments for unlawful images.

In cease-and-desist letters dated Thursday, Chiu demanded that the Silicon Valley companies end payment-processing relationships with developers that sell “nonconsensual intimate deepfakes” and regularly review their app stores for similar activity.

The companies have 28 days to explain how they will comply, according to the letters. Chiu’s office said it could otherwise pursue civil enforcement, with penalties of at least $25,000 per violation.

“Apple and Google are profiting off apps that exploit women and girls by generating nonconsensual intimate deepfakes,” Chiu said in a statement. “While the companies cut ties with some problematic apps, Apple and Google have a responsibility to be proactive and vigilant to prevent sexual abuse. Bad actors are always looking for opportunities to profit off this technology, and new apps are popping up every day.”

So-called nudify apps use artificial intelligence to alter ordinary photographs and make people appear nude or engaged in sexual conduct. The images can be created without the knowledge or permission of the person depicted and have increasingly been used to harass students, public figures and other victims.

Chiu’ accuses Apple and Google of knowingly facilitating or recklessly aiding the sale of those images by hosting the apps and handling in-app purchases. 

They do not identify the apps publicly, referring instead to confidential attachments that were not provided to The Chronicle.

Representatives for Apple and Google did not immediately respond to requests for comment Friday.

Google told Wired that it had removed hundreds of apps for violating its policies and had restricted search terms associated with nudification tools. Apple did not comment to the publication.

The dispute represents a test of a California law enacted last year that broadened legal liability for businesses that enable deepfake pornography services.

People who knowingly facilitate or recklessly aid the creation or distribution of nonconsensual sexually explicit deepfakes can face civil claims. The law also allows city attorneys and other public prosecutors to seek injunctions and civil penalties.

A company providing services to a deepfake pornography operation may be presumed to have violated the law if it receives sufficient evidence of the activity and fails to take necessary steps to stop supporting it within 30 days.

Chiu’s office contends that Apple and Google have been aware of the problem for nearly a year.

The letters cite an August 2025 warning from a coalition of 47 state attorneys general and two investigations this year by the Technology Transparency Project, a watchdog organization that studies major technology companies.

In January, the organization said it identified 55 apps in Google Play and 47 in Apple’s App Store that could digitally remove women’s clothing or place their faces onto sexualized images.

The organization said the apps had been downloaded more than 705 million times worldwide and generated $117 million in revenue, citing estimates from the app analytics firm AppMagic. The figures have not been independently audited.

Apple told CNBC after the January report that it had removed 28 of the identified apps and warned other developers to address potential violations. Google said it had suspended several apps and later removed 31, according to the Technology Transparency Project.

A follow-up report in April said searches and advertisements in the two stores continued to steer users toward apps capable of generating sexualized images. Chiu’s office said it found that several apps identified by the group remained available and continued to accept payments.

Both companies maintain policies restricting sexually explicit or exploitative material. Google’s policies specifically prohibit apps that claim to undress people or see through clothing, even when described as entertainment or a prank.

Chiu’s letters go beyond demanding that individual apps be removed. They call for Apple and Google to sever payment-processing relationships with developers selling unlawful deepfakes and establish recurring reviews to ensure new apps do not replace those taken down.

The letters also allege that the companies may have retained all or part of the revenue associated with certain apps after ending their relationships with the developers. 

San Francisco has previously sued the operators of 16 websites accused of creating and selling nonconsensual sexual deepfakes. Chiu’s office described that 2024 case as the country’s first civil enforcement action targeting widely used nudification websites.

The demand also follows broader scrutiny of platforms that enable sexually explicit deepfakes. In January, California Attorney General Rob Bonta opened an investigation into Elon Musk’s xAI after its Grok image generator was used to create nonconsensual sexualized images of real people, including apparent images of minors.

Three U.S. senators separately urged Apple and Google to remove the X and Grok apps until xAI adopted stronger safeguards. xAI later restricted some image-generation features and said it would block certain edits of real people where they are illegal.

In the new letters, Chiu argued that pursuing individual website operators would have limited effect if the same businesses could move their services into major app stores.

“While the companies cut ties with some problematic apps, Apple and Google have a responsibility to be proactive and vigilant to prevent sexual abuse,” Chiu said.