In a landmark legal move, the city of San Francisco has filed a comprehensive lawsuit against 18 websites and apps accused of generating unauthorized deepfake nudes of unsuspecting victims. The lawsuit, filed on Thursday, comes amid growing concerns over the proliferation of these platforms, which have collectively attracted over 200 million visits in the first half of 2024 alone.
San Francisco City Attorney David Chiu, who spearheaded the lawsuit, expressed deep concern over the exploitation of women and girls through these technologies. “This investigation has taken us to the darkest corners of the internet, and I am absolutely horrified for the women and girls who have had to endure this exploitation,” Chiu stated. He emphasized that while generative AI holds great potential, its misuse for creating non-consensual intimate imagery (NCII) is a gross violation of privacy and human dignity. “This is not innovation—this is sexual abuse,” he added.
The complaint targets a wide array of websites and apps that offer services to “undress” or “nudify” women and girls through the use of AI-powered tools. Although the specific names of the platforms have been redacted, the scope of the lawsuit is clear: to combat the alarming rise of NCII, which has left victims with profound psychological, emotional, and reputational damage.
The lawsuit also highlights the challenges faced by victims in removing these images once they have been disseminated. The ease with which these deepfakes can be created and shared has left many women and girls with little recourse, further exacerbating the trauma inflicted upon them. The rapid spread of this content, which is often used to extort, bully, and humiliate, has sparked a global outcry for stronger measures to curb the practice.
One particularly concerning aspect of the lawsuit is the inclusion of AI-generated child sexual abuse material (CSAM). According to Chiu, some of these sites have enabled users to create child pornography, a practice that severely complicates efforts to protect real victims and prosecute offenders. The Internet Watch Foundation, which monitors online child exploitation, has warned that AI-generated CSAM could “overwhelm” the internet, making it increasingly difficult to track and eliminate harmful content.
In response to these growing concerns, a new law in Louisiana specifically banning CSAM created with AI went into effect this month. However, the spread of AI-generated NCII and CSAM has already outpaced regulatory efforts, with major tech companies struggling to keep harmful content out of AI training datasets. Researchers at Stanford University have also flagged that AI-generated images have already infiltrated these datasets, raising serious ethical and safety concerns.
Also Read: Figure Markets Launches Exchange with Decentralized Custody and Democratized Prime Brokerage
The lawsuit seeks to impose a fine of $2,500 for each violation, effectively aiming to bankrupt the offending platforms. It also calls on domain name registrars, web hosts, and payment processors to cease providing services to companies involved in creating and disseminating deepfake content.
As this legal battle unfolds, it marks a critical step in the fight against the misuse of AI technology. San Francisco’s bold stance could set a precedent for other jurisdictions grappling with the dark side of AI, as governments and organizations worldwide seek to protect individuals from the devastating impact of non-consensual intimate imagery.
Disclaimer: The information in this article is for general purposes only and does not constitute financial advice. The author’s views are personal and may not reflect the views of Chain Affairs. Before making any investment decisions, you should always conduct your own research. Chain Affairs is not responsible for any financial losses.
About The Author
