FBI agent explains how easy it is to ID people posting AI porn without consent

The earliest arrests under the Take It Down Act (TIDA) suggest that cops don’t have to work too hard to identify people illegally posting and selling nonconsensual sexualized deepfakes of women online.

Last week, the FBI arrested two men after visiting porn websites and clicking on hashtags like #AI #Deepfakes or video titles like “AI_tits” or “Ass_AI.”

One suspect accused of violating TIDA was 20-year-old Arturo Hernandez. He allegedly posted 113 albums viewed nearly a million times featuring AI-generated sexualized images and videos of approximately 50 women. Victims included political figures, actresses, and musicians, as well as women who are not public figures, such as female individuals who attended his Texas high school and an Instagram friend.

Geo-location data helped cops identify Hernandez as a suspect. In his affidavit, an FBI special agent, Christopher Powell, explained that cops investigating the porn site found a second account re-posting all the content that Hernandez allegedly uploaded. That second account was linked to Hernandez’s PayPal account, the complaint said, and an IP address often used to log in to it was the same IP that Hernandez’s Apple records showed he’d used to log in to his iCloud.

While sexualized deepfakes of celebrities and politicians may be easiest to spot in the wild, cops also sought evidence tying Hernandez to AI content depicting people he knew. It likely simplified their search to find that not only did Hernandez follow the Instagram account of one victim, but cops discovered that Hernandez had also saved in a folder on his own Instagram account the specific image used to create AI porn content viewed more than 36,000 times.

Hernandez seemingly tried to distance himself from some of the activity, for example, by registering his Gmail account with the nickname “Ryan” instead of his actual first name. However, cops noted that Hernandez used the “Ryan” nickname elsewhere online, including on his Snapchat account.

However, the other man arrested, 51-year-old Cornelius “Neil” Shannon, was allegedly less careful, Powell’s affidavit for that arrest showed.

Shannon is accused of publishing approximately 360 AI-generated albums that have been viewed more than 2 million times, featuring approximately 90 women, primarily political figures, actresses, and musicians.

Powell’s affidavit suggested it was trivially easy to link Shannon to the porn site account because Shannon apparently used his own photo as the profile pic. Cross-referencing Department of Motor Vehicle records and surveillance photos, cops alleged that a man seen posing in a Mets baseball shirt on the account’s profile appeared to be Shannon.

Both Hernandez and Shannon risk up to two years in prison if cops can prove they violated TIDA.

FTC warns 12 nudify toolmakers

Officials appear motivated to track images posted online and enforce the law.

In a press release announcing the recent arrests, Joseph Nocella, Jr., United States Attorney for the Eastern District of New York, accused the suspects of using “cutting-edge digital technology to create images that degraded and violated victims across the United States.” And James C. Barnacle, Jr., assistant director in charge of the New York FBI field office, confirmed that his agents would continue investigating similar cases.

“This predatory conduct represents a disturbing abuse of technology that inflicts emotional harm on victims, violating their privacy, dignity, and security,” Barnacle said. “The use of this emerging technology to victimize individuals is not innovative—it is criminal and will be pursued with the full force of the law.”

However, some people charged with TIDA violations may continue using the technology to harm victims, as it remains readily available and relatively cheap to make realistic-looking content sexualizing real people. An Ohio man who was hailed by the US Justice Department as the first arrest under TIDA notably continued making sexualized deepfakes while on pre-trial release, apparently undeterred by even the threat of imminent consequences.

To block people from using AI services to “undress” people and share harmful images online, the Federal Trade Commission announced last week that it sent warning letters to 12 companies offering so-called “nudify” tools.

Those companies appear to be violating TIDA, the FTC warned, and need to implement “a process through which victims can request the removal of nonconsensual intimate images appearing on their platforms” within 48 hours or risk “civil penalties of up to $53,088 per violation.”

It’s unclear if the social media platform X—which has been sued by three girls who claimed its chatbot Grok turned their real photos into AI child sexual abuse materials—received a warning. But X’s Safety account posted last week, notifying users that TIDA victims can report harmful content through the Help Center. They can also report any post by “tapping the three-dot menu (⋯), selecting ‘Private or Non-Consensual Content’ > ‘Report content under the US Take It Down Act’ and then completing the form.”

“Our team reviews these reports as fast as possible and well within the Act’s 48-hour timeline,” the X Safety account said.

More broadly, the FTC is seemingly stepping up enforcement after a deadline passed this month when all online platforms were required to have such a process in place. A week before the warning letters went out to nudify services, the FTC sent additional letters to operators of major platforms, including Amazon, Alphabet, Apple, Automattic, Bumble, Discord, Match Group, Meta, Microsoft, Pinterest, Reddit, SmugMug, Snapchat, TikTok, and X. Those letters told platforms to be prepared to face penalties for any non-compliance.

Even if such a process were in place on every online platform, however, TIDA seems imperfect, since it doesn’t stop the initial sharing and still puts the burden on the victim to monitor and flag harmful images across the Internet. Critics have warned that the process could also be abused by people, including possibly Donald Trump, who hope platforms will automatically remove any reported content they don’t like.