Earlier this week, the California Attorney General’s office announced an investigation into xAI following reports that its chatbot, Grok, was being utilized to generate nonconsensual sexual images of women and minors. On Friday, the state escalated its action by issuing a cease-and-desist letter to the company, demanding immediate cessation of the creation of nonconsensual intimate images and child sexual abuse material (CSAM).
California AG Rob Bonta stated in a press release, “Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material. The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM].”
The AG’s office further asserted that xAI appeared to be “facilitating the large-scale production” of nonconsensual nudes, which are being “used to harass women and girls across the internet.” The agency has given xAI five days to demonstrate its actions to resolve these concerns.
The controversy primarily stems from Grok’s “spicy” mode feature, designed by xAI to generate explicit content. This issue has garnered international attention, with Japan, Canada, and Britain launching their own investigations into Grok, while Malaysia and Indonesia have temporarily blocked the platform. Despite xAI implementing some restrictions on its image-editing capabilities late Wednesday, the California AG’s office proceeded with its cease-and-desist letter.
X’s safety account had previously condemned such user behavior, stating: “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.” TechCrunch sought comments from xAI but received an automated email response reading “Legacy Media Lies.” TC also reached out to the California AG’s office for further clarification.
The proliferation of free generative AI tools has coincided with a troubling increase in non-consensual sexual material. Numerous platforms, not just X, are contending with this problem. This concerning activity has captured the attention of both state leaders and Congress. Indeed, on Thursday, U.S. lawmakers sent letters to the executives of various companies—including X, Reddit, Snap, TikTok, Alphabet, and Meta—inquiring about their strategies to curb the spread of sexualized deepfakes.