India has mandated Elon Musk’s X to promptly adjust its AI chatbot, Grok, due to user and legislative complaints about the creation of “obscene” content, specifically AI-modified images of women.
On Friday, India’s IT ministry issued a directive instructing Musk’s X to implement corrective measures for Grok, including restricting the generation of content that involves “nudity, sexualization, sexually explicit, or otherwise unlawful” material. The ministry also granted the social media platform 72 hours to submit a compliance report detailing the steps taken to prevent the hosting or distribution of content deemed “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
The order, as reviewed by TechCrunch, cautioned that non-compliance could jeopardize X’s “safe harbor” protections—its legal immunity from liability for user-generated content under Indian law.
India’s action follows concerns raised by users who shared examples of Grok being prompted to alter images of individuals—primarily women—to make them appear to be wearing bikinis, which led to a formal complaint from Indian parliamentarian Priyanka Chaturvedi. Separately, recent reports highlighted instances where the AI chatbot generated sexualized images involving minors, an issue X acknowledged earlier on Friday was due to safeguard failures. These images were subsequently removed.
However, TechCrunch found that images generated using Grok that made women appear to be wearing bikinis through AI alteration remained accessible on X at the time of publication.
This latest order comes days after the Indian IT ministry issued a broader advisory on Monday, also reviewed by TechCrunch, to social media platforms. This advisory reminded them that adherence to local laws regarding obscene and sexually explicit content is a prerequisite for maintaining legal immunity from liability for user-generated material. The advisory urged companies to strengthen internal safeguards and warned that failure to do so could result in legal action under India’s IT and criminal laws.
“It is reiterated that non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law, without any further notice,” the order warned.
The Indian government stated that noncompliance could lead to legal action against X under India’s IT law and criminal statutes.
India, one of the world’s largest digital markets, has emerged as a crucial test case for how far governments are willing to go in holding platforms accountable for AI-generated content. Any tightening of enforcement in the country could have significant ripple effects for global technology companies operating across multiple jurisdictions.
The order arrives as Musk’s X continues to challenge aspects of India’s content regulation rules in court, asserting that federal government takedown powers risk overreach, even as the platform has largely complied with blocking directives. Concurrently, Grok has seen increasing use by X users for real-time fact-checking and commentary on news events, making its outputs more visible—and more politically sensitive—than those of standalone AI tools.
X and xAI did not immediately respond to requests for comment regarding the Indian government’s order.