India has mandated that Elon Musk’s X promptly implement technical and procedural adjustments to its AI chatbot, Grok, following complaints from users and legislators regarding the creation of “obscene” material, such as AI-modified images of women.
The Indian IT ministry issued an order on Friday, instructing Musk’s X to rectify issues with Grok, specifically by curbing the generation of content containing “nudity, sexualization, sexually explicit, or otherwise unlawful” elements. The platform was also given a 72-hour deadline to provide a report outlining measures taken to prevent the hosting or distribution of material categorized as “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.”
According to the order, which TechCrunch examined, non-compliance could put X’s “safe harbor” status at risk, potentially removing its legal immunity for content uploaded by users under Indian legislation.
This action by India stems from user complaints showing Grok being used to modify images of people, mainly women, to depict them in bikinis, which led to a formal complaint from Indian parliamentarian Priyanka Chaturvedi. Additionally, recent accounts highlighted cases where the AI chatbot produced sexualized images of minors, a problem X admitted earlier on Friday was due to safeguard failures. These specific images were subsequently removed.
Nevertheless, TechCrunch observed that AI-altered images generated by Grok, which depicted women in bikinis, were still available on X when this article was published.
This recent directive follows a more general advisory issued by the Indian IT ministry on Monday, also seen by TechCrunch, which reminded social media platforms that adherence to local laws concerning obscene and sexually explicit content is essential for maintaining legal immunity from user-generated content liability. The advisory prompted companies to enhance their internal protections and cautioned that non-compliance could lead to legal proceedings under India’s IT and criminal statutes.
The order sternly cautioned, stating, “It is reiterated that non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law, without any further notice.”
The Indian government indicated that failure to comply might result in legal action against X under India’s IT law and criminal codes.
As one of the largest digital markets globally, India is becoming a crucial example of the extent to which governments will hold platforms accountable for AI-generated material. Stricter enforcement in India could have significant implications for international technology firms operating worldwide.
This directive arrives while Musk’s X is still contesting certain elements of India’s content regulation framework in court, contending that the federal government’s content removal powers are excessive, despite the platform largely adhering to blocking orders. Concurrently, Grok has seen increasing use among X users for live fact-checking and commenting on current events, thereby increasing the visibility and political sensitivity of its generated content compared to independent AI tools.
Neither X nor xAI provided an immediate response to inquiries regarding the Indian government’s directive.