Social media platform X has accepted lapses in its content moderation, blocked 3,500 posts and deleted more than 600 accounts after the Union government directed it to remove sexually explicit content generated by its artificial intelligence chatbot Grok, PTI quoted unidentified officials as saying.
The social media platform reportedly told the government that it will comply with Indian laws and that it will not allow explicit imagery on the platform, ANI reported.
On Friday, the platform limited Grok’s image-generation feature to paid subscribers, which effectively meant the names and payment information of those using the feature would be on file.
Grok had earlier allowed requests by users to digitally manipulate photos of real persons – mostly women – by undressing them and sexualising their images without their consent. It had been creating thousands of such images every hour, Bloomberg reported.
Even after the feature was limited on X, the standalone Grok app was still allowing users to generate such images without subscribing, according to the social media platform.
On January 2, the Union Ministry of Electronics and Information Technology told X to “undertake a comprehensive technical, procedural and governance-level review” of Grok to ensure that it does not generate content that contains nudity or sexually explicit material.
The Centre warned the social media platform that not complying with the directive could lead to action against its officials under the Information Technology Act, the Bharatiya Nyaya Sanhita and the Bharatiya Nagarik Suraksha Sanhita.
In the letter to X’s chief compliance officer in India, the Union government said that users were misusing Grok to create fake accounts to generate and share obscene photos and videos of women with the intent of denigrating them.
“Importantly, this is not limited to creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs,” the ministry said.
Hosting or publishing obscene and sexually explicit content, including through AI-enabled tools, is invasive of bodily privacy and attracts serious penal consequences, the Centre said.
The government also directed X to submit an action taken report within 72 hours, detailing the measures undertaken to oversee the Grok application, the oversight exercised by the chief compliance officer, the action taken against offending content and the mechanisms put in place to comply with mandatory reporting requirements.
Amid criticism, X owner Elon Musk said on January 3 that “anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content”.
You’ve read Scroll.
Now help sustain it
Scroll is funded by readers, not corporate owners. If you believe our work matters, support our newsroom. Become a member today!
We’re not driven by clicks or corporate interests – just honest, independent reporting. Keep us going. Support Scroll today!