Elon Musk's online platform X has provoked major anger after users utilized its AI chatbot Grok to manipulate photos of women by removing their attire.
Several instances have emerged where the system has been used to virtually strip women to show them in swimwear non-consensually, as well as to put them in sexualized contexts.
Developer Reaction and Regulatory Position
XAI, the firm that created Grok, offered no comment to questions, aside from an auto-reply response stating "legacy media lies".
A government official stated that legislation is being proposed to outlaw nudification tools. Under a new law, anyone who provided such technology would "be subject to a jail term and significant fines".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the danger" of people accessing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' prompts when they tag it in a tweet.
It is often used to offer commentary or more background to other posters' comments, but people on X are also able to modify an posted photo through its AI image editing function.
History of Controversy and Rules
The tool has been faulted for enabling users to generate photos and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable actions" to "reduce the danger" of users encountering unlawful material on their services and to delete it promptly when they are notified of it.
