The online platform X has caused significant backlash after users used its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Several cases have emerged where the tool has been employed to digitally undress women to render them in bikinis without consent, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the company behind Grok, offered no comment to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to prohibit image stripping technology. Under a new law, anyone who distributed such tools would "face a jail term and significant penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional features—which answers X users' requests when they reference it in a message.
It is commonly used to give commentary or additional background to other posters' remarks, but people on X are also able to modify an posted image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The chatbot has been faulted for permitting users to generate photos and clips with explicit content and sexually suggestive material, and it was previously accused of making sexually explicit material of celebrities.
"The platform has been permitting the production and sharing of these images for a long time without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and stated this included sexual deepfakes made with AI.
It said networks such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering unlawful material on their platforms and to remove it swiftly when they learn of it.
