The online network X has caused significant outrage after users used its AI chatbot Grok to change photos of women by removing their attire.
Several examples have surfaced where the technology has been used to remove clothing from women to render them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the company behind Grok, remained silent to requests for comment, aside from an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being introduced to ban image stripping software. Under a upcoming law, anyone who supplied such tools would "receive a jail term and significant penalties".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some additional features—which responds to X users' queries when they mention it in a message.
It is frequently used to provide reactions or additional context to other posters' comments, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The chatbot has been condemned for enabling users to create photos and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been enabling the production and spread of these images for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of people in a graphic way".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and stated this included explicit AI images created with AI.
It said services such as X were mandated to take "suitable actions" to "reduce the danger" of users viewing unlawful material on their services and to take down it quickly when they learn of it.
