The online platform X has sparked major backlash after people used its artificial intelligence assistant Grok to alter pictures of females by removing their clothing.
Several instances have surfaced where the system has been employed to virtually strip female subjects to make them appear in bikinis without consent, as well as to insert them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the firm behind Grok, did not respond to questions, aside from an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to prohibit nudification tools. Under a new law, anyone who distributed such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the potential" of people seeing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give reactions or additional background to other users' comments, but people on X are also can edit an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The chatbot has been condemned for permitting users to generate pictures and clips with explicit content and sexualised content, and it was in the past accused of generating graphic content of celebrities.
"X has been enabling the production and sharing of these images for a long time without taking any action and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "showing likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said services such as X were obligated to take "suitable actions" to "minimize the danger" of users coming across illegal content on their platforms and to take down it quickly when they become aware of it.