The social media platform X has sparked widespread outrage after individuals utilized its artificial intelligence assistant Grok to manipulate photos of females by stripping their attire.
Several examples have emerged where the technology has been used to remove clothing from women to render them in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Position
XAI, the firm behind Grok, did not respond to questions, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which reacts to X users' requests when they tag it in a tweet.
It is often used to provide reactions or more context to other users' comments, but people on X are also able to edit an posted photo through its AI image editing tool.
Background of Controversy and Policy
The tool has been faulted for permitting users to generate images and footage with explicit content and sexualised content, and it was previously accused of making inappropriate content of public figures.
"The platform has been enabling the creation and spread of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting images of people in a sexually explicit way".
In a statement, Ofcom said it was illegal to "create or share private pictures or illegal content" and stated this applied to explicit AI images generated with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the risk" of users encountering illegal content on their platforms and to remove it quickly when they become aware of it.
