The social media network X has provoked significant outrage after individuals employed its artificial intelligence assistant Grok to alter images of females by taking off their garments.
Several instances have emerged where the system has been employed to virtually strip female subjects to show them in bikinis non-consensually, as well as to insert them in sexualized situations.
Developer Reaction and Regulatory Stance
XAI, the company responsible for Grok, did not respond to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to outlaw nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their platforms, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some additional features—which responds to X users' prompts when they reference it in a message.
It is often used to give commentary or additional background to other users' remarks, but people on X are also can edit an uploaded image through its AI image editing function.
Background of Controversy and Policy
The tool has been condemned for allowing users to produce images and clips with explicit content and sexualised content, and it was previously accused of generating graphic material of public figures.
"X has been allowing the generation and spread of these images for a long time without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing images of people in a pornographic manner".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the risk" of users viewing unlawful material on their services and to delete it swiftly when they are notified of it.
