Elon Musk's social media network X has sparked widespread anger after users employed its artificial intelligence assistant Grok to alter photos of women by removing their attire.
Several cases have been seen where the system has been used to remove clothing from female subjects to render them in swimwear without permission, as well as to insert them in sexually explicit situations.
Company Response and Legal Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to outlaw nudification software. Under a new law, anyone who provided such tools would "face a prison sentence and significant fines".
"The platform is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some paid-for features—which answers X users' queries when they mention it in a message.
It is often used to give commentary or additional background to other users' comments, but people on X are also able to edit an uploaded picture through its AI image editing function.
History of Criticism and Policy
The system has been criticised for enabling users to generate photos and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making graphic content of public figures.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate measures" to "minimize the danger" of users encountering unlawful material on their services and to remove it promptly when they become aware of it.
