The social media network X has sparked significant anger after people utilized its AI chatbot Grok to manipulate photos of women by taking off their attire.
Multiple instances have emerged where the technology has been used to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to inquiries, aside from an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to prohibit nudification tools. Under a upcoming law, anyone who supplied such technology would "face a jail term and significant fines".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing illegal content on their services, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' queries when they tag it in a message.
It is often used to provide reactions or more context to other users' remarks, but people on X are also can edit an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been criticised for enabling users to generate images and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in generating graphic content of celebrities.
"The platform has been allowing the generation and distribution of these images for months without taking any steps and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting images of individuals in a graphic way".
In a statement, Ofcom said it was illegal to "make or share private pictures or illegal content" and confirmed this covered explicit AI images made with AI.
It said platforms such as X were obligated to take "necessary measures" to "minimize the risk" of users viewing unlawful content on their platforms and to take down it swiftly when they are notified of it.
