The online platform X has sparked major backlash after people utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Several cases have been seen where the system has been used to virtually strip women to render them in swimwear without permission, as well as to place them in inappropriate scenarios.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to requests for comment, except for an standard message stating "legacy media lies".
A government representative stated that legislation is being introduced to ban image stripping tools. Under a new criminal offence, anyone who distributed such technology would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "assess the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was currently examining X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium features—which responds to X users' queries when they reference it in a tweet.
It is often used to give commentary or more context to other posters' remarks, but people on X are also capable of alter an posted picture through its AI image editing function.
History of Controversy and Rules
The system has been condemned for allowing users to produce images and footage with nudity and sexualised content, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"The platform has been permitting the generation and sharing of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "depicting likenesses of people in a graphic way".
In a statement, Ofcom said it was illegal to "create or circulate private pictures or illegal content" and stated this included explicit AI images made with AI.
It said services such as X were obligated to take "suitable actions" to "lessen the danger" of users viewing unlawful material on their services and to take down it quickly when they are notified of it.
