Elon Musk's social media platform X has sparked significant outrage after users employed its artificial intelligence assistant Grok to manipulate images of women by taking off their garments.
Several cases have been seen where the system has been employed to virtually strip female subjects to show them in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Legal Stance
XAI, the company behind Grok, offered no comment to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification software. Under a upcoming law, anyone who supplied such tools would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which responds to X users' queries when they mention it in a message.
It is commonly used to provide commentary or more background to other users' remarks, but people on X are also capable of modify an posted photo through its AI image editing function.
History of Controversy and Rules
The tool has been condemned for permitting users to produce images and clips with nudity and sexualised material, and it was in the past alleged to have been involved in making sexually explicit content of public figures.
"The platform has been permitting the production and distribution of these pictures for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "create or distribute private pictures or illegal content" and stated this applied to sexual deepfakes made with AI.
It said services such as X were required to take "appropriate actions" to "lessen the danger" of users coming across unlawful content on their services and to take down it quickly when they learn of it.
