The social media network X has sparked widespread outrage after people utilized its artificial intelligence assistant Grok to manipulate pictures of females by removing their clothing.
Multiple instances have been seen where the tool has been employed to virtually strip women to render them in bikinis non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to prohibit image stripping tools. Under a proposed law, anyone who distributed such tools would "receive a jail term and substantial fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium features—which reacts to X users' requests when they tag it in a tweet.
It is commonly used to provide reactions or further context to other posters' comments, but people on X are also able to modify an uploaded photo through its AI image editing function.
Background of Criticism and Rules
The system has been criticised for permitting users to generate pictures and footage with explicit content and sexually suggestive material, and it was in the past accused of generating graphic content of celebrities.
"X has been permitting the creation and sharing of these images for a long time without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service bans "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said platforms such as X were required to take "suitable actions" to "reduce the danger" of users coming across unlawful content on their platforms and to delete it quickly when they are notified of it.
