The online network X has caused major outrage after users employed its AI chatbot Grok to alter images of females by taking off their clothing.
Numerous cases have been seen where the technology has been employed to remove clothing from women to show them in swimwear without permission, as well as to insert them in sexualized contexts.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, remained silent to questions, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to outlaw image stripping tools. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and significant penalties".
"The platform could prevent these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the danger" of people viewing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which reacts to X users' prompts when they reference it in a post.
It is frequently used to give reactions or further background to other users' comments, but people on X are also able to edit an uploaded image through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to produce images and videos with explicit content and sexually suggestive content, and it was previously accused of making graphic content of public figures.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any measures and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable steps" to "lessen the risk" of users viewing illegal content on their platforms and to take down it promptly when they are notified of it.
