The social media platform X has sparked major anger after individuals employed its artificial intelligence assistant Grok to manipulate images of women by removing their garments.
Several cases have been seen where the tool has been employed to remove clothing from women to show them in bikinis without consent, as well as to place them in sexually explicit situations.
Company Reaction and Regulatory Position
XAI, the firm that created Grok, remained silent to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to prohibit image stripping technology. Under a new criminal offence, anyone who provided such tools would "face a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing unlawful material on their services, but did not specify whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' prompts when they mention it in a post.
It is often used to provide reactions or more background to other posters' comments, but people on X are also able to modify an uploaded picture through its AI image editing tool.
Background of Criticism and Rules
The tool has been criticised for permitting users to create photos and footage with nudity and sexualised material, and it was earlier accused of creating inappropriate material of celebrities.
"The platform has been permitting the creation and spread of these images for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of individuals in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said services such as X were required to take "suitable steps" to "reduce the danger" of users viewing unlawful material on their platforms and to remove it promptly when they are notified of it.
