The United Kingdom’s online safety regulator announced Monday it was formally investigating X over concerns that Grok had been “used to create and share undressed images of people,” including children.
The agency, Ofcom, said it was carrying out an “expedited assessment” and that X had responded to its requests for information on Friday. If Ofcom finds that X violated U.K. law, it can institute penalties ranging from sanctions of 10% of its global revenue, prohibiting British companies from advertising with X, or directing the country’s internet providers to block the platform.
The U.K. investigation is just one of the actions taken by governments around the world in response to Grok’s deepfake abilities.
Over the weekend, regulators in Indonesia and Malaysia announced they had
temporarily blocked X in their countries, citing their laws against deepfakes and Child Sexual Abuse Material (CSAM).
Mary Anne Franks, a George Washington University law professor who studies laws on nonconsensual pornography and has helped both state and federal legislators craft laws on the topic, said she does not expect the U.S. federal government to step in. But threats from other countries with stricter CSAM laws may force X and Musk to rein Grok in, she said.
“If there is some slowdown, I’m sure it’s because of the backlash, and then separately, because these other countries have different definitions of what constitutes illegal CSAM,” Franks said.
Last week, after being prompted by users, Grok began creating thousands of sexualized AI-generated images every hour using the faces of real people and posting them on X, according to an analysis conducted by deepfake researcher Genevieve Oh. Many of the deepfakes depicted identifiable women who were placed in bathing suits or revealing poses by artificial intelligence; but Grok also depicted some people, including some children, according to some watchdogs, as fully nude.
A spokesperson for the European Commission, the executive body of the European Union, told NBC News that the agency has recently observed Grok generating sexualized images of children.
On Friday, Australia’s eSafety Commissioner, a government agency dedicated to online safety, said it had received several reports of Grok creating exploitative images and had asked X to clarify its safeguards, noting that it could issue removal notices to X. Both Australia’s Prime Minister, Anthony Albanese, and the opposition leader, Sussan Ley, condemned the images on Saturday.
Musk and X did not immediately respond to a request for comment. Musk and X recently posted that they prohibit users from making “illegal content,” but have refrained from publicly indicating that they would substantially change Grok’s ability to manipulate and sexualize images of people. The creation and possession of CSAM are widely illegal around the world, but many countries also have laws restricting or prohibiting the creation of sexualized images of adult people without their consent, known as nonconsensual pornography.
Early Friday morning, X made the Grok reply bot, which was driving the deepfake trend, a premium feature for paying subscribers, though users could still produce the deepfakes using the Grok stand-alone app in addition to other interfaces where Grok was available. While the number of such images posted to X appears to have slowed, critics like Sen. Ron Wyden, D-Ore., have said the move is tantamount to making “some of its users pay for the privilege of producing horrific images on the X app.”
Ofcom said its investigation would focus on whether X has been following U.K. law by quickly taking down illegal content as it became aware of it, and whether it was carrying out successful risk assessments before making “significant changes to their service.”
On Thursday, British Prime Minister Keir Starmer said on a radio interview that Ofcom “has our full support to take action.”
“This is disgraceful. It’s disgusting. And it’s not to be tolerated,” he said of the image creation. “X has got to get a grip of this.”
India’s Ministry of Electronics and Information Technology, MeitY, also reportedly warned X last week that it was potentially violating its laws. MeitY didn’t respond to a request for comment.
It remains unclear if any U.S. regulators will take on Musk and his companies over the manipulated images. The Federal Trade Commission did not respond to a request for comment. The Justice Department told NBC News last week that it was committed to holding “individuals who exploit technology to harm our most vulnerable” accountable, but didn’t mention specific companies.
A number of state attorneys general have declined to comment to NBC News on whether they are considering charges over the images.
Musk has broadly pushed for X and his AI company that powers Grok, xAI, to be more lax on content moderation than most of their competitors. On Friday, he posted on X that reports of the U.K.’s Labour Party wanting to potentially block X over the Grok images were evidence that “They want any excuse for censorship.”
Thomas Regnier, a spokesperson for the European Commission, said in a press conference Monday that it has given Musk and X an ultimatum to stop the production of CSAM, but stopped short of mentioning any specific penalties.
“Drawing a parallel between freedom of speech and an AI tool that generates Child Sexual Abuse material is dangerous nonsense, especially when it comes from the owner of a tech company. Frankly speaking I cannot even believe we are speaking about this and engaging with this from the commission’s podium in 2026,” Regnier said.
In December, the European Commission fined X $140 million, saying that the platform broke Europe’s laws against hate speech and misinformation. That prompted criticism from the Trump administration, with both Vice President JD Vance and Secretary of State Marco Rubio calling it an attack on American free speech.
