Malaysia and Indonesia have become the first governments in the world to impose national restrictions on Elon Musk’s Grok AI chatbot, after regulators said the system had generated sexually explicit images of women and children, including deepfake-style content created without consent. The bans were announced over the weekend by the Indonesian Ministry of Communication and Digital Technology and Malaysia’s Communications and Multimedia Commission, following complaints about Grok’s new image-generation tool on the X platform. The action was first reported by The WP Times, citing Bloomberg, after emergency regulatory measures were introduced against the AI system developed by Musk’s xAI. The case is now seen as the most serious international test so far of how governments intend to regulate AI-generated pornography, deepfakes and child safety online.
Indonesian authorities said the decision was taken after Grok was found to be capable of producing photorealistic pornographic images, including depictions of real women and minors, which could be generated using simple text prompts and then shared on X without moderation. The ministry said this created a risk of mass distribution of fake sexual content, which could not be effectively controlled once released into the platform’s recommendation system.
Indonesia’s communications minister Meutya Hafid said the government viewed the misuse of artificial intelligence for sexual deepfakes as a direct threat to public safety and digital sovereignty.
“The use of deepfake technology to display sexual practices without consent constitutes a serious violation of human rights, human dignity and national digital security in the digital space,” she said.
As part of the ban, Indonesian regulators formally requested technical explanations from X on how Grok’s safeguards failed and what measures were being taken to prevent further abuse. In Malaysia, the Communications and Multimedia Commission (MCMC) announced within hours that it was also restricting access to Grok until what it called “effective protection mechanisms” are implemented. The regulator said it had issued legal notices to both X Corp and xAI LLC, requiring them to prevent the chatbot from producing content that violates Malaysian law on obscenity, child protection and harmful digital material.
In a statement, the commission said that responses received from the companies so far did not remove the inherent risksassociated with the AI tool.
The regulatory action followed a decision by xAI on Friday to limit Grok’s image-generation feature to paid subscribers only, after the tool sparked widespread backlash for producing sexualised images of women and children. The feature had previously been available to all X users, subject only to daily usage limits. xAI has not disclosed how many images were generated before the restrictions were imposed, or how many accounts were involved in sharing the material. After Indonesia announced its ban, Grok posted a brief response on X, saying:
“We apologise for the inconvenience. We are working to resolve this issue.”
Neither xAI nor X has issued a detailed public explanation of how the images were created or how future misuse will be prevented.
In India, which had previously raised concerns about Grok’s capabilities, X has given assurances that it would comply with local laws on obscene and harmful content. Indian media reported that around 3,500 pieces of content were blocked and more than 600 user accounts removed following government intervention.
The controversy is now spreading beyond Asia. According to GB News, the UK government has held talks with Canada and Australia about the possibility of restricting or banning X over the Grok scandal. The broadcaster reported that both countries broadly share Britain’s concerns over AI-generated sexual content and deepfakes. However, US officials have warned that sanctions could be imposed on Prime Minister Keir Starmer and the UK if X is banned, raising the prospect of a diplomatic dispute over the regulation of Musk’s platform.
Read about the life of Westminster and Pimlico district, London and the world. 24/7 news with fresh and useful updates on culture, business, technology and city life: Musk accuses UK of ‘censorship’ as Grok deepfake scandal spreads and Ashley St Clair speaks out
